 So I'm so excited to be here speaking at a developer conference because a lot of times lately when I give talks to business people, you guys are kind of my people because I am a developer. I got my start with startups, which is kind of what I know and love. I have also done development for some big companies like Oracle and Microsoft and helping them with how to be more lean and agile. But really what gets me excited is working with people who are really pushing the edge on what's possible. And so for the past few years I've managed to work with literally hundreds of early stage tech startups and just really focused on helping hackers and entrepreneurs who are building really innovative disruptive products with how to be successful at market. So that's what I'm going to share with you guys today, whether you're building a new product for a big company that wants to enter a new space or whether you've got your big idea for the next great thing and want to go do your own startup. So in thinking about innovation, the first rule of innovation is there are new rules of innovation because if you're doing it right, then you're breaking more rules than you're following. And if you're going to be innovating, then you're going to be trying a lot of different things. Thomas Edison famously said, I have not failed. I've just found 10,000 ways that won't work. But don't worry, it's not a complete black hole. What I wanted to do today is share some patterns that I've seen innovators who are successful do time and time again and kind of walk through these and give you some examples of how innovators have done them. So the first one is eliminate waste, which if anybody does anything with lean, this is not going to be anything new to you. There's actually this lean startup methodology which has tried to take the principles of lean and apply them to how to do a startup. And Eric Reese, who's the creator of the lean startup methodology, says the number one most important thing for a startup is learning to tell the difference between value and waste, which is kind of weird, right? Why would you not know the difference between what adds value and what is wasteful? But of course, it actually comes from lean from the Japanese term muda, which doesn't just mean something that's never going to add value, but eliminating anything that isn't explicitly adding value today. And when you're innovating, think of Thomas Edison, you're going to be trying a lot of things and finding out what doesn't work. And if you are trying to do this stuff that you think you're going to need tomorrow, you don't know what tomorrow is going to hold yet. And so if you wind up going over here and you build all this stuff over here, then that is wasteful. Or for any XP, Extreme Programming folks in the room, just think, Yagney, you ain't going to need it. So the truth is a lot of companies think that they're going to do one thing, and then they wind up doing something very different. So to give some examples, PayPal started off as a way to beam payments between PDAs, but it turned out that the world was not yet ready for mobile payments in 1999. And Flickr actually started as a massively multiplayer online role-playing game. They actually built out the game, and then it turned out that the most fun feature of the game was sharing photos. And Instagram started as a gamified Foursquare. And again, they built out the entire iPhone app. I don't even know if they put it in front of anybody though. They just looked at it and said, oh my god, there's too much shit going on here. They threw the entire thing out and just focused on photos, which they thought was going to be the most effective. So these are examples of companies that are successful today. But when you look at the stats, it's pretty bleak. The stats are that nine out of 10 new products fail, which is pretty abysmal. And specifically, when you look at tech products, they tend to fail not because the technology doesn't work, but because people have created something that there isn't a real market for. A great example of this, or a scary example of this, depending on how you want to look at it, was this company Actuality Systems. They were actually here in Boston, so some of you guys might have heard of them. They created a holographic 3D display. OK, that's pretty badass, right? They actually created it. They got it working. And then they spent the next 10 years trying to find a customer for it that would let them have a sustainable business. And in the end, they finally had to shut down. And all that they could do was really sell off the license for the technology. And so the question is, you know, were they successful? I mean, in innovation, they created this thing that had never been created before that was absolutely amazing. And so from that perspective, absolutely yes. But then they weren't able to actually make a company out of it, and they spent 10 years of their life trying and not succeeding. And so I'd like to see technologies like this do a little bit better. So it's kind of interesting. There was a survey done into what is the single biggest predictor of failure amongst startups? And what was found sticking to the initial business plan? So this is kind of scary, right? Because when you're trying, when you're doing any new venture and you're trying to figure out if you're on track or not, even that word on track is like, how are you tracking to plan? And so if staying on plan actually means you're going to fail, it gets kind of confusing, right? And so that brings us to pattern number two, which is to start small, which is usually the exact opposite of what I see people do when they're all excited about they've got this next big thing they want to go off and build. And so hackers are like, I want to build this end-all device that's going to do everything, right? And entrepreneurs say, I'm going to build the next Facebook and get a million users and make a billion gazillion dollars, right? Go big or go home, baby. But the question is, how do you go from this big vision that you've got? We have nothing else, but your big vision to a million billion dollars or users or devices or whatever it is. I mean, Facebook today has over a billion users. How would you even create something that from day one would have enough features to be appealing to a billion different users? And the other thing is that would you guys want to get on the next Facebook if nobody else you knew was using it? Most people want to wait and see, right? And so when I think about kind of this quest for how do we come up with an innovative product that's going to be successful, I think of it as a search for the intersection between our big vision of what we want to do and what reality can actually accommodate today. So a lot of companies wind up starting small in order to do some experiments. A great example, Microsoft, not doing so great with their phone, but I think everyone would agree they're pretty big doing OK. They actually started with basic for the Altair. Now, I don't know how many Altair computers were ever made, but I'm thinking like a few thousand. It wasn't very big, right? Or even Facebook, which is sort of our quintessential example of going big. They started at Harvard, and there's only 20,000 students at Harvard. And so it feels very unintuitive, but this is the way that you want to be thinking about things. Start from your big vision. You want to have that big vision, that's great, but then go small. Figure out a way to dominate a small market, and then you can build on that success to go bigger. There's several reasons that you want to do this. One is, of course, what we already said, that if you're innovating, you're going to find 10,000 ways that won't work. And so you don't want to go big and spend a year or 10 years building something out, and then it doesn't work, and you're like, oh, OK, that was all the time and money and patience that I have for that. So small experiments are good. But even beyond that, by going small, you can go after this very niche market. And all you have to do then is find these people that really want something bad, and find that one thing that they're dying for, and then just build that one thing. You don't need to build a billion features to be exciting to them. So Altair users really wanted a way to program their computers. I mean, I think all they had were, like, blink and lights and toggle switches, right? And then Harvard students really wanted a single centralized student directory. So Facebook was able to build that out without building all these features that they have today. So this is how you want to be thinking about things, like Microsoft, like Facebook. Like, actually, most successful companies actually start by going small with a very small market and taking that to billion gazillion. So of course, in order to be able to dominate the small market and understand that one thing that they're dying to have, you really need to be able to deeply understand your customers. And I would even go so far as to say, if you're entering a space where there's, like, competitors as there almost always tends to be, this is the number one way to dominate that space. Because if there's a bunch of different people entering the same space at the same time, he who best understands his customers is going to be able to create the solution that best resonates with them and likely going to win. A really great example of this is Dropbox. This is a bit of a conversation that Drew Hauston had with some VCs when he was starting Dropbox. And they said, I don't even understand why you're entering a space. There's already a million, billion cloud storage startups out there. And Drew said, yeah, but do you use any of them? So what Drew did that all the other competitors that were trying to enter a space didn't, is he actually understood who his customer was. I think because he was one of them, and he built it specifically for them. So he understood that his customer was, or at least his initial customer, that small market, was the hardcore techies who have a bunch of devices that have the problem that you need to transfer between them. He understood what they wanted so well that he put up this web page, very exciting web page here. You can see this was built in 2007. It looks a little bit like 1990s web page, right? He actually, he built out the tech. He did a proof of concept on the technology because he was doing tech that hadn't existed before. He had to make sure he could get it working. But once he figured that out, he didn't go build the full product. He put together this mock-up prototype. And he did this really scrappy three-minute video if anyone has seen this, you can still find it on the net. Three-minute screencast of him walking through this mock-up with him as a voiceover. Nothing flashy, nothing exciting. He posted it to Hacker News, so one place. And overnight, he got 75,000 sign-ups, right? Now you think, OK, well, today, if Dropbox put out something and got 75,000 sign-ups, that would be kind of impressive. But they've got like 300 million users. They can do that. When he put this out, nobody knew what Dropbox was because it didn't exist yet. And he put it one place because he knew exactly who his market was, and overnight, 75,000 sign-ups. So it's pretty astounding. So I feel like there is this myth around innovation, especially in the last couple of years with so much hype around startups. Everybody wants to do a startup. And there's these stories that get told that make it look like if we build it, they will come. But when you really dig deep into what's going on, what you find are these stories of these founders, the founders who are really successful, have actually gone to extraordinary lengths to really understand their customers. So Airbnb, I guess, having some legal problems lately, but still doing pretty good. One of their co-founders did not rent or own his own home. He just went around and stayed in Airbnbs. Like, I can't even imagine. I guess he lived out of a suitcase. But how's that for understanding your market? Or Ben Silverman, the founder of Pinterest. He actually personally reached out to the first 5,000 users and invited them to coffee and gave him his cell phone number. Like, can you even imagine? And so the thing that you want to be thinking about as you're going off and you're talking to all these people and you're proving at your technology is that you always need to be learning. In fact, a lot of this comes from sort of lean startup if people are familiar with that. But they say for innovation, what you want to be thinking of is how can you be scientific about this? So I'm not a scientist. I don't know a lot about science, what they do. But as I understand, when they want to prove something, they come up with a hypothesis and then they do a test to either validate or invalidate the hypotheses. And when you are trying to innovate, when you're trying to come up with this new product that you want to be successful in the market, you have nothing but a bunch of guesses or hypotheses if you want to be fancy about it. And so the question is what can you do to test if your guesses are right or not without going and building the entire product and then possibly failing, right? What are these small tests that you can be doing to be constantly learning what works and what doesn't? So initially, you can do some of these tests by talking to people. You know, the holographic display, I think they would have done really well if they just, I assume that their customers were commercial businesses, if they'd just gone out and talked to people in different industries and understood better, you know, what is your budget? What are some problems that you have today that maybe could be dealt with with a 3D display? Drew Houston from Dropbox could have, and I'm sure that he did go out and talk to people and say, okay, what devices are you using today and what are you doing today to transfer files between those devices? And that's really good information to have because, you know, carrying around a USB stick sucks, right? And so all you need to do is come up with a solution that's better than that. It doesn't have to be super elegant. It just has to be better than what people are doing today. But here's the thing and this is a mistake I see a lot of people make is talk to people absolutely to understand sort of what problems they're having and what they've done in the past but don't say, hey, I've got this great idea, isn't it a great idea? Because people get caught up in your excitement and your passion and so, of course, they're going to say, yes, that's a fantastic idea. Oh my God, you're building a holographic display. Yes, I want one and you say, would you buy it? And they want to encourage you and they say yes and then you hand them a device and I mean, I don't know the details but say it's $50,000 and you have to actually write this real-time 3D rendering code to get anything to output to it. And they're like, oh, are you serious about that? So you don't want to, your validation for if people will use a product or if they'll buy it, should not be asking them. It should be actually putting something in front of them. So I like this quote from Linus Travales who says, talk is cheap, show me the code. In startup world, we have this phrase which I kind of love and hate because I feel like it's overused like a buzzword. It's called minimum viable product or MVP for short. And the idea here is don't build out the entire product but figure out what that minimum viable thing is that you can get in front of that user to actually view what their behavior is. And usually you want to put like a lot of analytics on that or something so that you can actually see what they do with it. So we've already seen one example of it, kind of the quintessential MVP is no code required at all is something called a landing page. That's what Dropbox did. So you put up a description of your product or maybe a video on it and then a place for people to enter their email address. And you put some analytics on it so you can see how many people you're able to get to the site and how many people are signing up. To give you kind of walk through a little bit more elaborate of an example of this, there's a company called Buffer. They do social media sharing. And so say you like to tweet a lot and you go and you like, I like to read all my news at like 2 a.m. or something on Sunday night because I'm procrastinating, I don't like to go to bed. And so I'll find these 10 awesome stories to share and I can tweet them all out right now but like nobody but me is up. And the people who are up are probably like, why are you spamming me right now, Abby, with all of this stuff. So I love them because I can put them in this queue and they'll kind of buffer or post them out over the week, which is really nice. So this is how they look today. When the founder just had an idea for whether this would work or not, he put up a landing page. It was a simple concept, they didn't need a video, they just described what the initial version would do, which would be just for Twitter. If people are interested, they can click the plans and pricing button and when they click it, you get a hello, you caught us before we're ready but leave us your email address. Tweeted that out, people clicked through, they left their email address, he said okay, that's pretty good validation but not enough for me to go build it because I wanna know if I can actually make a sustainable viable business out of this. So I wanna know if people will actually pay for it. So all he did was add a middle page to his landing page. When they clicked plans and pricing, they got three plans, one was free but two were paid. People kept clicking through, most people clicked free but a couple clicked paid and he said, you know what, this is good enough not for me to quit my day job and spend six months heads down building it but good enough for me to put out that first really minimalistic version and see if people actually use it. And so that's what he did and very quickly, like it took him a week and a half I think to get the initial version, it was very minimal but very quickly he had 500 users using it, he even had some revenue coming in so now he was able to actually evolve the product along with seeing how users were actually using it. Another obviously great example, especially if you're doing any physical products like hardware, smartwatches are all the rage these days is to do a crowdfunding campaign like Kickstarter and actually get people to pay for it before you go manufacture it. Now again, whenever you're doing something that's technologically new you want to make sure that you can actually get the tech to work but you don't need to build out the final end-all piece, you can do a prototype, make sure it's working and then if there is enough interest because you get these pre-orders then you actually have the money to go manufacture it. So as we like to say in startup world these are all great ideas but ideas are kind of a dime a dozen it's all about execution. So, how do you focus and get shit done? So I want to spend the second half of this talk talking about what are some development practices that we can use to make sure that we're focusing on the right thing. So make sure that we're focusing on things that add value not waste and allow us to move very quickly while continually learning and adapting. So the first thing is just get into this mindset and this is really painful but of minimum viable everything. Like you're really excited about the idea you want to go build this huge thing but stop and wait in what is the minimum thing that you can do to kind of get to the next stage. So we've already seen examples of this if my assumption or hypothesis is if I build this product people will want it, great, do a landing page. That's a great way to test it. If your assumption is if I sell this product people will buy it you can add a pricing to your landing page or you can do a Kickstarter for it. Now if you want to build the product I think the next thing you're probably assuming that you want to test out is okay I have identified who my customer is and what that one thing is they're dying for and so if I build this particular solution I'm thinking of that people will use it. So here what you want to do is think about like what Buffer did and think about okay what's kind of the minimal the absolute bare bones minimal feature set that I can put out and they kind of say if you're not embarrassed by your first release then you waited too long to put it out there. So I don't actually know all the details but I took some guesses at what Buffer's initial minimum feature set might have been. Obviously people need to be able to create a user account link it to their social media networks add posts to the Buffer and then edit and delete them add like what your posting schedule is when you want the tweets to go out. Buffer needs to then automatically be able to post from your Buffer to the schedule and then just let you view a history of the post. So that felt pretty minimal. The thing that you always want to kind of do is go look at okay I think this is my best absolute minimal thing. Ask yourself one more time like what is it that I'm trying to prove and do I really need all of these things for it? And so again I'm making guesses but I think that what Buffer initially wanted to see is okay people will download the app but there's like a big issue with people download apps all the time and never use them will they actually post something into the Buffer? And if that's what they're trying to prove then some of the stuff can go away at least initially. So if you haven't had anybody post the Buffer yet you don't need to allow them to view a history of their posts or even edit or delete their posts and that's painful but if you see people are using it then you can kind of very quickly add some of these simple features. Similarly if you're just trying to address the issue that they don't want to post all of their tweets at 2 a.m. on Sunday you don't necessarily need to let them set up their own schedule you could hard code something in and say this is based on when we found or the most popular times for people to look at Twitter people are gonna be happy with that. Again you're just trying to see if they're gonna post or not and then of course you don't really need to add Facebook and LinkedIn just start with one whichever one you think is gonna be most important see if that works or not. So that's sort of how the process looks now we've gotten down from what we thought was the absolute minimal to only four of those items and you can get that out pretty fast and see if people are gonna be using it. So okay you know what features you wanna build in now now what is kind of your minimum viable software development process to use for this. I'm kind of an agile girl that's my background of I remember when Lean Startup first came out I was asked to present at a scrum club on about Lean Startup and I was really excited about it and I was at actually a Lean Startup meetup and I was bragging about it I was like oh I'm so awesome you know I'm gonna go present about Lean Startup to scrum club and one of the organizers of the Lean Startup meetup looks at me like horrified and goes scrum oh my God I hate scrum it's so heavyweight and I felt like a really big dummy. And I was really embarrassed and I've kind of but it was a big learning moment for me and I've noticed that many of my big learning moments have come when someone's kind of screamed at me and made me really embarrassed. I don't know maybe that's how I learned things but I realized who was really right I mean when you're innovating on something you are going at such a different pace and at such a different scale than we're used to in software development that scrum just isn't even built for this it doesn't even make sense for this and if you try to do something like scrum it's gonna slow you down and I don't necessarily mean all aspects of scrum I think there's still a lot of good theories from this but in particular the time boxes that scrum sets up. You know in scrum you have these sprints they're one week or they're two weeks and it's really arbitrary and honestly two weeks can be like a lifetime in the beginning it's way too long. So I just wanna see if anyone's interested I just wanna put up a landing page that takes me you know there are tools that you can do this without even coding you can do it in like an hour right? I just wanna do it and put it up there in an hour why would I wait two weeks or even a week for the sprint to end it doesn't make sense. And then to be constantly looking at how are users responding and be able to adapt to that immediately. And so I put up the landing page and I'm looking at the stats and I noticed that people are coming to it but no one's leaving their email address so I say okay I should try put a different message on the page maybe that'll get more email address signups. Again I can just do that in 10 minutes I don't need to wait for a time box. Even when I get to the point where okay I figured out that first version of the product that's those four features and I'm gonna build them will say it takes me two days or a week and a half whatever it takes me once it's done I should just be able to post that up it doesn't make any sense to put it in a time box. Time boxes make sense when you're kind of building towards this actual list of known features they don't make sense with when you're at this stage and you're trying to figure out you know there's 10,000 things that don't work. So what I like instead is Kanban. So you can start with sort of a simple task board I'm sure you guys have all seen this before you know what's our backlog what are we working on what do we have what have we finished. A problem that I see when startups just do this is that they're in progress. Column starts to look like that very quickly which is particularly bad when you realize there's like one person working on it. And it's just it's tough because you have especially if you're like trying to actually make a business out of this even if it's just you're doing it for yourself you're excited you have all these things that you want to be able to build into it and then you talk to people and they have ideas you talk to investors and they're like if you build in X, Y and Z it'll totally be fundable and you're like oh my God we better stop everything and do that and if you talk to customers or potential customers you're like oh if you just built these features and we would buy it and they're like oh my God money I should build in these and pretty soon you have something that looks like this and nothing is getting done. And so what we're missing from Scrum that was really nice was Scrum had these nice sprints where you'd actually have to finish everything in the sprint before you can then go work on the next items from the backlog and that was really nice so we want to still have that constraint in place but we don't want it time based we want it more task based based on our capacity so if there's just one of you or maybe there's just one developer on it you know you put a work in progress limit on that column of one meaning everything else has to wait. If there's two of you then the work in progress limit would be two. So this way you make sure that you're actually completing those things before you can go work on anything else. One thing that might strike you here is obviously I just talked about how you have to be able to move really fast and be really flexible if that one item is like a two week or three month or whatever long item you obviously can't wait that long to do anything else. So I would keep these really short you know it's a little different for everybody but I usually recommend that places start off with one day keeping the tasks less than or equal to one day. The beautiful thing about that is then every single day something completed and it has this momentum that's going forward and usually no matter what's going on it can wait until the next day. You know to pull in the next thing. And while you're working on that the to do unlike Scrum this to do backlog can be moved around it can be changed priority can order can be changed to whatever you want it just matters. Once you start working on it it just needs to stay the same until it's done. But I'm looking at that to do column and that still looks pretty overwhelming to me right. I mean would you guys be like oh my God there's so much to do. And I feel like when you get to do columns this long first off if you're really early and you're still trying to like figure out what the right features are for the customer how do you even know what all that stuff is that's like a bunch of yeses right. And you should be just getting little things out in front of the customer to kind of figure out what the next thing is. So first off a lot of that could possibly change. Second off like if I'm the developer and I'm working on A and I see all these different things I'm like oh my God when I'm doing A I need to think about how B and C and D and E and F and G and blah are gonna like work into this and you get overwhelmed and it slows you down. And so I say you know if you want to write all these ideas down great put them in a spreadsheet put them in an notebook or whatever but put a limit on what you can put in that to do column as well. And I would put you know at a maximum how much you can do in like a week or two. Get rid of everything else that's a maximum not a minimum so it can be less. And then you are hyper focused on just what are we trying to do this week right. And it's helpful if it's just you it's helpful if you have a team of people everybody's kind of focusing on the same thing. So okay now you're gonna go start developing it you want to think about what's our minimum viable design which of course we would really call emergent design. I actually for anyone who has been here for Katrina has talked very nicely about how to refactor which is great. You know this is another thing if you keep thinking yagney you ain't gonna need it you don't know what's gonna come tomorrow so don't try to build out this architecture that's gonna scale to a million users when you have zero users. The truth is a lot of startups build a product totally wrong they throw it out they build a product, throw it out build a product, throw it out. So you don't need this built up architecture you're just wasting time and that's sort of kind of what Eric Green was talking about when he thinks about the difference between value and waste. Similarly you don't need to go design for we like to follow all these principles like okay I'm gonna hook Buffer up to Twitter but I know other social networks are coming in so I better do this abstraction layer so that it can then easily be replaced with Facebook or LinkedIn but in order to do that I kind of have to research a little bit about the different social networks we might use and the differences between them and how you interact with them and then I have to figure out the abstraction layer which is really hard to do until I built one and oh my god just do it for Twitter and then if later on you determine okay we're gonna need to add another social network again great refactor it figure out what the best design is for it at that point just keep thinking Yagney like very aggressively much more than you might on a normal project. So okay you're developing the code you're moving really fast I know that this looks a little bit water folly here but these are kind of generally the steps that you're gonna be doing when you're doing development and if we think about okay if a big success factor or a big indicator of success on whether or not our products are gonna be accepted in market or not is understanding whether the customer is gonna like them or not then in kind of learning that then we wanna look at this and say okay where is our learning occurring and so you know there's some learning occurs when we do requirements because hopefully we've gone out and we've talked to customers we kind of deeply understand what their problem is and we're trying to come up with something to solve for that but really most of the learning about our product happens when you release something put it in their hands and see how they actually use it and unfortunately the place where we spend the majority of our time and as developers is often the most fun there's very little learning that happens here and so the question is how do we maximize our chances for success how can we maximize this learning that's going on? So another way to look at that is to say kind of how do we shorten the time frame between when learning occurs and so instead of doing a whole bunch of features and batching them up and putting them out you know a great thing to be doing is continuous deployment I put a little smiley face because I'm very happy that there's like a whole continuous deployment track here today so I won't go into a whole bunch of details on that but I did wanna kind of talk about how some startups are using it. So Etsy is fantastic if anybody's doing continuous deployment they've got a terrific blog that talks about what they're doing they're currently deploying 50 times a day actually I got that set about a year ago so it might even be more now but I know in 2011 they deployed 10,000 times to production amongst 100 engineers which is pretty extraordinary and what they said is ever since we started deploying so many times ever since we started deploying 10,000 times our system is so much more stable because we've already done this 9,999 times like we got this and also when you think about you wanna be in this mode of a scientist where you're doing experiments Etsy also said this allows them to experiment so much more so before when they were only releasing once every few weeks or maybe once a month they were constantly having to pare down what actually went out because they were frantically trying to make the release. Now once they started doing over 30 times a day or something crazy they're like this allows us to do things like throw a feature out there in a minimum version see if people actually use it or not if they do we can either pull it back and kind of build it out or we can just quickly add features to it it just gives us a lot more flexibility. Another example that I like a lot is Ash Moria he has done a lot in the lean startup space he was actually one of the really early lean startup practitioners when there was just theory and he's like okay I'm actually trying to do this for a startup how does this really work and he had this great blog which he since turned into a book about kind of his experiences and so I feel like so his company at the time was called Wired Reach I feel like they were one of the early practitioners of continuous deployment and he just kind of compared his before and after experience with it and what he said is you know before they were doing two week release cycles after they were releasing multiple times a day. Before releases were all day events like I know I've been there before when you're like oh my god and it's frantic and the whole team's like trying to make sure it works right now that they release all the time releases are non-events before releases were hundreds of lines of code now releases are under 25 lines of code and before they had a lot more emergency releases because they'd only put something out every couple of weeks if something didn't work like it was such a big deal to have to go release something again because it was like an all day thing to release it and they had just deployed such a huge amount of code that it was really hard to find where the error was now it was no big deal like if they pushed out the release it was like a couple lines of code so A they could either just pull it back in or go look at those couple lines of code and find the bug in like two seconds and do another release so it was no big deal so I really liked kind of that before and after and how much easier it makes things which I don't know I feel like it's unintuitive you feel like it's gonna make things harder and it actually makes things easier so I kind of like I feel like I've heard that a lot in Agile like if you're not good at something just do it a lot right and then you get really good at it. And then one other thing that Ash Moria does which I really like and I feel like continuous deployment makes this a lot easier is he actually adds a validated learning column to his Kanban board and the idea is that a feature isn't actually considered done until they put it in front of a user and actually watched how that user behaves with it either in person or through analytics that are going on and I really like this because now it actually builds learning explicitly into the feedback cycle and the idea is hopefully that learning that you get is now going to feed figuring out what your next features are so that makes a lot of sense. So I just got the 10 minute warning but I guess I'm a little bit ahead of time so we'll have time for questions. These are the patterns that I have seen time and again for successful innovators and thank you for your time. Thank you for your time. So yes, so questions, yeah. I'm sorry. You don't want my cool hacker check up there? This is kind of just to add on to your Kanban. We use Kanban, I love it. I think it's definitely better than Scrum. One thing that you mentioned the learning column we have sort of a review column and I find that really helpful because it's like this pause in the development where people think that they're done but it's nice to have a time for reviewing the pull request making sure the POs are on board making sure QE is on board and I think that's really valuable and with Kanban you can kind of add in whatever you want. Yeah, absolutely. So how do you determine when something is ready to move out of the review column? When something's ready it's when the developer thinks that they are finished with the feature and they put it in review and that's when everybody basically reviews the pull requests and comments on it and only after then actually our QE department then is the one that's responsible for merging it into the release because they are also kind of watching the pull request and they can say, okay, this looks good everybody's happy with it and that kind of gives them the control over what goes into the release as well. Nice, thank you. Something I've seen people do is say basically, well, if it's not perfect when I put it out there people will get the wrong impression about what it is and not try it later. How can you combat that when people say that? Because I worked with some people who were doing a startup for like two years and they never put anything in front of anybody and I was trying to convince them to actually put something in front of somebody and they're like, well, but then they might not use it later. Right, so I have several different answers to that. So I'll just run through a few of them. One is actually a personal pet peeve of mine. I said I kind of hate the term MVP but it's like a love-hate relationship. I feel like people have taken the MVP, the minimum viable product concept, a little bit too far and I feel like people started creating MVP with crappy product and I don't actually advocate putting out a crappy product so I just want to be clear on that. I feel like you could do a very minimal feature set but not have it a crappy experience. Another thing that I would say is and it also depends so much on who your user is. Different users have different tolerance levels for this so you have to be sensitive to that. But another thing that I would say is I hear sometimes startups say this concern when they have no users and I'm like, so who are you going to offend? You don't have any users today. There's no one to piss off. Maybe you'll actually get a couple by doing this. If it's that absolutely terrible, you had no users, you can create a new company name, right? I mean, that's extreme but you don't have any users yet, right? The third way I would answer this however and actually it's funny because I did a talk here for constant contact and I had a great slide but it's not here anymore. I have this great example of TripAdvisor doing something very similar to this. So TripAdvisor actually I don't even know they have a lot of users, right? Millions of users I guess. They sometimes have ideas for new features but they like this lean startup principle and they don't want to go take the time to go build a feature until they know if people want it or not. And so with their hundreds of thousands or millions of users, they will do something. Sometimes they gave me an example. They're like, sometimes we think maybe what we should do is actually make hotel recommendations, right? You can go on TripAdvisor and you can see all the reviews but wouldn't it be nice if TripAdvisor just recommended one for you? They're like, I don't know would people want that or not? And so they actually like just put a dialogue box up as soon as you loaded the TripAdvisor page. Like you didn't even like click it to hotel or anything like just boom right in your face, super obnoxious. Do you want us to recommend a hotel for you? Yes, no. If they said yes, it actually gave them an error message saying system unavailable. Like it was totally ghetto, right? It wasn't even elegant. But the thing is they have so many users coming in that they were able to put that up for like two hours and get, I don't know, like 100,000 tests from that. And they were like, that's really good. And we pissed people off for two hours but we got this great test. And no one's, we're not losing customers over this, right? So those are my three very different ways of answering that. Does that help? Hey, Abby. I know Dropbox got lucky with posting in hacker news and gaining 70,000 users just overnight. What are some other great ways of acquiring users? So what you really, I don't know that I would totally say that they got lucky. They did get lucky in the fact that they had a very easy way to reach their audience but I think they knew who their audience was and their audience were these techies who happened to hang out. And so that is sort of the answer is figure out who your people are and where they hang out. If you don't know where they are, there's actually another story of Pinterest. Apparently when Pinterest first released they had like 3,000 users very quickly because the team was very well connected. Like I think they'd been through Y Combinator and stuff but I think like all of their users were actually like them and so like other young like 20-something techies who probably lived in the valley and I don't know, Pinterest's target market is probably like moms in the Midwest or something. I don't know. It was not the right target and so they had these users and nobody was using it and they're like, oh my God, I don't know how to reach my users. And so they tried to think of who would be the people who would use this type of thing and they realized that people create these, I don't even want to call it vision boards or something like physical boards where they like do collages and they put up pictures that they like and things that inspire them. And so they actually created pin it forward meetups. I think it was called so physical meetups to bring people together who were interested in this stuff and what they said is not a lot of people came but the people who came, I guess they let them do the vision board and then let them do it online. They were so excited about it and that was kind of the momentum that they needed is to start reaching these passionate people who could then spread the word. So long answer but you need to just figure out who your user is and where they hang out. Yeah, what's a good way to control for false positives and validations? For example, say I built an app, I put it up on Facebook, I get a thousand likes, I feel great and then I check the next day and have zero signups. What do you think is a good way to control for false positives? Yeah, I think you just have to be careful about what you're actually proving because it is, I don't remember the stats but the stats are really abysmal like in the number of mobile apps say they get downloaded and never used and so that's almost a false positive too. And so kind of be very deliberate about what is this actually going to test? Is it going to actually test that people are just curious and will download it or can I actually put a feature in there and see if they use it or not like Buffer did with that first like week and a half release? Thanks very much. Thank you.