 All right, good morning. We're going to get started. We're already five minutes late, six minutes. So it's a pleasure to see you here. I'm going to be talking about my experience of having run through a couple of startups. And in this specific case, I'm going to talk about a case study about a startup that I've been involved. And we're going to go into some of the gory details of how we pivoted seven times before we figured out what we want to do or what we were actually trying to solve. So the title MVP Hacks is about kind of trying to clear some misconceptions about what MVP itself is and then how you can be smart about designing MVPs so that you can spend very little and still get maximum learning. So I actually picked this up from Jeff some time ago. Jeff talks about if you think of a typical design thinking process, you have a list of big learnings. You have a big idea. And basically, you build a big prototype out of it. You take the big prototype and you run through a bunch of tests and you try and come up with a long list of learning. And then you go and iterate through this process. And that's kind of done in an iterative cycle till you have enough confidence about what you're trying to build. So in some sense, if you were to simplify in many sense, that's what design thinking flow would look like. Now if you compare this with the lean startup flow, it's kind of slightly different in the sense that you do have a big idea. And you highlight all the big assumptions or the riskiest assumptions that you have. Out of the riskiest assumptions, you pick the most important assumption that you want to validate. You pick that assumption. You build a simple prototype or you come up with a simple technique to basically validate that hypothesis that you have around that riskiest assumption. And then you go through some focused tests to build some learning and then iterate by refining your idea. So by taking one thing at a time, you're basically refining your idea and in the process, maybe pivoting or preserving to go forward. That's in a nutshell, if I were to compare design thinking with the lean startup approach of an MVP. So where this comes from is the BML cycle, which is the build, measure, learn cycle. You have an idea. You build something which produces some code. You measure. Once you deploy that, how things are working, you collect some data. Based on the data, you have some learning. And the idea is to basically go through this loop as quickly as possible. That's in a nutshell what the lean startup, the build, measure, learn cycle is. In fact, Ken Beck came up with an interesting twist on this cycle where he said, instead of going from an idea to code and then to learn, what if you reverse the cycle? You come up with what you want to learn first. And then you work backwards and you say, OK, if I were to learn this, what I need to do, what data I need to collect to be able to learn something out of this. And if I were to collect this data, what are the experiments that I need to run to be able to collect the data and have some validated learning? So that's an interesting twist on the BML cycle, but that's the spirit behind that. And that's where the term MVP came from. The term MVP basically is, I mean, I'm going to let you read the bookish definition of what an MVP is. Make sense? So what is the goal of an MVP? As we talked about in that diagram earlier, the goal of the MVP is essentially to have validated learning as quickly as possible, as cheap as possible. That's the idea of an MVP. Now, I think a lot of people have read this, and we've all concluded different things from what we mean by an MVP. I've been at a company where an MVP means release one. Six months of building a product, and they call that as an MVP. I've been in companies where they don't build anything, and they call that as an MVP. So this MVP seems to be very loosely used in all different contexts. So without arguing who is right or wrong, I think what we need to focus on is what is the outcome that we want to achieve, and what can we do to get that outcome as quickly as possible. I'm actually going to play a quick audio. So my name's Paul Howell. I'm going to talk about a specific technique that my startup has used to conduct really realistic, really effective user tests of our ideas. So about a year ago, we had an idea for a social purchase sharing app, where you would stream out what you're buying to your friends, and they would share back with you what they were buying, and it was going to be great, and it was going to be a social networking take on product reviews. And being a lean startup, we mocked it up. Static prototypes, we got it in front of a lot of people, and they said, I'm not going to use it, but I could see how other people might use it. And we heard that again and again, and we said, well, why wouldn't you use it? And they said, because I don't know which of my friends would actually use this. And it made sense. It's a social application. And if they don't see the real faces of their friends that they can emotionally connect with, they can't actually grok what it's going to be about. So we said, all right, we need to make this a more realistic test than what we've done. And we drilled down on the most important interaction on the site, which is when someone does a purchase and shares it on Facebook, and it appears in their friend's news feed. And we were interested, would people actually click? Would they care? So we thought about how can we make this realistic? Do we have to build the whole thing and build this pretty serious sized app to do this? Or could we fake it really well? We decided to try to fake it. And the technique we used was a grease monkey script. So if you're not familiar with grease monkey, it's just a simple little JavaScript that can change the way a website appears. So here's an Amazon product page. After I installed grease monkey, it now pops up every time I go there, a little yellow box that shows competitors' prices. So it's specific to one page, and it alters that page. So I went on renegoder.com, described what I wanted. And I found a guy in the Philippines who's willing to build their script for $40. We sent her a $40 across the ocean. And a few days later, sure enough, back in the script. And it's pretty simple. There's the whole thing right there. You just drag that into Chrome or to Firefox. And it wakes up every time it gets on its target page. So every time we went to Facebook, it would wake up and it would run itself. So the next thing we do is our standard procedure. We post an ad on Craigslist. They were running a social media focus group. We bring people in. So what's your favorite social media site? And invariably, they'd say Facebook. We'd say, great, why don't you log into Facebook? That's a great idea. And up would pop their news feed. And they're feeling totally in control. Seems very realistic. Their news feed pops up, and the grease monkey script runs, and it inserts fake content into their feed. And it's using their real friends' names and faces. So it's pixel-perfect real. If we'd built the whole thing, it wouldn't have looked any different than that. And so then we sat back and said, well, are they going to notice this content that our app would normally insert into their feed? And sure enough, no prompting. They were like, wait a minute. My friend, Michelle, bought a Lady Gaga album. And my friend, Charlie, bought an iPad. And they noticed. And they reacted very strongly, hated it. This is how Facebook is going. The hell, it used to be about friends. And now it's about commercial stuff. I remember back in the day, we would share poetry. And now I hate these ads. And we did it 50 times. And there were three people who liked it. So it was a very different reaction than when we did the initial prototypes. And it really wasn't that hard to do. It was a $40 script. So that's my main point. Don't think that you just have to do with paper. You can do something better. So one thing to explain about grease monkey, it only works on the. So I guess you get the point. What was the biggest risk? Or what was the biggest hypothesis they wanted to validate? Product will sell. What they wanted to know is that would people notice these ads and click on these ads? If they clicked on these ads, that's how they're going to make money. That's the fundamental business model. They wanted to validate that business model early. So instead of going and building the entire product or an MVP of the product, the way they designed their experiment was to essentially spend $40 and get a grease monkey script. Now to me, that's a brilliant hack. That's a brilliant hack to validate your business model. That's a brilliant hack to validate your riskiest assumption. And in fact, he goes on and talks about how two other companies at the same time went all out, got VC money, and then started building it two years. They made it big on tech crunch. And they were covered on tech crunch and all of this stuff. And these guys are thinking, our experiment seems to be telling something totally different. How are the VCs funding them so much? I mean, did our experiments be anything wrong? Did we do something wrong? While they kept pivoting and trying different things, these companies started building the entire product. And two years later, they said that was a stupid idea and they shut it down. So this is an interesting way of learning early and fast. And I think if you guys know the Zappos stories, the Zappos founder did the exact same thing of kind of doing something, literally going to the stores, buying a bunch of shoes, taking the pictures of those, putting it up, and then asking people if they would buy the shoes. He didn't go build a warehouse. He didn't put a procurement process in place. He didn't do any of that stuff. He wanted to validate if the market is right for people to order shoes online. And they started with shoes and they've kind of got acquired by Amazon and they've grown since. But that was a very cheap experiment. They ran to validate their business idea. Let's look at one other quick video where I think we will see again this in action. Hi, I'm Mike, founder of DollarShaveClub.com. What is DollarShaveClub.com? Well, for a dollar a month we send high quality razors. Hi, I'm Mike, founder of DollarShaveClub.com. What is DollarShaveClub.com? Well, for a dollar a month we send high quality razors right to your door. Yeah, a dollar. Are the blades any good? No, our blades are f***ing great. Each razor has stainless steel blades and aloe vera lubricating strip and a pivot head. It's so gentle a toddler could use it. And do you like spending $20 a month on brand name razors? 19 go to Roger Federer. I'm good at tennis. And do you think your razor needs a vibrating handle, a flashlight, a backscratcher, and 10 blades? Your handsome-ass grandfather had one blade and polio. Looking good, pop up. Stop paying for shave tech you don't need. And stop forgetting to buy your blades every month. Alejandro and I are going to ship them right to you. We're not just selling razors. We're also making new jobs. Alejandro, what were you doing last month? Not working. What are you doing now? Working. I'm no Vanderbilt, but this train makes hay. So stop forgetting to buy your blades every month and start deciding where you're going to stack all those dollar bills I'm saving you. We are DollarShaveClub.com. And the party is on. And yeah, so when I'm coming to see you, see you. Was this a start-up? Not really. They existed much before that. But then they said that this is not really helping us. So they made this commercial. This is that the founder is actually making the commercial himself. And I think they spent $20,000 to rent out that warehouse and all of that stuff to do the ad. And then they had the sign-up page where people would go and sign up saying, yes, this makes sense, and I want to get the dollar. I think they raised $26,000 in the first week of the ad. So who needs VCs? But what was the hypothesis that they were trying to validate? Selling of the idea. But online raisers, there was something more specific. They would prefer not to spend $20. They would prefer to spend $1. But then that's obvious. Why do I even need to validate that? I just build the product and just start shipping it. He already knows what they want. Basically, the problem he's trying to do, the biggest hypothesis he has and what he wants to validate, if I give raisers for $1, would people buy it? Would they consider it's too cheap? It's bad quality and no one's going to buy it, right? Because people are paying $20 for raisers. And if I'm here giving these crazy blades for $1, would people even buy? Because the moment you put a price to something, you associate that to the quality. Cheaper something, we think it's the bad quality. So he's trying to validate if I put something for $1, would even people care? Would they buy? That's one hypothesis. The second hypothesis that he wanted to validate in this. The second hypothesis was, do people forget buying blades and would they prefer it getting shipped to them automatically? Because there is a certain rate at which you consume shaving blades. There's a predictable rate at which you consume shaving blades. So if we took that off your head and we just kept shipping you blades by knowing your rate at which you buy the blades, then would people care about that? So he talks about, did you forget to buy the blades? Don't forget, we will ship it to you, things like that. So he wanted to validate those two hypotheses and see if people actually would care about buying the blades. Does it make sense? So one approach would have been, basically, build up an inventory, source all the blades, then create a full massive supply chain infrastructure, do all of that stuff, and then launch the website and see how many people would go and buy. That would require a significant amount of investment and time. And what if people don't buy because they think $1 is cheap? It's not a good idea. So instead of going doing all of that, he wanted to validate his two riskiest assumptions by making a video. Did he build a product? Yes, he did. This is the product. Product doesn't have to be software. The moment we say product, we think of software. It's beyond that. The product is not the software. Yeah? It's cheaper things. So I think they could have validated it. Just want to add to that. I don't know. Nano, to me, is a big success, in my opinion. So I think they achieved what they wanted to achieve because they were market changers. They would change the trend. You see a lot of big cars now coming out with cheaper cars for that price range. How many people bought? Maybe that didn't meet their expectations, right? But it did change the market trend. So it's interesting. Let's kind of talk about one other thing. So when I remember when I had this idea of creating a job fair, I said, oh, let's go build this entire website and build all this infrastructure because that would be awesome. We said, no, no, no, let's just do a Google Forms and see how many people are even interested. So the first thing we did was actually to validate the idea if the job fair that we talked about is even viable if someone would be interested, is to just go create a Google Form and see how many people would sign up, right? And we saw actually about 200 odd people signed up and then we said, OK, now how do we even visualize this information for other people to see? That's when we actually went and built a little website saying, OK, this is how different people and different parts of India are signing up for a job fair. Again, the point I'm trying to make here is kind of, you don't have to, so the MVP was really this for us to validate whether people will go and apply for jobs because there were a lot of hypotheses. Would people want their information to be available like this in some job fair because they're coming for a conference? A lot of different things were risky assumptions for that. So we wanted to validate that quickly. Before we get into the meat of the stuff, let's take a quick commercial break, right? So my name is Nareesh. I live in Mumbai. I was part of a company called Industrial Logic where we built e-learning for software professionals. Post that, I started another company called Adventure Labs where we built games for kids to learn mental arithmetic, built a bunch of other stuff. I started the agile movement here in India. I've gone on to do a bunch of other conferences. So right now running conferences is kind of a part-time thing that I do. As part of that, I've actually started building a startup called Confinement, where we basically build software for managing conferences. I think right now we have about 16 conferences using this. So it's still in its early stages of evolution, but I think we're seeing some good traction. These are bunch of companies that have worked as a part-time employee or as a consultant. Or as a full-time employee. So that's a quick brief about myself. Lot of training, consulting, background, but then over the last few years, focusing more on product development and kind of trying to do startups. So I've had all the Kool-Aid of the lean startup and failed miserably multiple times. And that's what I'm here to share, the experience report of Adventure Labs. So the kids game company that I talked about, that's basically, I'm gonna take you through the journey of where it started. So me and my cousin kind of got together at a family gathering and we were talking about, I'm doing this e-learning for adults. He's like, why don't you do something for the kids because that's a big market. And I also started realizing that trying to change programmers is a wasteful effort. It's not helping too much. Our completion rate of our e-learning was less than 20%. While it was better than industry average, it was very depressing that you've built something with so much of effort and only about 20% of the people who start the e-learning completed. So it was kind of very depressing for us and I thought trying to change programmers is not really helpful. And at this gathering, we started talking about, this cousin introduced me to this concept of abacus or mental arithmetic. So people do this mental arithmetic where they basically, like this is a competition where triple digit numbers flash on the screen and kids have to keep adding those numbers or multiplying those numbers depending on what grade you are. And I think they run about in two minutes, 120 problems and they go that fast. So just looking at some of that stuff was amazing. Then he had his two daughters who had done this abacus and Vedic math courses and you would read out numbers as fastly you could and they would multiply them faster than you could actually do it on a calculator. So there was something very interesting going on over there. They could remember up to 20 numbers in the exact sequence that you read out. So pretty good memory power for kids. The other interesting thing is actually you can go watch this video online where actually you read out numbers like three, five, seven, two, eight, six and the kid is actually doing three multiple calculations simultaneously. So I was like, wow, I mean, this is something very interesting. So how about us trying to do something to help this, to do this? And that's where we came our vision saying every kid can actually calculate faster than a calculator and they don't have to be born genius. This is a skill they can acquire. That was our vision, like you could actually acquire the skill. We wanted to empower a generation of kids who are not afraid of numbers because you see that that's a growing trend these days where kids are really afraid of numbers. And then we started doing the Kool-Aid, Lean Startup stuff right by the book. We said, okay, let's put, what is the parent's value hypothesis in what we are building? Why would parents care about something like this? So we put together a bunch of parents' hypotheses based on various interviews we did with parents with various interactions we had with parents. Bunch of different hypotheses we collected. Then we said, okay, what about students' hypotheses? Why would students care about something like this? What do they want? So we put a bunch of hypotheses for the student. Then we also wanted to put, how are we gonna scale? What is our growth hypothesis? How are we gonna be able to scale this? What should be our model for us to be able to scale it? So we said, okay, here are a bunch of techniques we could be using as a growth hypothesis. And that's where our journey began, or I should say our hacks began, to figure out what should be the product that we are gonna build to meet that vision that we have. The vision is we want parents to believe that their kids can calculate faster than a calculator and they don't have to be born genius to be able to do that. That's a skill they can acquire. So our first idea, which we thought, is like we said, let's look at the traditional abacus, which is what the kids learn. That's the first step in this process. So they learn the abacus and this is to visualize the numbers and then convert the numbers into patterns. So basically what you're doing is you're moving from a number into a pattern and then they use the abacus as a way to train their brains to crunch patterns. The problem with this was basically there was no pinpointed feedback when the kid made a mistake. So the learning process, and we believe in agile as well, is that the faster you give the feedback, the better the learning is. So if you don't get the feedback, if it's a class of 20 kids, then 20 kids are gonna do something and the teacher's not gonna be able to notice every single kid. So there needs to be a way of getting kind of feedback and the other thing was around self-pacing or adaptive. None of this is adaptive. The teacher says first problem, everyone does first problem. The teacher says next problem, everyone has to do the next problem. So it's not adaptive. If you're really smart, you could keep going faster. While you need more hand-holding, you could go at your own pace. Those two things we said is the real problem with this thing. So that's where came our hypothesis that we wanted to validate is providing instant feedback and allowing kids to go at their own pace. Those are two things we said, will people care about this? We believe this is important, but will people care about this? So there came the idea of, okay, where do we solve that? Let's build an electronic version of the abacus, which basically will give feedback as the kids move the beads. If it's a wrong bead, then it'll give the feedback to the kids saying, no, this is not right. So we actually started with that, but we realized it's really hard to build all these electronic gadgets and stuff like that. And that's where we pivoted and we said, can we actually build an iPad game to start with to at least help kids learn the abacus? So we were thinking of doing a physical electronic version of the abacus from there to validate some of their ideas. We said, let's move to actually just doing it on an iPad, on a tablet, where kids could move the beads and we could see if they actually helped them. So we actually didn't even build an iPad. We found an iPad app which had an abacus. It was freely available. We took this and we went to a bunch of kids and we said, can they actually learn from this? Would they use something like this? And what we found was kids were actually very interested because those beads look interesting. They wanna keep playing around with this. We're talking about kids age five and six. So then we started thinking about how will the kids actually learn the abacus, assuming that's what we want to start. So we started with saying, okay, we're gonna use a little animation and that animation is gonna teach the kids how to move the beads to start with and then the kids will follow on, right? So we actually went to a few years ago. Now that everyone is settled down, let's begin by pressing the reset button. It will clear the abacus to the smallest quantity with which you deal in your day-to-day life. Any guesses as to what that number is? It has to be 14, my birthday. Seven. Seven. Seven. Seven. Everyone be quiet. Why don't we answer it one by one? Whoever is sure. What was the issue with that? That was bloody expensive to make. And then each of those you make and then you have to change a little wording somewhere because culturally something is not right or things like that. And then you have to go re-render the entire thing. And each of this would cost us about 6,000, 7,000 rupees. As a startup, you don't have that kind of a money lying around for you, right? And we had to build two years worth of content. So for building two years worth of content, that would be about 18 crore rupees just in making animation. We said this is basically not very useful. But the other challenge we also ran into is, well kids watch the video and then by the time they actually get to the abacus they've forgotten what did happen because they got all these interesting things was happening about the storyline of kids and all of that. They forgot what was happening. So the next pivot we did was essentially embedding the video in the game that we talked, in the abacus software that we talked about. Now that everyone is settled down, let's begin by pressing the reset button. It will clear the... So basically what this would do is it would play the video for every small section and then there would be an abacus below so you would follow along, right? And we put that really dirty looking UI very quickly together. In fact, we didn't even built it. This is built out of Keynote or PowerPoint and basically we dragged the video in it. We put those beads and we asked the kids to move them and whatever they move doesn't matter. It'll move because it's pre-configured. We just wanted to see if the kids how they would react to something like this. Then we started saying, okay, we wanna basically add a game element to it. So there's a little game kids can do. We tried to do some simulations through this to see if kids would actually use this. That was like a second pivot, embedding the video in the abacus and trying to get the kids to do that. We're now very happy with that because what we realized is out of the, I think, 60 odd tests that we did with kids, only about 10 of them were able to actually complete all the numbers, one to 10, they were able to represent. And so we said, well, this is not really working out. While we've given all of this, this is not actually helping. Unless we wanted to eliminate the video that I talked about earlier because that is very expensive. And that's when it struck us that why don't we do something like a game because in a game, like if you look at Angry Birds or any of that, there's no video, there's nothing. What they have is basically self-discovery and a simulation. So they show some kind of a simulation and that's what people do. So they will go figure out. So there's a little simulation that would play and then you would basically go and do that yourself, like now your turn, go do this. So we knocked out the video. We tried to focus on this. This is all simulations that we just built using Keynote and then we would put it as a, export it as a video, take it to kids, put it on an iPad plate to see how they would react to it. So we ran a whole bunch of tests to see if this is interesting for the kids, right? Gonna kind of jump quickly. This shows like how you could go faster instead of doing one at a time and things like that. Now, our challenge was how do we make this viral? How do we get kids to actually use this on an everyday basis? How do we get this to be viral? That's where we started building the gamification element. We started saying we need a storyline that will get kids attracted to this. Started building a storyline around this. We came up with this whole activity-based learning where as soon as they come in, they start moving the beads, they start learning what needs to be done. Bunch of other stuff like a whole storyline, a store where kids can do. We introduced a character which you need to feed and things like that, and that's what will give you consistency for the kids to keep coming back and playing this game. If you don't feed the character, it's gonna starve and die, right? So you have to keep coming back and playing this game on an everyday basis. Otherwise, your pet is gonna starve and die. And how to visualize what progress you're making? How do you visualize where the kid is progressing? So one is on your thinking ability, another one is on your memory power. So we try to introduce all these gamification elements into what we were trying to do. We added practice sessions where we started building a little, so you've gone through this, now play a little game to practice what you have just done. And then we started showing this to people. We said, okay, now we've run all these tests. We knew right from the beginning that we are nobody in the education space, so if we launch this on our own, probably even my wife wouldn't buy this, right? So we need to actually go and have a tie up with someone big in the education space who's gonna push the envelope for us, right? And what we found was basically people's reaction saying, parents were not really convinced that this will actually help them, nor were the companies that we were talking to convinced that this is the right time. They said this is fantastic, but it's too futuristic. We don't see how kids will learn something from this. They'll get lost in the games and stuff like that. So that's when we pivoted again and we said, what if we take one small portion out of this and launch it on the App Store? So we actually took this little piece and put it on the App Store and made a little video which shows how to use this particular thing for the kids. And what we found was we actually had pretty phenomenal success when we put that out. As you can see, in one week's time, we had crossed 100,000 downloads on the Google App Play Store. So that was very encouraging. But the only reason we did that, so we could go back to those companies and say, you know what, the market is ready. Look at this, we are number three on the Google Play Store in these countries. Not that we wanted to directly sell it because we knew games wouldn't sell online. You would probably charge $1 or $2 and that's not gonna be enough for us. So we took this data and went back to the companies that we wanted to talk about, hey, the market is ready for something like this. Look at this, there are 100,000 downloads that happened in just a week's time. Are you guys with me so far? I'm kind of rambling on my story, getting to a point basically saying, how we kept changing all along the way is we're trying to figure out what we need to do. Somewhere in between, we said, these guys are not gonna get it, like forget it, it's not gonna, every time we approach some of these big companies, they said, this is fantastic. Forget what you have done, that's very specialized. Can you take our content and digitalize it? That's what we kept hearing from them. They said, this is fantastic form factor in how you've done things. Can you take our content and do this? So they were pushing us for us to actually become a services company instead of a product company. And we were like, no, no, no, we are interested in this vision. We don't wanna go off our vision and start doing a bunch of services. So at that time we said, there is a lot of demand for this. What if we actually change our app to a platform? So we pivoted in between from an app to becoming a platform. That didn't fly really well. Finally, light at the end of the tunnel, we went talk to a company which was at the bottom of a list saying, never we're gonna go talk to these guys because we're basically sabotaging what they're doing today. And so we said, well, beggars can't be choosers, let's go talk to these guys because that's our only resort. And what we found interesting is they said, we like what you have done, but we wanna kill it because it's gonna kill our business model. We said, yes, we thought so. That would be your response. But they said, these are our problems. Can you help us solve these problems? And one of the problems they talked about is basically they wanted to run a contest online. So they said, can we use your software to run a contest online? Mental arithmetic contest online. We said, well, that's interesting. And then as we started talking to them, we found a few other problems because these guys had franchises. And they didn't know quite how each student in each franchise he was performing. So they wanted some way to get that data. So that's where came the birth of our final stage, which is basically converting this into more of an adaptive learning platform for teachers to get feedback about the students and to run contests online. And kind of show all these fancy reports and stuff like that. Anyway, the moral of the story in this long history that I talked about is basically we started with a vision. We came up with a strategy. We said, we were gonna achieve this vision via this strategy. We said, this strategy is based on a bunch of hypothesis that we put together. And then we ran a set of safe fail experiments to validate our hypothesis. And through that, we got some validated learning which helped us go and tweak our strategy. And we kept going through this till we found some light at the end of the tunnel which basically is kind of where we started from an electronic abacus to a practice platform. Quite different, but still helps us achieve our original vision of empowering the kids to learn faster than, to be able to calculate faster than a calculator. So in a nutshell, that's what I would say is that starting from a vision, you come up with a strategy, put a bunch of hypothesis together, validate the hypothesis through a bunch of safe fail experiments and go through this cycle. We probably did this entire thing over a period of two years while we're still building a team and trying to figure out but it was mostly part time for all of us. This was more of a part time thing that we were doing. We were conscious about how much money we were spending because a lot of what you saw we were doing was all really quick experiments through keynote or through other things. We were not really building heavy software to do this. In fact, till date, we've not built the entire thing. We started selling it, but we've not actually started building it. Have you guys seen the Simon Golden Circle where he talks about why, how and what? So right at the center of anything is the why. Why are you doing something? And then you describe saying how are you doing it and then you describe that further by saying what helps you achieve that. So I kind of tried to map that back into what I was talking earlier around the vision strategy and the product. So you have a vision which rarely changes which is at the center right there is why. Why we wanted to do something. And then the strategy is basically what we kept pivoting on based on our validated learning. And a product that we ended up with is quite different than what we had originally imagined but still focusing on our vision. So again, how do you go through this cycle cheaply so that you can keep pivoting at a very low cost. And to me that's the very essence of what the MVP thought process is. Basically getting validated learning as quickly as possible. So there are a bunch of books that might be helpful on this subject that talk about that. But yeah, I'm pretty much done at this stage. Any questions? Yes please. So that's a fantastic question, right? What is the take away from this, from an agile perspective, right? Like how do I apply this thought process in an agile kind of an environment, right? So I would summarize that one of the things I have a big problem with agile is that we only iterate on the product, on the features. But sometimes where you need to be iterating is much higher, right? Like what we were trying to do here, we were not really iterating on the features, we were iterating on the strategy itself. How do we achieve this thing? And sometimes you think you have the right strategy and you keep going and iterating in an agile way and keep doing sprinting after sprinting. But you might end up building something that's totally irrelevant, right? So I think real agility would come into action when you start applying some of those principles at a much higher level to basically come up with the strategy itself, right? And constantly keep validating, have more of a safe fail experimentation approach where you have some data that you want to measure and then you run it like a bunch of science experiments, right? Once you understand what you're doing then go invest a little bit in it and start building it, get faster feedback. So you don't have to iterate only at the feature level, you could iterate at a much higher level and I would actually argue that that's where you will get the maximum benefit than just iterating at the feature level. So for a, so I'll just read out his question so others can hear. I think your question if I can summarize is did we go through this entire cycle to be able to identify a product owner who could then take the ownership of this and run with it? I look at it slightly differently. I don't care about the product owner. I think that's a term that is very badly abused in the agile world. I've actually never seen a product owner being able to balance all of these things, right? The myth that one person would be able to do all of this I think is flawed, fundamentally flawed in fact. What we were trying to do is, we were trying to figure out what problem we want to solve because there were many different ways we could solve that but which problem to solve and how do we even strategize it? So even till date we don't have a product owner, probably we'll never have a product owner. I don't think you need one. If everyone in the team is focused around the vision and what needs to be built, I think the team can take on that role instead of having one person playing that role because that could lead to one person's wrong judgment and the entire product failing. So I am not a believer in the whole product owner role. Your title says sell before you build. So, you know, but you seem to have built a lot without selling anything, that's right. I mean, so did you actually sold something, got an order and a check and started building it or you actually built something and then started going out and trying to sell it? So that's a great question. Did we build something? Now, you need, before you go and sell something, you need to know what you're gonna sell. Not the, so we didn't actually build the product as I told you. Even till date, we've actually not built the entire thing. There's a lot more. There's a two years worth of content that needs to be built. What we've only done is maybe less than 1% of it and that also we were forced to build because when we tried to go and sell just on the prototypes that we had built, people said this is fantastic, but it's too futuristic. So the only reason we had to build the game was to actually put it in the app store and that was maybe less than a week's worth of effort. And that, again, that was a throwaway prototype in my opinion. We don't really make that as a part of our application itself. That was something we built just to validate that there are actually people with tablets and they will download and use this. So to your question, the idea is still that you wanna sell before you build. So have we built the platform that we eventually saw? No, right? I mean, now we have built, but when we signed the contract and we got the money, we actually not built even one single line of it. So what I'm trying to get at is that you need to actually have, you need to be certain what you want to build before you start going and building it. So a lot of this is how cheaply can you do it without actually having to build it. And the examples that I showed earlier also, right? Where they were using grease monkey scripts or a video or things like that, right? Those are all, again, techniques where you're selling before you're building, right? Makes sense? Yeah, I had a question on how do you reconcile kind of the lean startup model that you described with a typical agile software development, right? So in essence, every time you learn something or every time you would pivot from one thing to the other, would you go back and execute a sprint or create a user story? I'm just trying to understand how those two pieces kind of fit together. That's a great question, like how do you reconcile this back into the agile model, right? What would happen? So now you ran an experiment, you got some learning out of it, what would happen? Do you go back and write a story and put it in a sprint and execute it? Hopefully not, hopefully not because you don't need all that ceremony, right? That ceremony is good if you're a big giant company in a specific context trying to get people to work together and things like that, right? But if you are a really small group of people, really sharp set of people trying to move very quickly, all those ceremonies will get in your way, right? So you don't really need to write a story. You know you ran an experiment, you got some validated learning, now you want to try and do something different, you just go and do it. Now do you care about the estimates? Do you care about how long it's gonna take? How many sprints do you do, estimation, story points? No, it's learning wheels for people to get to that stage and when you get there, you remove the learning wheels, else they'll slow you down, right? So to me, agile is good to get started and get into the rhythm of some of these things, but once you're there, you should move away from those training wheels and you should start focusing on how quickly you can learn. I think if you were there in Jeff's keynote yesterday, he talked about, you know, agile seems to be obsessed with velocity, velocity of output, which means nothing to the business, right? We should be obsessed with velocity of learning. How quickly can we learn and how quickly can we change directions? Because I think that's what will make us successful in the long run. Okay, so lean doesn't imply agile is what you're saying. It doesn't have to be agile. I don't think even remotely I said that. Yes? As part of the validation you did and then you started, depends on the market analysis, you started building it, but in the mid of two years, how do you get the, whether you are on the right track? Probably sometimes after end of two years, we may, the customer's response to the app may differ. So how was your- So if you saw the continuously we were validating with actual users, right? The only thing that we made a big mistake in hindsight is that we assume that once we've actually worked with the real users and solve their problems and we've got something useful for them, then we will be able to find a company who's gonna basically market this for us, right? So we left out the marketing and the tie up till the very end and that was a big mistake for us. But there was constant validation going on throughout the two years to make sure that we are building what is actually useful for the users. Now we focus too much on the users, we left out the marketing company in this sense and that's where I think it took us a much longer time than we would have ideally liked, right? But ideally, because you're working with the actual users you've built something which is useful for them. Now the challenge is that what's your business model? That's something that we had kind of assume that we will make money through some kind of a commission or one time sign up with some kind of a company, right? And unfortunately that kept going in different directions than we had expected. So that's what took us so long, right? But there was continuous validation with one set of users which we believe will actually use the product. That was successful, right? I don't know how to define success or failure. Like for me it was disappointing, it took two years but on the same hand it was a great learning experience and we were not doing this because we thought this is how we're gonna make money. We all had full time, you know, our own jobs that we were doing, running our own company. This was more of a part time thing. So it's hard to define success in that sense. I think we'll have to only watch four, five years from now if all kids are able to achieve the vision that's when I would say we are successful. I think it's too early to say that. Thank you. Regarding being sort of, my thinking was that and has been so far is that as far as I was concerned basically is a kind of development wheel and building things efficiently but many times product owners are not very sure what exactly needs to be built. I mean we are building things rightly but what exactly needs to be built there can be the element of the new startup where they can conceptualize the MVP with the minimal effort that needs to build that up and show to the market and basically come up with the very good learning and based on that. So I'll just kind of rephrase this question. He's asking that basically with, you know, his understanding was that, you know, when product owner is not sure what to build then they would use lean startup principles and design an MVP and then the team would go through a very quick iteration, build the MVP, whatever needs to be built and then do customer validation and get data to see if they're moving in the right direction and that's how you think lean startup and agile could be married together. Now I think what you're describing was happening much before even lean startup happened. I don't think that's a new concept. People always, you know, I've worked with a lot of product owners, I've worked with a lot of really good customers what we used to call them who would actually design things in an incremental fashion and that's how we would execute it. Now lean startup might have given some terminologies around that but that's not where I think lean startup is really helped us. I think where lean startup is helped us is more in the customer validation phase, right? Where you don't need to even build anything, you don't need to even sprint anything, right? You can validate a lot of that by using various different techniques. For example, like on e-learning what we built, one of the things we did was basically build a fake feature. We said, you know, chatting is some, right now in our e-learning when you go you have a question, you basically leave a comment which gets sent out as an email and then one of the people from the company would respond to that email. So we said it would be fantastic if people could actually chat live and that would help people, right? So that was our hypothesis. We said, well, to validate that hypothesis, let's actually put a fake feature which is basically just a button which says that 25 experts online click now to chat, right? And then people would click on it and we would say how many people actually clicked on it. And then it turned out out of 200 people that locked in very small percentage clicked on it. So he said, okay, let's, that was a bad idea, we're not gonna do that. And things like that. So, you know, that didn't require, that's to me what Lean startup is trying to bring or at least some of the thought processes from there. Now, some fake features was not a concept really in agile back in the days, in my opinion.