 All right So let's get started welcome My name is Aaron on the co Founder of bit genics a technical startup If you've been to any of my previous talks or have seen some of them align This is not going to be like any of them This is a this is a an impromptu talk The rash asked me to do But this is going to be a whole bunch of stories around a piece of work that I did I think it was sort of 2012 And it's it's basically some of the sort of biggest career highs and Definitely some of the sort of career lows around a product that I build I helped build in a in a big telco and All of the amazing things that we've done And how ultimately we completely failed At delivering anything of value And and there's probably some sort of lessons in there to learn So to start things off I Have bought a very technical background and a very edgel background That was one of the reasons why I was asked by by a big telco to join What we call that what they call their strategic strategic innovation team and We were working on it was a telco and Basically our job the small team that we have we had a team of about sort of But when we started there was about six seven people grew to about sort of ten But our team our job was to reimagine the way media is consumed Period that was it Reimagine how media is going to be consumed and we were talking about TV but also about movies about series about video on demand and Then sort of that sort of took a life on the zone and and we built An insanely amazing product With that small group of people so Having bought that technical background lean startup had just come out a couple months before And I was sort of really excited to start sort of putting these These practices into sort of practice Do do rean startup for late for real in a in a massive Massive telco with millions of customers So this was extremely exciting Stuff for me. The other thing is I was going to be responsible for the sort of the all of the Technical aspects of of this amazing new thing that was basically going to be built from from scratch so Extremely excited when I got started on this and together the the team that we had was an interesting one we had the the guy who rend strategic innovation was a Someone who had years and years of telco experience Absolutely years and telco experience knew the domain very well We had a project manager who had been in the in the business for a long time that I knew very well very capable very Personal project manager, I there's there's very few good project managers in my book He's definitely one of those people Allows you to get to get some motivates teams to actually get stuff done But then sort of takes away a lot of that sort of bureaucracy really good p.m. and Some other people were product managers product designers UX researchers That were that were some of the best I've worked with So the first thing that we did when we had a thorny problem like Reimagined the way media is consumed Was user interviews We've been out to stores we met with customers our current customers and we talked not about what they would want That's not how we can start like don't ask your customers what they want But we started asking questions around sort of how they can currently consume media How when did they watch TV when did they buy? Video on demand did they have any subscriptions to sort of Netflix and sort of other? services out there What did they download sort of net sort of so? Don't don't sort of start out if you're starting this out with these sort of unknown massively unknown problem The trick wasn't to sort of start out asking what people want Because people never know what they want People people the only thing people can tell you Is that they don't they don't want that after you showed them something? That's the only thing you can get out of human beings reliably So yeah, we started talking And then sort of when we had a whole bunch of these data when we had had a bunch of these conversations We started to sort of see patterns. You start to see patterns This and that sort of how does that work? and That's where we sort of identified potential pain points Identified potential pain points And started forming some of these hypothesis around that And then we then we went back and talked to more people Talked to more people Does that's just sort of look familiar to you? One of the one of the things that's too doubtful for us was there's so much stuff to watch Like what am I what am I gonna watch? Turns out people make them home from work whether the kids are in bed and I have an hour People people ended up watching some shitty stuff Whatever was on right now because they couldn't be bothered trying to figure out what to watch so they watch whatever Like the first thing that wasn't sort of terrible was the thing that ended up watching So we didn't ask people what their problem was we observed how they were doing this asked what they were doing And then we identified a particular pain point and that resonated a lot with our customers But yeah, they they never really watched anything really really good. I just watched whatever whatever was on on the TV So that got us to that first stage sort of finding finding that problem And we're they were more but that was sort of the big problem that we sort of latched on to The next and then sort of we took that up to that next level, right? So if If if this is a problem, what are some of the solutions to that? What's the solution to this particular problem where we and we always end up watching something something crappy Because we can't be bothered looking up something and and that's when we had a lot of brainstorming and sort of talking and sort of within our group to sort of have a So have that have that sort of conversations around how are we going to fix that? What are what are we what do we think we can do? Yeah, and one of the bigger bigger things that we sort of realized is On wasn't wasn't a feature. I wasn't we're gonna build this thing But again, it was this really big high level But if we need to split We need to split the the looking up and Defining out about things and the actual watching By the way, if there's any questions pop up, right? This is about as free formats is gonna get is that a question because you're standing right in front of a lamp This was like I think was 2012. I had to go back, but it was it was sort of late 2012 Well late 2011 like early 2012. I think this was yeah a couple years ago So yeah, that's sort of where we were at right? It's the we we we're still not talking features We're still talking sort of very high level. What can we do? So so the first thing we realized is the behavior that we want to see that the change in behavior that we wanted to see Was that people that we were going to split those two things we're going to split out to the Looking up the knowing about things to watch and the so we need to make it really easy for people to sort of Save and keep track of things are interested in you would you would be at work and someone would say have you have you seen that? That new Syria that was just released and like no no, I haven't heard of it And you could easily go I'll I'll sort of save this and if a new episode would come around it would automatically be added We can do a bunch of recommendations. So so without going into very highly very very high level features This is the behavior change you wanted to see and then only then did we start to think about features Only then did we start to think about well what what features? With this thing need to have So first we spend a whole lot of time talking about the different Features that the different problems finding out about the problems once we found out about the problem. It was very much Confirming that problem after that that sort of that high level What's the behavior you want to see and only then did we get into that? So more feature-based Feature-based thinking And the first thing to do there after we after we had done all of that was around Prototyping guys he's trying to get your attention So we were after we that when we got the features right we were making paper prototypes Paper prototypes One of that sort of UX researchers were doing quick sort of prototypes on paper show them to customers This is what the screen would look like What what would a be if what would you do instead of? And if they sort of pressed a button and pressing a button was sort of pushing on the paper, right? We would show them the new Piece of paper. Well, this is what the next screen would look like very very low fidelity Because they were just Pieces of a for wheat we lines and scribbles on them But they were enough to get some really good feedback From those sort of customers It's got some feedback the flow was confusing and it didn't make any sense and we could reduce steps and Sort of changed the way things were called and sort of labeled so With that really low fidelity Literally minutes and this was sort of we were talking minutes worth of work like not just of hours or days a lot of these mock-ups were done in Minutes worth of work and That's sort of and we progressed in that fashion So after that Paper prototype we started doing HTML mock-ups So people could actually sort of click through it Still didn't do anything still very much And this was like a good question like we're talking sort of certainly tools-wise this was sort of back in 2012 To do that sort of stuff now There's there's a whole series of tools up there that makes this very easy for you The one I've seen you sins is a tool called pop But there's many more of it where you can sort of you can very quickly Take photos of a wireframe of something you drew and then say if they click on that They sort of click on this particular area off the screen go to that That one right now and so there's there's a lot of tools that will make this easier for you that we didn't back that We didn't have back then But but yeah get get that certainly if you're building these very user-centric applications go out with the lowest fidelity thing And start having conversations with your with your potential customers Fine potential customers have it to have a chat walk through what you what you're suggesting So you went from paper prototypes HTML prototypes And this is where things got interesting We had great feedback along the way People absolutely loved loved the product. They love the things that we were doing with it and Everyone was very enthusiastic about And that's where we made a relatively big investment Just remember right now. We're talking so the investment of days and then we spend a couple of weeks building a prototype and This was a set-up box that had to fit under people's TV for Literally like ten tens of tens of dollars, but we sort of took off the shelf components. We sort of started out with the Mac mini And we had some some a bunch of software and we cobbled together a working prototype like wasn't wasn't a toy It wasn't anymore that the This is sort of if you click here you get another static page, but this was actually working stuff Yeah, it wasn't ready to go into production right because it was the hardware was too expensive The the server would collapse if we if sort of more than ten people would use the thing at the same time So it definitely wasn't production ready, but it gave us again It's tremendous amount of feedback because now we could observe people using the system Using our product has anyone ever been in in a user Session where you have seen people use your stuff Like it is the most humiliating thing you can do Who's talking about it? I think it was suggest to talk about it on Monday I'm not here, but but your your list really sort of shouting at your users like it hopefully inside your head There submit buttons right there Red pick it pick it or or or yeah, like you'd ask people to go Could she go into the configuration? And change this setting like ask them to accomplish things Don't tell them what to do go so we would go Watch watch this particular movie Change this setting and and people would especially that one. I remember that As it was yesterday, so I'm sitting there with it with a user they go so Change this they change this particular setting And they spent two minutes looking at the screen And I'm like that two minutes is a long time right if you're if you're observing people for two minutes. That's an eternity and And you ask them to sort of vocalize sort of what they what what they're thinking and My god, I can't find where would the where would the settings be? Or it's anything they'd sort of go over the entire thing. There's a gear I could write there Can find it so one of the most humbling and humiliating things at the same time is observing actual users using your Beautifully designed carfully crafted stuff and and just failing miserably out And again, that's that's extremely valuable feedback Extremely valuable feedback To use because apparently a gear icon only works for Mac people Which is what all all of us sort of designers and IT people were using and didn't work for People that were using windows because that wasn't the icon for settings And it's that those little things that make a tremendous difference in how people perceive the quality of your product tiny tiny things But even more fun When we have this working prototype well that we could now observe people using it in real time We could get this into sort of living rooms Install it in a couple living rooms and sort of see what people would do with it and get some actual data And there's some hilarious stuff that came out of that Because the other thing is we could now test new ideas We could test new ideas So one of these ideas one of the execs When we need We had on that with an iPhone app But the execs went we need To be able to control the TV not with a remote, but with your phone your smartphone and we went Doesn't sound like a great idea and and but we went Well, let's give it a shot We've got this prototype we can add stuff to it. We can remove stuff to it So we spend a couple days and we built The remote but we had a problem one of the one of the problems was is we if you had multiple screens in the house You you couldn't figure out which TV you were controlling So what we did is we built a before you got into remote control What we did is we showed pictures of What was on the different screens in your house and then you could select the right TV to then control And so we built that functionality And we released that and we we to the people that were using it and to our user focus groups and No one liked it No one liked it except for one demographic One demographic loved this particular feature of using the remote control but Here's the interesting thing. They never got past the selection screen. So they were using the remote but not actually the remote control So you started talking to these people. I'm like, what's going on? What this new feature? What do you think of it? I'm like, it's amazing. It's fantastic It's fantastic These were the people that had teenage kids and They could now on their phone see what they were watching in their room So he very very quick quickly killed the remote control Because that was useless. No one wanted the remote control. That was that was pointless What people but what we did was had a parental parental control You can you can see what the different screens in the house are showing and And through through many of those iterations We we build literally an amazing product, right? It's it's five years on And there's still nothing really on the market That sort of does what we were trying to do like able to be the new able to be is great but even then I certainly certainly think that We had some amazing things that sort of aren't aren't properly done in there yet Certainly to the sort of the breath of media that you could sort of have So it's an amazing product Everyone loved it everyone loved it the customers loved it the Sort of employees of the company loved it the board loved it and And so everyone thought this was an amazing things thing and we should build it But as you can sort of guess with this being a post-mortem There's there's none of these things None of them were ever manufactured none of them were ever built Certainly none were ever sold and this is sort of where another whole bunch of learnings came from because The biggest mistake That we that we made Was that we at no point in time? Had a go-to market strategy It wasn't our job and to be to be sort of be very defensive for a bit It was not a job our job was just to Figure out what the product should look like not to actually build And that's what I've seen a lot of these sort of labs Like banks are they're very popular now They have sort of labs type situations where we try to figure out what to build And we weren't exactly that situation Our job was to figure out what to build and then we would hand it over to it And they would do the actual building of the thing and That model does not work at all Has absolutely no chance of success For a bunch of reasons The first one is that Real IT The IT people over there have very very different KPIs These were the people that had to supply TV to millions of customers To do the kind of stuff that we were suggesting would be like enormously disruptive To what they were doing So they didn't want any change They didn't they don't want any change because that would only increase the risk of Of their service is going down and that would be Hard the other thing is this was a massive piece of work So even though we had like board level Approval for this The other thing is they had millions of requests coming in They had feature enhancement for their current stuff They had feature enhancement for their next bits and then they had this really vague built this massively different thing And all of them were high priority of course all of them had to be done yesterday So yeah, what we they built quite rightly so they built the things that were predictable and understandable Not the things that were actually disruptive So that was one of the sort of the a major problem that we encountered very relatively early That we're trying to overcome Then the next bit Was that even if we could have overcome that The financials didn't add up our financials didn't end up In that the proposal that we were having would require a whole bunch of capital investment The boxes that we would send out to our car we have to have to sort of recall all of the set of boxes and send them new expensive ones Now our research more than more than suggested that people would be able would want to pay for this People would be more than happy to pay for a new box but The way the financial structure was though the the finances were structured made it almost impossible for the company to make capital a big capital investments Even if they could recoup that money but and and and sort of lastly that that There's this sort of big brand reputation that people were afraid of It's we're we're a big serious sort of telco where the biggest in In the country so what we are gonna we can't risk Having an outage Just can't risk it we can't risk Delivering something that people would not want and this is where where a lot of that sort of what I mentioned is we are our go-to-market We had no Go-to-market strategy a because we weren't asked to give one But even if even if we did it would have been very very hard I talk a lot about how Organizations can't make small decisions big organizations can't make small decisions So for them it was almost impossible to do this in the small They couldn't go well. We'll just sort of build 100 of these things or a thousand of these things and then we'll we'll Give them out sell them Because that would a be Brent weird for their brand because some people get some people don't and It will be too expensive They would be too expensive And that's sort of where where things started falling apart very very quickly and So the most important thing we tried to do there and even felt there is Get so get we the thing we're trying to do sort of get around this is is launch in wall of their one of their smaller brands Because they had they had numerous brands They have their big big important one and they had that a couple of smaller ones And unfortunately that that almost succeeded that that filled as well So looking back To that particular project that particular gig We did some of the some of some things extremely well Like extremely well the product that we built was of immense quality people were interested and we knew people were willing to pay for it Well, we didn't we very very certain people were willing to pay a massive premium on it But We had no go-to-market strategy And I think that's most important if you're if you're in these sort of big Big organizations in these market hubs and I've seen this over and over with other other clients and people I know who are in these hubs is that Just figuring out to build is only a tiny part of the problem There's no chance of you succeeding if you do not have a strategy to go to market One of the first things we should have done and we didn't is Figure out how we are going to take this to the market Where are we going to do that? And the other thing is that handover Just can't work. You can't just hand over a massively new product to IT for them to rebuild whatever they're doing right now So if you're in these situations You have to be able to build out to your capability to do this for real and Have a market to go to market strategy that allows you to test in the market You have to figure out you have to figure out what the world of their financials that are associated with can we actually afford this? The other thing you need to be careful is how are we going to mitigate brand reputation? damage, how are we going to mitigate the risk to Brand damage and how are we going to Do that who's going to build it? How are we going to do it? IT's not going to build it Your main brand is probably not going to risk it So yeah, find find a small brand start working with them secure enough funding to Go out to your market And and continue testing there. Yeah Any questions we've got plenty of time I think Yeah, good ten minutes of questions left Are there any any questions around sort of how how we did some of the the user research or Some of the problems we were having With our go-to-market. Ah, so the question was how are we Protecting our ID from our competitors. We weren't the thing is ideas are useless ideas are pointless As as we figured out, right? We had the idea. We had all the knowledge about the idea and we still couldn't execute Ideas are cheap I Can there's a there's a really cool example that drove this home to me Where on a conference was on a conference and I can't remember who it was But it was it was a massively sort of big important investor, right? Someone who invests in In in sort of tens of companies make made millions In the process and to sort of reiterate that point he sort of had of an envelope he held up an envelope And he goes here's a billion dollar idea Yeah, if you if you do this You can you can build a billion dollar Company with it And he was trying to sell it for twenty thousand dollars Would you? Would you have bought a billion dollar ID for twenty thousand dollars? How much do you think he sold it for in the end? twenty bucks So the for twenty bucks Because someone wanted to Was fun. I'll just look that was almost curious. Well, that he was like we all know he didn't execute on the idea So having having these ideas It's great, but it's useless and if you don't have the the capacity to back that up and actually do something with it We hear a lot about these startups and they're in stealth mode and there's What was it? there was a Company Transmetta The company of the company that Linux torvils Alina's torvils the creator of Linux Sort of was working with and they were they were building a new revolutionary chip and they were in stealth mode for years Everyone knew he was working on something exciting and Because this was this was limits and then years later millions and millions and millions of dollars later they Launched massive launch party and absolutely nothing happened Nothing No one was even remotely interested in this stuff So ideas are cheap Don't worry about protecting your ideas from everyone else Sure, there's a little leak legal made a sign NDAs with these people, but really How are we ever going to enforce any of that stuff you weren't If someone on the leaks wants to leak this to their to a competitor they are and you're never gonna find out So we make them sign NDAs Because that was a legal apartment did we could have but even we didn't Right we talked about it and here's the funny thing we because the funny thing was After after that thing was camped all all people that worked on that thing left the company Basically one of the most demoralizing things I've done in my career Spent a year and a half building something absolutely amazing and then just go someone's going yeah So we all left I was an external consultant even the even the employees all quit And there was talk of shall we shall we shall we just take this to market we had all the knowledge and even we went It's a bit risky do we really want to do we really want to do that? So having all that information is extremely valuable if you're in if your intent is to go to market with it, but We we could and actually there's there's one guy that actually tried to sell it There was there was a bit of interest, but even that didn't go anywhere, so it's completely invaluable That information, but it's not it's not easy to sell or or monetize Yep, so if I understand you correctly It's sort of even if you've done all these prototype things unless you actually go to market You don't know for sure what's gonna happen. Is that is that sort of your question? Yeah, and that's exactly true That's exactly true Does anyone remember new coke in the sort of early 90s? Steve does good new coke was a So here's what I'm like Pepsi came along And and sort of coke's market share coca-cola's market share started to sort of erode Not drop dramatically, but it's sort of it was sort of constantly eroding and they figured we need to do something So they spent an inordinate amount of money on user research to to perfect their their coca-cola recipe Millions and millions of dollars on on the user research and Then they spent like tens of millions I'm sure on marketing and campaigns and and everything there's a new like this was the first time they were going to change the tried and tested Sort of coke recipe massive worldwide launch and everybody hated because it turns out it was just too sweet Which is really nice if you take one sip in a user Control setting where you go like can you which of these six do you like when you got a tip sip from there? No sip from there and and that one is the best one because but if you had to drink a can of the stuff Oh No So until you actually go to markets, you don't know anything for sure So there's there's definitely a lot of that But that doesn't mean you can't get any information from your user research All right, you can get very valid information from it But yeah, you don't know for sure until you actually go it's on the shelves or whatever the equivalent is for your product And and people can actually spend their hard earned cash on it They can say they want to buy this stuff Until you actually say well, that's fine. Can I can I please have your credit card? That's where where it's it's The the rubber meets the road that's where it gets gets interesting someone said MVP So there there is there so how much time do I have six minutes let's do a quick a six-minute MVP rent The first thing I want to get out of people's mind is there's no such thing as the MVP The MVP is What I always call the crappiest thing we can get away with Is that right? Does that bring a bell with people? Yeah Don't do what we what MVPs are are our ways that we can learn things that we can experiment with So what we had was like we had 2030 MVPs. Yeah, our first paper prototype Was an MVP? Now here's here's a quick mole. I want to draw Who here knows of the Kano model? So you know about the Kano model? There's a couple of PSTs there to do that's good to know So So Kano was a Japanese researcher And What he was trying to do he was trying to group Product features in different categories And what he did he plotted them on an on an axis on the set of a quadrant where The left here is we've got none of those Yeah, we've got none of this feature and Here is if we have all of them everything This is how sort of unhappy your users are and this is how happy your users are And what he found out is that there's three types of categories the first one he called basic needs and Basic needs if we don't have basic needs I'm very unhappy if I don't have basic needs. I'm very unhappy if I have the best basic needs. I Don't I don't really give shit So the car analogy is bricks Yeah, if I have no brakes on my car, I'm unhappy if I have the best brakes in the world I don't know as long as you're good enough. That's good enough. It's fine The next one is performance needs About performance features and what they are is the more of these I have the happier I am The more I've got none of them. I'm unhappy if I've got lots of them. I'm very happy horsepower More horsepower in my car the happier. I am I've got none. I'm good. I've got lots of horsepower. I'm happy and the other one is what we Call the lighters Now what the lighters are is are things that people don't know they want Like if I've got none of them. I'm not unhappy item So for example years ago, that would have been cruise control on a car Especially so North America or Australia where I'm from where we drive lots of big distances and can actually move Like cruise control is this sort of but I didn't didn't know I wanted it But when someone said would you like like we've got cruise control and we don't know what this is amazing But there you can also see the other thing Is these the lighters quickly become basic needs or performance? Arturia, right? Would I buy a car without cruise control these days? No, I wouldn't no one no one ships a car like cruise control And it works. It's just part of those sort of basic needs So now we have auto park the other or or line Sister or so now we have different features now if you What happens if you do the the MVP thing that I see in lots of organizations right now If you do that MVP thing that you go. Well, what do we have to do? well, what we have to do is Make sure that we have the basic needs up to up to a reasonable level. So our basic needs are here And we need enough sort of of the performance thing that users will actually sort of go like this is okay So the performance needs in difference is what you get People do not care about your product They don't care because it has all the basic stuff It's got some of these performance things, but it's got none of these lighters because the lighters are hard We don't know whether they're where we're actually going to happen. We don't know The feature I was talking about the parental parental control feature. We didn't even know they wanted it We accidentally stumbled on a feature that was a delighter to a lot of these people We weren't even trying To figure out whether that was a lighter or not. So these are tough. These are these are hard. These are uncertain But if you want to use MVP's what MVP's are really meant to be is finding the lighters This is where you should Use your MVP's and the thing is Even if you have crappy MVP is of even if you do MVP's you cannot have crappy MVP's If you built a crappy MVP You will never know whether your users aren't buying it because it's crappy or Because they're not interested in that the lighter. So you cannot ship crappy MVP's You what so the thing to do is how do we ship less? How do we ship less and that's an interesting question but What we ship has to be good it has to be high quality Otherwise, we have no information on Sort of what it is and isn't you cannot ship And expect people to buy crappy products Even if it might or might not have some of these the lighter Thank you very much Greatly appreciate it. I'll be around if we want to have some more chats