 Everybody, how's it going? I'm Chris Morrill. I'm a Senior Product Manager of Technical Products with Amazon Alexa. I've been at Amazon for about four years and I have shipped consumer facing products in both Alexa Entertainment as well as Alexa Shopping where I currently am running the all of Alexa's book related shopping capabilities as well as things like if you ask Alexa to read a book or if you ask Alexa to recommend you a book, those are all part of the product portfolio that I manage which is why I'm really excited to be here today to talk about the power of the prototype which is something that has helped me as I've gone through my career thus far in product management. Let's kick things off but checking in with our agenda. So introduction, great news is that that's already complete. Next we're gonna move into a little bit of context setting and then jump into three different types of prototype techniques that I've used in presenting different types of products across my career and then we'll end with a couple of takeaways. All right, so some context setting. I learned best through application and so the focus of my talk is gonna be on applications and less on theories and the output of this is gonna give you a couple of ideas to try the next time that you happen to be in this particular phase of product development. I will at full disclosure is that my talk is not mutually exclusive nor collectively exhaustive so do not tell my business school. There are many different frameworks that you can use for prototyping. There are many different times that you can use prototyping from the very start of the product development cycle all the way through software development, shipping and then iteration post launch. The purpose of this talk is not to catalog all available prototyping methodologies. It's just to tell you my experience using prototypes which is reflected here in Merriam-Webster's dictionary definition so it doesn't help us at all here. It's first full scale and usually functional form of a new type, no, that's not what we're talking about here. What we're gonna talk about today is prototypes as part of the introductory phase. So that phase when us as product managers are out there trying to communicate this idea to stakeholders, we're trying to add evidence or weight to our hypothesis of customer value and we're just trying to or we're trying to prove product market fit. It's prototyping in those, the first two phases during product development and introduction. And my experience so far is with prototypes is that it using a prototype helps others understand the vision. It doesn't matter how well I am able to speak about a concept or write about a concept, a stakeholder actually seeing the concept helps them understand in a way that talking about it, I've noticed doesn't. The other thing that it helps you do is or it has helped me do is identify hero use cases in P0 features, which at the beginning of product development, especially as we're, at least for me as I'm writing a, Amazon, the PRFAQ, which is how we fund projects. That's what every product starts with. As I'm writing through that, the specificity of what is the MLP look like? What are the features that go into that? What does the customer absolutely have to have? And what does that customer journey look like? Prototypes are a great way to drive to specificity with some of those very questions. And then finally it adds confidence to product market fit. It can help me gather information and it has helped me gather both quantitative and qualitative information to help add data to the hypothesis of whatever customer value opportunity I've discovered. So the three approaches that I have used, I've sort of, there's the paper prototype, which I think is fairly straightforward. Turns out that if you do an internet search for paper prototype, there are plenty of both design and product management websites that have assets for paper prototyping. We're gonna talk about some really low fidelity stuff that I did. Then we're gonna talk about the hackathon special, which is one of my favorite things to do, very low fidelity as well. And then we're gonna talk about the video supercut, which also can be either scrappy or spendy, depending on your budget and most importantly the risk level of what you are trying to prove. So let's go back to 2018 and let's talk about Choose Your Inventor. So as a fresh product manager, I had just started and the scope of my responsibility was to ship the first three Choose Your Inventor books starting with the Abominable Snowman and turn those into interactive audio experiences on Alexa. We were partnered with Audible, so we had full access to their number of very talented narrators. And we were gonna take that audio, take that narration, turn it into something you could interact with with your voice, which was all well and good. But when you get into the specifics of, well, what does that pattern look like? Well, sure, the narrator will read through a segment of the book. In fact, so how do we take this and turn it into something that goes on a device? If we look at here on the book, there's no prompt that says, aside from something that says, turn to page 18. Well, how would we represent that without a visual? Would Alexa just say, turn to page 18? No, that's not gonna happen. We're gonna add some abstraction there to make it more delightful. Then what does Alexa say? And most importantly, what will the customer respond? How will they respond after Alexa says, do you wish to turn to page 58 or turn to page 59? In the case where the story branches, it is after all, choose your own adventure and each of the books advertises at least 30 different endings. We had to build an interaction model and I needed to specify the requirements for that interaction model, but we didn't even know what the interaction pattern would be. We didn't know how customers would respond to these navigation prompts that we had just written and iterations were expensive. So every time we put a concept in front of a customer with some mocked ideas of what we thought they might say, if we were wrong and the experience broke, we wouldn't get the feedback that we needed to continue iterating. And if we didn't get enough customer feedback, each of those iteration cycles costly in terms of engineering and time. So, well, this is where paper prototyping comes in. This is a definition from concepttesting.com. Design teams, they draw sketches, low fidelity, say bias towards cycle time. So what the heck did that look like? Well, that looked like me and the designer writing up some scripts. And then we went and did seven user studies, which usually involved us patrolling the hallways of Amazon looking for someone who would make eye contact and pulling that, asking them if they had an hour that we could use to do a user study. In the context of a user study, it'd be three people in a conference room and I would pretend to be Alexa and I would read off a script. So it would be me sitting here saying, he thanks you profusely. I slipped into the gully two days ago, he explains, and I was beginning to think I'd never get out. Do you go with FATON or set off to the Palace of the Sun? And the customer would reply as they would reply to the device, that was sort of how we instructed them. And then we would just take note of what they said. And after the second or third paper prototyping session, which absolutely started as awkwardly as you would imagine, but what was interesting is that after the first couple of interactions, the customer relaxed and just started responding as normal. We coalesced around a set of 80%, around an 80% solution. So for each of those types of interactions, we identified about 80% of what customers, how customers would respond. And that enabled us one to tune the interaction, tune the questions, the prompts that we were giving customers after each one of these was read, but it also allowed us to generate a allowed me to write the requirements for the spread of customer input that we needed to handle. And as a result, so here's a great little TechCrunch article from the launch of the launch back in 2019. But so at the end of the day, the value was I was able to write more precise requirements which meant we spent less time iterating in QA because we found fewer bugs. We shipped overall a higher quality experience that handled a broader range of customer input. And we shipped it faster. It was all because we were willing to walk around with a paper script and just grab someone. There was no fancy customer recruiting. I wasn't overly worried about segmentation and the sample size. I just needed to go from total unknown to a closer known. The point is paper prototype is low fidelity, shockingly low fidelity. It can be as simple as something you have written down. It can be on a whiteboard. It can be a series of index cards. The point is to get customer feedback so that you can start at, or for me, so that I can get really clear on what are those requirements exactly? What is P0? What is P1? P1 so that when I hand that off to the engineering team to be built, I have higher confidence that the requirements I'm giving are the actual requirements that need to be built. And it's nothing fancy. There are very, there are higher fidelity paper prototyping tools like Figma that you can usually create very interactive prototypes. But for my purposes, when the unknown was so broad, all I needed was a script and customers willing to pretend. And the real value of paper prototyping for me comes from the observations. So excellent. Talked about paper prototyping. Now we're moving on. Next situation, Starfinder. So in 2019, I'm pitching a new type of entertainment where the hypothesis is an audio book means choose your own adventure. We, I had data that showed that there was a sizable number of customers who would enjoy more agency than an audio book, but they didn't want something as actively engaging as a video game or watching something visual. So our idea was the interactive audio adventure. We knew that we needed intellectual property. So my business development partner secured a meeting and I knew a prototype wouldn't convey that the concept adequately. We needed something a little bit higher fidelity. Well, that's where what I call the hackathon special comes in. So this is a single hero use case where everything's hard coded. If the wrong person sneezes, everything goes wrong. I just, but the hackathon special is what is that absolute minimum hero use case that you can't, you and maybe a developer can knock together in three to four hours that only works if you use it in a very specific way, but it works and it will convey the value. So let's talk about this hackathon. So I went back and my designer and I, this is actually a screenshot from the tool that we were using at the time to mock this out. We built a happy path in about four hours. It represented less than five minutes of gameplay because I knew that I would have a very limited time to showcase the functionality that we were proposing to this partner or to this collaborator. And, but it was key that we be able to showcase it on a device to get them to envision the concept. I shamelessly bribed a developer with Shake Shack, both a burger and a chocolate shake to build in dice rolling because a fundamental to the IP partner that we were pitching was this idea of rolling dice as part of a tabletop role-playing game experience. So we had to have dice because that was an essential part of the experience. And then we spent a lot of time making sure that it functioned on a device only in the use cases that we needed. So we got there. This you can imagine, this is what I see is I'm walking into, I've driven out to Redmond, Washington. I am walking into the headquarters of Piezo, Inc. And they own the rights to many, many iconic tabletop role-playing game franchises. And this is the conference room that we're taking the meeting in and walking in, this is what I see. All of this history and intellectual property. And so we sit down. My business development partner goes through the pitch deck and then I plug in a device and we go through the demo. Six weeks later, we signed the deal. Why? And I think we may have gotten there eventually, but in that meeting, we played the demo and you could see the people, the folks around the table start to lean in as they start a brainstorming with us. If they started to think about, wow, this could be something really cool and asking us questions like, well, could you build in this? Could you build in a dice rolling mechanic that also took into account an armor class and another modifier and well, could it save your state over time? And what about monetization options? It gave them enough of the functionality to be able to inspire further conversation and for the folks around the table to see, well, this is a really interesting opportunity. This was the result. So this was a article in variety because obviously we needed a voice cast, stellar voice cast. So we recruited 12 exceptional narrators from Audible as well as Nathan Fillion. And we ended up shipping seven interactive audio experiences and ended up proving the hypothesis that there were indeed a number of a business material from a business perspective, number of customers who are willing to pay a premium for this type of interactive adventure. And the point of the prototype in this case was that I needed something more than a paper prototype because going into a pitch meeting where I had a very limited amount of time where I'm trying to convey the value of the product, I couldn't ask them to pretend that I was Alexa. I needed something better than that. But the key was that it was focused on the happy path. If I had tried, if I had deviated from any of the pass that we had, that we presented in this five minute adventure on Rails, the experience would have broken. But it communicated just enough of the concept to where everyone in the room could see the potential value which then led to additional conversations. Now, this was not sufficient to ship that project. There was significant work that went into establishing the business opportunity, negotiations, establishing a schedule in the actual execution. But in the ideation phase where what was important was my ability to convey both the business opportunity but also what could this be and what could it unlock? And that was what this hackathon special did and I've used this multiple times in across different scenarios. So that's the hackathon special. Now let's talk about the video supercut. We're gonna talk about this digital unboxing experience. So it's late 2020. And I'm working on sort of this intersection of commerce and digital goods. And I've got this observation and it just hits me one day because I ordered this physical book. I was looking forward to it and then I got the box and I opened it up and I was like, yeah, I was really looking forward to that and realized there's really, there's no synonymous experience for that in when it comes to digital goods. And in fact, when I order a number of digital things it would be kind of cool if there was some type of experience that was synonymous to the excitement that I get when I open up a physical box because I've ordered something that I'm looking forward to. So the hypothesis was that delightful digital unboxing experience could be a differentiator that might lead customers to purchase one format over another or may lead them to look for a badge when making a purchase decision. And so on, the hero use case of pre-ordering a video game for the Xbox. And in this particular case, the reason for selecting Xbox is because Xbox has an existing Alexa skill and there are people who use Alexa to control their Xbox. So that we had some existing tailwinds with that specific use case. That was sort of our hero use case, our beach head. Now, what I need to do is kick off a discussion with Microsoft. And in this case, that brought me to the video supercut. So in this case, prototype doesn't have to work. And the primary tools, it's a cell phone, trim tool and then, you know, whatever I need to sort of show what that functionality would look like. So in this case, a monster load of Alexa devices, one with googly eyes. I'm not still not sure why that is, but that's what happens when you have children and then working from home. The point with the video is that in the case where the prototype doesn't work and I can compensate for that by using the trim tool and stitching different video clips together and it still communicates the end-to-end experience. I'm gonna come back to this though because this has bitten me a couple of times. So I knew that I was gonna go the video route as a demo to show some of the folks from Microsoft as we went into a business meeting with them. And here's the video that I showed them. Alexa, open shopping menu. The owner Exosuit is now complete. Even though this technology will save humanity in the Lord to come, it all means nothing until you step inside. Halo Infinite will release on October 21st, 2021 and is available for pre-order. Because you've used the Xbox skill, I see you have an Xbox Series X, the standard edition for the Xbox Series S is $59.99 and will be delivered digitally to your console on release day. Would you like to pre-order it? Yes. Fantastic. You have pre-ordered the standard edition of Halo Infinite for the Xbox Series S. It will be digitally delivered to your console on release day. I can send a notification to your devices and mobile phone when Halo Infinite is ready to play. Would you like that? Yeah. There was a problem with the requested skills response. So a couple of things to point out in here. So one is that obviously it does not work end to end. In fact, we saw the ridiculous latency at the very beginning. We saw it referred to something called the Xbox Series 10 which has never been released to the best of my knowledge. That is because with Alexa's semantic understanding, it interprets in X after a regular word as the Roman numeral for 10. So instead I needed to, there was a way I could have corrected that but I didn't have time to do that. So it says Xbox Series 10. You also noticed that it very clearly failed when I said that, when I responded yes. However, in the demo, I trimmed the latency at the front from the video and then I cut right before the experience failed. I didn't correct any of the small errors like Xbox Series 10. It also then refers to the Xbox Series S at the end of the demo, obviously inconsistent. We got shaky cam, the aspect ratio is not great. The lighting kind of sucks. But the point is that we kick off with an artifact from a trailer that I borrowed for the sake of the, borrowed from video, extracted the audio for the purposes of creating the demo. And then it leverages some an evocative background that's close enough, shows some interest. And then it's good enough to show what it might be and covers a couple of key elements of value that we thought were really important to convey. So what was the result here? Well, prototype it wasn't perfect as we just talked about but it did show sort of what could be and sort of where we were thinking. When we met with the partner teams and this was enough to get the jumpstart the conversation and to move along towards an initial agreement. But guess what? Product ended up defunded after we secured the initial agreement. About two months later, product was defunded, team pivoted. Key here is you can do all these things, get the right alignment and you still won't win them all. So what's the point kind of overall is we've talked about three techniques, the paper prototype, the hackathon special, the video supercut, this is an excellent example of this was recently last summer a paper prototype or sorry, a hackathon special that I put together. It was one day of effort that what we wanted to show was what could be possible with a certain books experience. You can see this is very clearly one of my test devices but the point was just the picture with the, this information on it was enough to show to our partners and to get them to work with us on a longer roadmap. Let's talk a couple of caveats very briefly, which is the more risk, the higher fidelity. So thinking about risk as the amount of investment, the opportunity cost of that investment versus the idea that I'm bringing to the table. Do I need one engineer for three months or do I need 15 engineers for 18 months? The difference in the information that I need to bring to show stakeholders and offset that risk is going to sort of drive the amount of fidelity that I need to have in my prototypes when I am interacting with stakeholders. So, for example, in this sort of on the left here, what we see is this is using the tool Figma. So from the UX collective, if you want to create mockup videos using Figma, you can also create mockup web prototypes all the way through TV television series. They film a pilot episode. A pilot episode is a prototype of what the TV series will be, who will star in it, what the setting will be, and the results, the customer results from that pilot drive, whether or not there's gonna be investment in the entire season or multiple seasons. So it becomes a factor as sort of a sliding scale of how much investment is required to make this idea work, both from an MLP perspective and then as sort of on a broader vision. And there's an art to knowing when to be scrappy and when to spend money. When I was looking at that first meeting with the Starfinder intellectual property holders, Piso, I knew that a paper prototype wasn't going to work because I needed them to quickly understand the type of interaction pattern of the device would provide content and a prompt, a player would respond and that was the fundamental cycle of the interaction pattern. And that we could add different types of content. We could add different types of features around the intelligence that we were able to build into that interaction pattern. But the fundamental interaction pattern was content prompt response. Those three things in cycle. That's what I needed them to understand. And it was gonna take me too long to convey that with a paper prototype and I needed them to hear it instead of have it be read to them. The difference there I think has not been something, it's a judgment call. It has ended up being a judgment call in there of times where I've gotten it, where it has worked in times where it is not. Most recently, I have had on the product that I'm currently working on, there were a series of stakeholder meetings where the prototype that we presented was the result of six months of effort, sizable investment using agency work, as well as leveraging both a two designers and an engineer. Because that was the fidelity of the investment we're asking for from these stakeholders. We had to show them a prototype that was not a hero use case that showed a number of edge cases that was exhaustive in the way that it handled some of the common questions to come up. We needed them to envision what this product would look like over a multi-year arc, not just what would the MLP look like. So that was a time where we spent much more time and a lot more resources in creating a prototype that would help them understand and de-risk their investment. In general, what I have found is the precision of the information that I'm looking for as an output of the exercise is the precision of the input that I put in. Which is to say a paper prototype, there's not much investment upfront because the output that I'm looking for, it's still very broad. I'm not looking for precise outputs with a paper prototype. The outcome of a meeting with stakeholders is more precise, can be more precise, especially the closer that you get to a meeting where the outcome will be a commitment or not. So therefore the precision of the input that I'm providing in terms of a prototype will then go up or down. As generally speaking, what I've found is the way for me to meter between which of these types of techniques that I use and they're not, like I said, they're not mutually exclusive and I use them sort of as I move, I may use all of them as I move through the initial process of product development. Thank you so much for spending time with me today talking about these three different types of prototyping, how I've used them in different phases of product development, how I've used them to communicate with stakeholders and how I've used them to reduce uncertainty in as I've written requirements and as we move through the product development cycle. Feel free to reach out to me on LinkedIn. Thank you again.