 Hello, everyone. Welcome to the webinar today. My name is Chris, and I'm a senior manager of product management at Unity. Prior to this role, I worked at Atlassian and Autodev. Today, we're going to talk about our favorite topic, prioritization. And I want to introduce you guys a series of prioritization methods that can be used in various cases in your day-to-day life. Before we get started, I would love you to take a moment and then to think about what is a prioritization method that you use most often? And what are the occasions that this method works really well? And what are the occasions that it doesn't? What problems are you trying to solve during that prioritization? Is that always a conscious problem, or is it sometimes a vague problem you are trying to solve? Or are the problems always the same? Or do they vary between these different situations? I'm going to pause here for maybe one minute to give you the space to think, and then we will resume. Alrighty, let's take a look. So I'm sure most of you have some of the favorite prioritization method in mind. And then here are a few of those common ones that we use. For instance, that ranking, I'm sure, you know, this is the most basic and the most commonly used approach that many of us use on a day-to-day basis. Feature bucket, Moscow, biofeature, value versus cost. If some of those fields are faring or unfamiliar to you, that's a little okay. We're going to go through some of those, but these are some of the most commonly used methods in the industry. So I ask you to take a moment to think about these, because I want you to revisit when you use that part of the method. Was that a conscious choice or not? Meaning that did you use data to support your part of the nation? How was the decision made? Was that something that you could make the decision after that part of the nation? Or did you have to incorporate opinions from other people? Or was it occasioned that it's completely determined by others? And how did you consider dependencies? Did you need to consider dependency at all when you're prioritizing? And after you prioritize those features, who are you communicating with? And how is that communication done in your situation? So depending on the answer of each of those questions, you should probably pick one of the predation methods on the list, or potentially others. In many cases, what I realize or what we realize is that the choice of a predation method is not conscious. People call it defaulted to the ones that we know well and then keep using it consistently. There's nothing wrong about being consistent. However, when your situation varies, especially when your project or your product and your stakeholder group changes, you should consider using something slightly different. It turns out there is this periodic table of predivation methods. And this periodic table gives us a framework in terms of what are the factors we should consider and when we're considering them, what are the methods available in our tool belt to choose from? So on the horizontal axis, this is the qualitative measurement, meaning internal versus external. This is talking about your stakeholder group and also talking about how your decision is what we made. Is that going to be completely made by you or your small team, or is it something that's going to be completely made by other people or outside of your team that's internal versus external? On the vertical axis, this is about whether you have data to support your decision-making. Do you have quantitative data, which is high up, versus do you only have qualitative data, which is lower on the y-axis? So you can see the ones, the predivation method, I called out in the previous slide, they reside in different places on this periodic table or on this matrix. For instance, stack ranking, future bucket, those are very internal and qualitative predivation methods. They work really well when you can get into agreement and then you talk about their relatively importance among themselves and then give them label. Versus some of the methods like Moscow, it's very external. It works super well when you're communicating with people that outside your team or even sometimes outside your company. And some other methods like biofeature or value versus cost would require you to do upfront investment in getting data, getting quantitative measurements before you can get into predation. So we're going to keep using this periodic table through the session and I will walk you through a few of those examples and then why they reside in this relative location on this matrix. It's not to say that you cannot use stack ranking for your external stakeholders. You can't absolutely do that. The reason why they reside on this table is to say that in many cases or in most cases, they would fit better with search and characteristics. All righty, let's take a look at this example. So this is the case where I am a restaurant owner and I want to start a restaurant website and I have a list of wishes I want to accomplish by doing the website. So I want to have a text view of the menu. I want to add images, gallery views of the menu if customers prefer. I want to have the ability to change the menu items in my admin panel without having to engage another designer or other people. I want to have the ability to feature items on my menu as today's special. I want to incorporate an online ordering system for takeout and I can't have a text-only contact us page if customers want to call me. But also potentially if we want to go fancier then I have a contact us form routing the submission to my email so I can go back to the people who ask questions online. And if I have people staffing my online contact then maybe we can't even have an online chat. I can use the business contact, business messenger or I can use something like Intercom. So this is the long list of wishes I have from my website. And how do we prioritize this? In this case, most likely as a restaurant owner I'm not going to have technical expertise to determine how much they cost and whether those are feasible or something like this. I would work with someone who has better experiences have better knowledge about beauty websites. So the way I communicate my requirements is to say some of those are must have some of those are things that should have some of those things that we could have or those are not to have things and some of those things are too difficult based on my conversation with other people so that I would have decided to postpone them so I'll label them as won't have. In this case you can see that the text only view of the menu is a must have otherwise there's no way of showing my menu. But the images in the gallery view of the menu is the should have where today's special is a could have it's nice to have if I don't have it it's fine people can always call me and at this point I decide I don't want an online ordering system maybe because I don't have people staffing my digital orders. So in this way this is a partition message we talked about external meaning I'm communicating my priorities with people who are outside of my team in this case I'm communicating with an expert who is going to help me do my website and then this is towards the qualitative measurements meaning that I use judgment to determine what are the things that should have what are the things that could have what are the things that must have I didn't have data to support all of those but I used my judgment. So this is the Moscow method that I think many of you are already familiar with this is more of a recap. Now when we get into this example we can see how different they are. So in this case this is an online furniture store. So let's say I am operating an online furniture store and I have a list of enhancement that I want to implement on my store as a product manager I want to do consider how I prioritize among these features. The first feature is the bought together recommendation so one customer is looking at a product then we show them a list of other products that's commonly bought together a typical recommendation feature and then we can do a follow-up email after card abandonment. It shouldn't be card, but card abandonment if customer add it up to card then we can do a follow-up to say did you forget to check out things like that and then we can introduce a functionality to allow customers to compare features between different items like the bat frame or my cabinet what are the differences between them. We can give the ability of adding items to a wish list so that I'm not ready to buy but I can add them to my wish list and we can add a feature to allow people viewing the furniture in their own rooms using augmented reality. It's a fancy feature that can work on their phone by leveraging the camera. We can introduce a 4-hour delivery window notification instead of saying that your furniture will be delivered today we can give them a window of 4 hours so they can plan better when they should be home receiving the furniture and then we might want to increase the website speed and make it easier to browse for customers and finally we can implement some tracking on the catalog. So this is helping us get more data get more insights for future operations. So if you look at this list what's special about this list is that it's a list of many things. They belong to different categories they talk about different natures, some of those are customer facing, some of those are not some of those have a higher impact and some of those have a lower or less obvious impact. So in this case how would you go ahead with this one? There are different methods you can choose from but for now I'm going to use feature buckets. So I'm going to assign labels to them based on the discussions and based on the judgment I have. For instance I'm going to call Altogether feature as a metric driver or metric mover. So what that means is that this is likely to increase our metric. Whether it's revenue or something else that we care deeply about for OKRs or those. Similarly the cart abandonment is a metric driver. Some others are less about metric driving but more of the customer for instance adding items to the wish list it's very obvious that it's not going to bring immediate revenues but it's helping customers to get better browsing and management experiences for reviewing and looking at your product and this is an area when we label it as a customer request typically it's something that we hear the customer sentiment very often and that we have enough of quality feedback from customers telling us that they need it. So for now I label them as customer requests. Those are the features customers would expect typically on your website. And then there are also things like viewing your room like the AR and the 4-hour delivery window. These are the things I label as customer delight. Which means customers are not expecting it today. If you introduce them it's going to bring a surprise in a nice way to customers. We call them delight but if you don't do them nobody's going to complain saying you don't have a viewing your room AR feature. Typically you don't have it when you're browsing websites like Amazon. And finally there's a bucket called strategic. These are the areas of work that may be harder to match to it right now. Maybe those are infrastructure related or maybe those are the things that would take you a long time to get. But if you don't start right now then you're missing the opportunity window. So you can label them as strategic but those are the things you do want to do but they don't have an obvious payoff right away. So when you label it this way think about what you're communicating right. So in this case you're communicating how you're categorizing your stuff and then whether they have impact to your revenue to your business offline metrics or whether it's potentially going to satisfy the customers better. Or are there things that you want to do as a strategic importance. So in this method you can see that feature bucket is more of an internal and qualitative or an additional method. The reason why the qualitative again is because I use my judgment. That judgment doesn't have to be my alone but it's still judgment. I might have consulted a few people but eventually all of us use judgment to say this is either a metrics or this is a customer request. I didn't put any data to back it up to say the metrics mover is going to increase this many revenue or whatever right. So you can do that but still this judgment of labeling them into each category. And this is also an internal method because if you think about this who you're communicating this with. You're mostly communicating to your internal team explaining to them why we're doing this and then how important they are and then why we're doing it now. You're really typically communicating this to your engineering team or to your design team as we're very close stakeholders of this. So that's a feature bucket and then it's also very commonly used for an additional method in the industry. The next one. This is an example where we probably encounter more often in personal life or in consulting. So let's say I want to remodel my kitchen and there are many things I want to do. The new appliances, new cabinets new floor tiles or hardwood floors new ceiling lights, new granite countertop and I want to paint down the walls. So my contractor gave me a project for each of it and then they would add up more than what I can afford. I only have a budget of 1500 but these things would add up more than what I can afford. So in this case how would I choose? And typically we would choose it by playing some scenarios. Plan A I can do my appliances with new paint that would add up to 15. Plan B, some smaller items like tiles, the the granite countertop, the new paint. Plan C, cabinet tile and paint, nothing else. Plan B it's a little bit over my budget it's a hundred over my budget but I think I can stretch it a little bit by having some of those stuff. So while you're looking at this example it's really important to think about what you're communicating about. In this case you're communicating about your preferences of this. Yes I'm making a choice but the choice of making is communicating my preference. If I choose plan A what am I really looking at? I'm communicating my preferences is to solve the big expensive stuff. Versus if I choose to plan B what I'm doing is the tiles the ceiling lights the granite countertop those are the very visible things that I probably would use or interact with on a daily basis. I'm communicating with a different set of preferences. So versus plan C I might be indicating that I want to get really things organized or maybe I am part of the thing that I can see versus plan D I can clearly show you that I'm willing to stretch my budget a little bit. So typically while we're using this method called buy me a feature when you're doing it with your customers you are soliciting or eliciting your preferences. So in this case because I'm doing a project for myself just to do to remodel my kitchen so it's pretty easy this way but in many cases in the industry when we're doing something like this we can't end up not either doing plan A, B, C, D or go with the itemizer stuff. We typically use this method to understand what customers value more and use that to design a different offering for instance a lot of those subscription offerings when we're trying to determine what to including tier 1 versus tier 2 then we can use this method called buy me a feature to prioritize among customer requirements or customer needs. So this one again this is a buy it a feature as you can see do we use data? Yes we use the data. How was the decision made? The decision that needs to be made by both external and internal because external is providing the data external is saying what item costs what how much and I am deciding how I want to come up with a plan so that's where buy me a feature is in between external versus internal it's more leaning towards external the part they are determined by external so in the real case for building product that the cost might be determined for instance your team would tell you or some other team would tell you that hey this is the cost thing maybe you know three story point or maybe this is the cost thing eight story point and that you can choose between those ones and this is the little bit quantitative right so we do have the data supporting us on the how much each item cost but it's not the most of quantitative because we only got the cost we didn't get the importance from measurement if we can both get an importance and get the cost then you would go up to a more quantitative measurements but this is a fitting in the middle buy a feature is also very commonly used not only for product validation but also for in general interviewing with customers and getting their preferences so with with those straight examples we just walk through I would love you to think about what are the common challenges we have in these prediction methods so common common challenges would include things like we don't have enough of data the data is insufficient sometimes we don't have data at all so we would have to choose a internal and quantitative method instead of any quantitative right so it's actually when you're doing MVP and you're under a very tight timeline that typically you don't have enough of data there you can do market research in some cases but not always alignment in most cases we need alignment with both internal and external stakeholders which is really important what are the methods that helps you to do do you have more weight internal like this or do you have more weight on the external resource external stakeholders that's another challenge and a factor to consider how do you balance between those alignment and we also have blind spots sometimes we fail to consider those measurements or factor that's not revenue or non-engineering for instance you know we talk about fight speed fight speed is super important for customers if your site or your page takes I would say 30 seconds to load up think about the impact most customers would just return they would just abandon your page upon landing because these days most customers only have that much patience they will not wait stay there and wait for a page to load so we all have some blind spots and in other cases some design factor can easily either increase or decrease risk so we don't want to we don't want to neglect those information project nature varies so the reason why I showed you all the three different examples is a way of illustrating different projects have different natures and then depending on your particular situation you should probably consciously choose a presentation message instead of using a consistent one that you have between you working on the product that's only for your company and all your customers are your internal employee versus when you're working as a consultant for external companies then your presentation message probably should vary because you're communicating different things decision makers are all very different and that part here for determining those parties would be different as well so the last method I want to show you is my favorite it's an adaptation of value versus cost this method is called importance versus difficulty the reason why I use it more often than any of these other methods is because it's the framework that helps me balance between those challenges so when between the time I have sufficient data versus the time I don't I can use importance difficulty between the time I have to I have very small team I'm completely just aligning internal with my team versus the time I'm actually aligning and soliciting parties from external parties I can use importance versus difficulty it's also a method that you can include different functions from design from finance from sales from marketing they can all be included if you want to go fancy with that to help us eliminate those fine spots and then finally all of those factors are saying that between different project natures we can use this framework and then you can do it up or do it down depending the available factors in your project so I'm going to show you how that framework works and then do a little bit of exercise to show the logistics and then to get an idea how you can do it up or do it down so this is called importance versus difficulty or in the previous period table it's called a value versus cause same idea it's a different naming so this method is an interactive group presentation method it's important because it's interactive you have to do it in a live session it doesn't work that well when you're doing it asynchronously and if you're doing it offline it doesn't work that well it also doesn't work that well when you're doing it by yourself if you are the only person who decides on the priority then use some others like stack ranking feature bucket that tends to work better but when you have a list of stakeholders you want to have an interactive session that importance versus difficulty work super well this also focuses on understanding and alignment what that means is that during this interactive session you're trying to truly understand each other's point of view why do you think this is more important you are putting the potential questions conflicts of doubt on the table and you're trying to also solve it and rely on the priorities during this session so that's another factor for this method and similarly you can like I mentioned you can use it really really straightforward as simple importance difficulty matrix or you can be really sophisticated you can have multiple factors and that one your project gets really big there are times that I had to do this exercise with people with a team of 30 or 40 and this do work it can definitely scale up and let's take a look at one example using this method so let's try to revisit this one this is the case 2 we talked about using feature buckets so previously we talked about all those different work items and then labeled them with the bucket in the feature bucket method and now we don't want to use the feature bucket anymore in this exercise we will use the important versus difficulty matrix so let me switch my browser tab and show you how it looks I'm going to put all of those bullet points into thickies in my canvas alright here we go I'm using this tool called mirror you can use any whiteboard or you can use mirror either way there are different tools you can do that I'm also using a template from the luma institute of design thinking methods you can use and then we have this really good importance difficulty matrix template that you can borrow if you're on mural so in this example I put all of those features from the online furniture store project into this canvas the card abandonment comparison wish list view in your room delivery notification type B tracking and ball together so now we'll have this list what I want to do is eventually I want to be able to plot all of these into this two by two matrix on the original 8th it's talking about the importance of them on the vertical it's talking about difficulty of them so once we plot them into this two by two what you can do is that you can either go by the quadrant or you can draw for to determine what are the things you want to work on first I'll show you about how to draw the curve but you can do it in other ways as well so let's take a look at how you actually put all the different stories into these sections on the matrix so the first thing is that I will need to do a stack ranking of those by importance so essentially plotting all of them onto my horizontal so let's say I'm meeting with you guys this is in the meeting and it's an interactive session each time I only want to move one sticky and I want to ask around and hear all my stakeholders point of views in terms of the relative importance between them for instance I'm going to clone the sticky and then say between the bought together feature and let's say catalog tracking which one is more important in your point of view and why in that case some people might say bought together is more important because it's going to bring more revenue some art and people would argue that catalog tracking is more important because if we don't have it it's garbage and garbage out but after a little bit of debate we can always battle between these two where we should land relatively between those this is the case where we are already prioritizing on this even though I don't have perfect data in this case I don't have the numeric measurement on these but I'm still using a quantitative hybrid qualitative measurement on this to compare the relative importance between them now I move on to the next one saying that side speed is that more important than catalog tracking less important and then we can debate a little bit and decide that we want to put that one in the middle this way and then go with a less further side of vacation and then talk about where does that fit each time you're only talking about one sticky and in this case I'm putting it to the very the least important among these four and viewing your room it's also less important the wish list let's say I got some data from customer things they really wanted so I'm putting it over here and then comparison feature we also really wanted but we thought that this might be behind the side speed because if your side speed is really slow then people wouldn't be using your side that often to do a comparison so those are the reasons and then card abandonment these are something that can really bring up a lot of revenue and I'm putting it over here so in this exercise what you can see is that my revenue and customers the revenue comes first and then it's customer I'm deported I think some of the delight in the previous presentation example whether I call it explicitly during this exercise to say this light or not you can still understand like the AR feature is kind of delight in the previous method so now I have that grant this list of buying for instance so what I'm going to do is that I will move that onto my canvas you only do you only need to do it in the relative locations so what I'm going to do is that the align them a little bit align them the top and just do that okay looking good let's do this again using difficulty so coming back to this area the same stories or stickings let's try it by difficulty again coming from the bottom bought together feature is that more difficult or less difficult compared to the catalog tracking I think catalog tracking is less difficult than bought together because it's a larger engineering effort so I'm putting it up here side speed is that more important more difficult or less difficult it's definitely more difficult than tracking because this needs to optimize quite a few things there might be low handing through we can do such as combining the style sheets whatever so this is the reason why again this is the interactive process you want to hear your team you want to eliminate your blind spots as much as possible by hearing people through and then just because you're putting them here very visually people would raise their hands I think I did agree and this is why I disagree so this interactive group exercise is going to give you that room and that space to hear other people out in that you will use a very concise period of time to do a level debate so I'm just putting it here for now as demonstration and delivery notification I would say this is super difficult because the four-hour window requires a lot of third-party tracking and dependencies so very difficult viewing your room really really it's an AR and new technology my team hasn't even done that before it's the most difficult among the wish list looks fairly easy so I'm going to be putting that above flight speed because it's just adding a lid it feels easy and compare it then this is probably more difficult than bought together because you have to do taxonomy and metadata management between different products and then you have to determine why you can do comparison when you cannot find a like hard abandonment looks pretty easy actually because capturing the data and send out an email that's to you maybe more difficult than a wish list okay putting it here alrighty now I have another list this is ranked by difficulty the higher it is the more difficult the story is now let me just put it over here move it into this matrix again okay back here I have the same story both ranked horizontally and vertically what I now need to do is just I need to just move all of those stickies either or another way or vertically the position corresponding to our ranking over here so let's debut in your room I'm going to just move it over here and get rid of this one get a re-notification I'm moving it over here and analog tracking where is that okay here it is getting rid of this comparison feature is here let's move size speed here and the wish list here and party bannerman is right over here and both together is over here alrighty so now we have ranked at a relative importance and difficulty of these ones so you have a choice to make now depending on the nature of your project sometimes you want to draw a curve this way let's say you want to draw a curve this way to say these are the most important and most difficult ones, eventually you're a big rock you want to tackle them first and then you want to draw your neck to curve this way so that you can move on to the neck the wave of things so that's one way of doing it the other way of doing it is from the lower right corner let me try that as well sometimes what you want to do is to say okay we want to optimize the go to market, I don't want to tackle those big rocks actually the most important ones I probably want to tackle them last so or not last but on the second wave MVP are typically that type of size, what you want to do is to think okay let me work on these ones first, I want to draw a curve here the most important but least difficult ones and then I want to move on to the second here which is the more important but also more difficult ones and finally we want to work on the least important but most difficult ones or maybe you never go to this area because those are just so difficult and you would have some rolling items into the first tier and second tier so all of those are the different ways you can do you can draw the curve and this is the exercise that's a little more artistic than scientific but you get the point right so you can determine whether you want to focus on the big rock versus the low panning group. Alrighty so that's a quick view about the partination method for importance difficulty, I just to show you how to look and then a quick recap about this one, the important difficulty matrix is an interactive group partination method to focus on understanding and enlightenment it can be really simple and sophisticated so in the example just to walk through, we only look at the routes of importance for based on the group exercise right so if you have a group of four or five it works really well for a stress talk but what if you have a group of 30 but in that case what do you want to do is that you can divide importance into different factors and have different sessions for instance you can have a session with sales and marketing team again that smaller group of five to talk about the revenue impact for your importance and then you can have another section with your designer with your customer service team to talk about customer impact for that so for the importance you can have different factors and then have different smaller sessions and then roll it up as long as you get to agree to say okay revenue is one third of the weight and then customer value is one third of the weight and maybe something strategic is one third of the weight as long as you can agree upon that then you can still come up with a stepped ranking list of importance after you do those separate exercises similarly difficulty typically you want to consider what is the cost of building at what is the cost of doing operations meaning maintaining it and operationalize that and what are the cost of doing change management some of those features are not difficult to build but it's very difficult to socialize inside of your company just because of the sheer amount of people that's going to affect so you want to consider those as your difficulty again dividing that into smaller sessions and with different stakeholder groups can give you the ability of making it more sophisticated and then ramp up this method so that's the reason why I really like this importance and the difficulty matrix and also like I mentioned depending on whether you have data we have some data in customer or some data in revenue area you can use it for a particular factor right so for some other area when you don't have that data as long as you can agree with that smaller group agree upon the relative importance you can still get the ranking working so this is a pretty versatile approach alrighty so for this session some of the key takeaways there is no one size fits all prioritization method like we looked at depending on your team setup your own company situation and who you're prioritizing with you should consciously choose a method that fits your situation best two, when quantitative data isn't available qualitative data can also help like the stack ranking with a smaller stakeholder what this does is that it makes alignment with your own experts for instance you can get a really good idea about their relative importance or this relative stack ranking for revenue with your sales and marketing people typically that smaller group will give you a really good idea how that works so reduce your risk by using that qualitative data and then prioritization exercise is about understanding and alignment in every case you really walk through think about who you are communicating what information do you need from the other party and what information you're giving back and how the decision will be made so that's a very important factor for you to choose and for the addition method so as a next step I would really encourage you to do a few things one, now you have some more knowledge about different ways of prioritizing so pick one of your current or your passive romance and fill the items on it and then think about whether there is a different prioritization method you would try with it and if you do then do you have enough data what your stakeholder looks like and what method would you choose differently and then think about what is your primary challenge you want to solve for the predation are you two trying to optimize the resources are you trying to get your stakeholders aligned or are you trying to do something else satisfy your customers depending on your need and you might want to choose a different method and finally if you find this helpful please share this webinar with one person just the one person at least one person who can benefit from it because I feel the more you align on how you prioritize the more you have a common language approach to prioritize what we are on producing our roadmap alrighty this is the end of the webinar and thank you all for attending it I appreciate your participation and I hope this was helpful have a good rest of your day bye folks