 Hello everyone, this is Deepika and welcome to the webinar on stages of product development from concept to launch. Before we go into the detail, a brief introduction about myself. I joined Amazon in 2016 as financial analyst and I joined right after my MBA. Later on, I switched roles within Amazon itself. I moved to program management and in 2020 I transitioned into a product management role. If anyone of you wants to understand how to transition from a non-PM role to a product role, feel free to reach out to me. These are the broad takeaways from this webinar. This webinar should be useful for someone who is a new PM or who wants to transition into a product role and understand the end-to-end journey right from ideation till the time the product goes live. I have also added a small case study in the end which will help us understand the different stages of product development. This webinar should also help you understand how a product manager interacts with different stakeholders during the launch of the product. These are the five stages of product development. This is just the basis of my experience. You can find different versions of these stages, different names for these stages on the internet. This is one of the representation for the end-to-end journey. Stage one is to identify the problem under this. You identify your end customers. You understand the customer's pain points and you define the problem statement. The second stage is to define the how part. How are we going to solve the problem at a very high level? We will use this to get tech funding for the product. Stage three is to define the product requirements in a very detailed way for the tech team. Stage four is the actual tech development where the tech team gets involved. They do the system changes, integration, coding, etc. to build the product based on the requirements. The last stage with the stage five is to test the whole end-to-end workflow of the product. This also has QAT and the final launch checklist. We will go into the details of each of these stages as we go further in the webinar. Starting with stage one, in stage one there are three parts. We identify the customer for the end users of the product. These can be internal stakeholders, mobile app users, sellers selling on your platform. These can be anything, but identify your end users of the product or the customers for whom you are trying to solve the problem. The second stage, and this is the phase where you as a PM should spend maximum of your time. In this stage, it's to understand the customer's pain points. There are different ways to do that. Firstly, you should start by looking at matrix and internal data. For example, if customers are churning, then understand what are the key drivers behind that churn. If the customers are escalating, then what are the major reasons for that? Once you have looked at the data, then to validate your hypothesis or your analysis, you should directly speak to the customer. You can create a sample set of customers, call them directly, identify four or five questions for the customer, or you can create a questionnaire, send it across to all the customers. The idea is to understand the end-to-end journey, end-to-end customer experience. Ask them to share their pain points within the journey. Ask them to rank the problems, rank the pain points, because the customer might have 100 issues, but you as a product manager in the beginning would want to spend maximum time in solving the top critical one first. So this will help you prioritize your product roadmap as well. Then the last part in this stage is to define the problem state. So clearly define what problem we are solving for the customer, how are we going to measure success as well? So define those output metrics, size the problem, size the problem, and you should work closely with finance teams in order to do that and when the numbers for sizing can be in terms of revenue increment, sizing can be in terms of cost reduction. But do estimate the impact which will come out of the product launch. Now once you have identified the pain point and have also identified the problem statement, then stage two is to define the solution at a high level. So this is the how part you will have to brainstorm on possible solutions for the customer pain point, understand and evaluate which one is the most optimal solution. So as a product manager, you should understand that you should not build something for the customer which adds more friction or which makes things more complicated for the customer. So keep that in mind when you are evaluating the solutions and when you are identifying the most optimal one. The other thing which you should keep in mind is that tech comes with a cost. So if there are programmatic ways to solve the customer pain point, hacky ways to solve the customer pain point before you start investing in tech, in a tech solution. Then once you are confident about what you are building, what solution you would want to solve the customer pain point, create a very short one or two page high level document just to describe the broad level requirements which you have for the product. This document will help you in getting leadership alignment. This document you should share with tech team as well. Using this they will do the tech scoping which basically means that they will calculate the estimated bandwidth or tech bandwidth required to build the product. So this is the cost part. You have already calculated in stage one, you have already calculated or estimated the business impact. So using the impact and cost, you can calculate the ROI of the product. And this is the ROI, rank your product. If you have 10 products to be launched in the air, then rank your product, prioritize your product, this is the ROI which is impact versus cost. Then stage three is to define the detailed product requirement. So once you have received leadership alignment on the product, once you have received tech funding for the product, then the next step is to define detailed requirement. And it has two broad level phases. So phase one is long term vision to start with the long term vision of the product and one mechanism to do that is working backward process which is widely used in Amazon. So working backward process is basically Amazon's way of ensuring that we remain customer obsessed. So you start with the moment your product has reached customer hand and you work backward from that moment. So there is one way to describe that moment which is called PRFAQ. So PRFAQ is a mock press release. It has a dummy headline. It has a future launch date. It clearly defines what customer experience you are building, what problems you are solving, who are the customers. So it will help you define the not start for your product. So you should always start from that stage because it will help you visualize what you want to build for the product, what problems, what you want to build for the customer, what problems you would want to solve. Then once you have the PRFAQ ready, once you have the vision ready, then you can use this vision to break your product into phases. And this can be a product roadmap. Phase one can be in year one, phase two can be in year two, and so on and so forth. Then the next step is to create the detailed business requirement document. And in Amazon, we widely use user stories to define the tech or define the product requirement. So using user stories, as a product manager, you will step into customer shoes and walk through the journey of the customer, the proposed journey. And then you come up with user stories. So user stories is basically a mechanism to define the product requirement from user point of view. So I mentioned the standard format for user story as well. So this basically says that as users, I want dash, so I want this so that I can do this. So you have the user clearly defined as the e-commerce, the customers, I want this so that I can do this. So this is a standard template for user story. You can rank your user story. So let's say there's a feature which is a good to have features and you can put that user story as a P1 requirement or a P2 requirement. Once you have these two documents ready, so you have the long-term vision, you have the business requirement document, then you can work on the mock-ups as well. And mock-ups can be part of these documents. So these are, so you don't have to start with a fancy mock-up. These mock-ups can be varied up, these can be handwritten drawings, and then you can work closely with UX, UI, POCs to help you create a better more sophisticated version of the mock-up. Then the last step in this stage is to get partner teams sign up. And this is very critical because this will help you reduce turn for your tech team. So you should review your product requirement document, both PRFQ and the business requirement document with your partner teams and relevant stakeholders. So identify partner teams, ask them to review the detailed requirement. If they have any requirement, ask them to share it in the stage itself before the tech team starts building the product. Then stage four is the tech development. So this is when the tech team gets fully involved. So this is the product requirement which you as a product manager have shared with tech teams. The tech team will create a high-level design document. So high-level design document is the end-to-end flowchart for software development. It basically covers what systems, what databases, what services will be impacted, will have to be changed in order to build the product. So generally the owner of this document is the tech program manager. It can be a senior product manager tech as well. Or it can be a solution architect. So it depends on company to company. In general, in Amazon, generally the tech program manager builds this high-level design document. Then the next step is to create a low-level design document. So based on this high-level design document, the software development manager creates a very detailed out version of the end-to-end workflow. The low-level design document will cover specific details about which component to be updated in the system in order to build the product or in order to do the required changes which the product manager has asked for. This is a very detailed version generally referred by SES when they do the actual coding. And the owner, as I said, will be either the software development manager or someone from the tech team. Then the last step is the actual development. So based on the two design documents, the developers or the software development engineers will do the actual coding. They will do the system integration and the changes required to build the product. The last stage is stage 5 which is the testing phase. This is after the tech team has completed all the development work. Then you, as a product manager, will engage with the QA Quality Assurance POC Quality Analysis POC. There is a dedicated QA team within the tech team who does this. So the QA POC will refer to your product requirement document and basis that requirement document. The POC will create test cases. So test cases will be very detailed. It will be used to test the end-to-end workflow, exception use cases. It will also be used to do load testing just to check how many responses the system can handle in per second and per minute. So you will, as a product manager, will have to review those test cases just to ensure that the POC has covered the entire business requirement which we have shared. Then you will need to align on the exit criteria for QA sign-up. You need to get a QA sign-up. So the exit criteria can be if 99% of the P0 test cases are cleared, then QA will provide a sign-up. So this is negotiable. But as a product manager, do ensure that you don't rush through this process. Do ensure that you spend enough time in this. So budget time for QA before communicating any timeline, ensure that you are taking into account the QA process as well. So generally, depending on the complexity of the product, generally it will take 4 to 6 weeks. Once the QA has provided sign-up, then the product manager will do user acceptance testing. So you don't have to replicate the entire process of QA, but you will have to list down the critical use cases which you would want to go through yourself and validate the experience. So you should create live orders, test orders. You should walk through the customer journey to test the end-to-end workflow and do input the customer parameters yourself to see how the inputs impact the downstream system. So validate the end-to-end journey. And then the PM, if the journey is as per the requirements shared earlier, then the PM will provide the UAT sign-up to the tech team. The last phase is the launch checklist phase of preparing the launch checklist. There's a standard process, at least in Amazon, which we call as the go-no-go meeting. In the go-no-go meeting, we invite all the relevant stakeholders, partner teams, and review the checklist. So QA sign-up can be one of the things in the checklist. UAT sign-up is one thing, tech readiness. So the SDM, the software development manager, will provide the tech readiness sign-up. Legal can be, if legal is your partner team or stakeholder, then someone from legal team will be there. So the POCs will either go on the product launch if their requirements are completed, or they will either ask the product manager to complete those requirements and set up a new meeting. So to demonstrate the five stages which we just discussed, I've selected a sample, a very basic case study. This will help you understand different stages, how we start from the problem, from the idea, and how can we come up with the final product. So we'll start with stage one first, which is identifying the problem. So in this case, we identified that the customers were customers purchasing on e-commerce website. And when we looked at the data, we realized that majority of these customers were working professionals between the age group of 25 to 35, and they were living in tier one and tier two cities. So we created a customer persona profile of the customer as well. And then we looked at data to understand their customer pain points. We looked at the customer escalation data. And we got to know that around 30% of the customers were reaching out to customer care support to request for redelivery of their order. And when we deep dived further, we realized that the reason for redelivery was that customers were not available at their specified location at the time of delivery. And hence they requested for redelivery. So this was one of the major pain point we identified. And then we spoke to a few of the customers, asked them the major reason for this for redelivery. And one of the things which came out was that customers have provided their office address and we were not taking that input into account and we were even if it was an office address we were attempting delivery after business hours. So this was the pain point which we identified then we defined the problem statement. So in this case the aim was to reduce customer contact and you can estimate a number for this as well. So the aim was to reduce the customer contact and ultimately reduce the cost by so and so million. Then in stage 2 what we did, we brainstormed on possible solutions to solve the customer pain point. So I've listed down these 4 possible solutions which we evaluated. There can be more solutions as well but I've listed down the top 4 ones. So one of the solutions to solve this particular problem was to introduce a schedule delivery. So in this case the customer can select a slot, they can select a date for each of their order and as per their slot preference. So from customer point of view when we evaluated this solution this was able to solve their problem and in fact since this was taking into account their preference this was helping them to solve the problem to a larger but there were 2 major cons which we identified. So firstly since we were asking customer for a slot for each order it adds another step in the order placement stage which ultimately results in friction for the customer. The other thing is schedule delivery was a required lot of investment from operation side. So you will operations team side. So you need to have more frequent run for your last mile order delivery. So those things had to be done before the solution can be implemented. So these were the 2 cons which we realized and we thought that this will take time to build. The second solution which we evaluated was and this was a very basic simple thing to implement basically if we will ask for each address we will ask the customer to provide the delivery preference. So the customer can choose between all days versus weekends. So let's say if it is an office address for which the customer has added then they can provide then they can select only these days for delivery and they can also provide their business hours. So there were 2 benefits of this. Firstly it is at an address level as compared to solution 1 which was at an order level so it adds less of friction for customers. The other thing is from ops point of view as compared to solution 1 this is easy to implement it is not that big an investment. So these are the 2 major benefits for this solution. Then the third solution which we evaluated is to auto classify the customer address automatically classify the customer address into residential and commercial. So this is a more sophisticated upgraded version of solution 2 only. So you don't have to ask for customer input and you can build a model to identify these addresses yourself in the backend. So from friction point of view this is the least friction but it adds like it will take time to build that kind of a model so we thought that we can start with solution 2 and then it can be our phase 2. The fourth solution which we evaluated was to have just like we just evaluated if we can deliver it even if the customer not present if you can deliver the product but when we discuss this with our internal teams finance losses team we realize that this will increase chances of theft and it will lead to more might lead to more escalation from the customer so they can claim that the delivery that it said that the item is delivered but they actually didn't receive the item so this was a risky solution so we discarded solution 4 and we prioritized solution 2 which was an easier thing to implement from operations point of view and didn't add too much friction for the customer as well then in stage 3 so once we have identified the optimal solution stage 3 we detailed out the tech requirement so we also created a long term vision so in this case the long term vision was to build a hassle free secure delivery service for customers based on their delivery preferences so this was the long term vision we detailed it out using the PR FAQ we also divided this into different phases so phase 1 was the solution 2 which we discussed basically in phase 1 we built a capability to allow customer to add delivery preferences for each of their new addresses and these delivery preferences were between residential and commercial so it was a commercial address then the customer can choose the weekdays only delivery they can select Saturday if they want to they can select Sunday as well but by default if the customer has specified the address as commercial then the delivery will be attempted on weekdays only then phase 2 was the solution 3 which we discussed in the previous slide which is the most sophisticated version the we built an ML model machine learning model to auto classify customer addresses into residential and commercial and we calculated the delivery date basis this classification phase 3 was to introduce a schedule delivery so since this was a heavy investment from operations point of view and even from tech point of view we thought that we will do it not for all categories we select some categories high value items, appliances furniture etc and we will also collect feedback from customers and then we will implement this these were the 3 phases this was the long conversion then we worked on the detail business requirement document so I have mentioned the sample user stories one user story from customer point of view is that they would want an capability to update their delivery preference to weekdays if they have selected office address and they would want a capability to update the timings as well so just as an example as mentioned 10 am to 5 pm as the preferred timing then from in this example from ops point of view ops is also a customer so from operations team point of view would want to store that information you would want to get that customer preferences and for that you can plan or you can schedule the delivery as per the customer preference then the third step was to prepare the mock-ups which created a very mock-up I have just mentioned that mock-up here as well so there were two options for the customer after they have updated the address either they can choose deliver all days 79 pm or they can choose deliver only on weekdays we also provided customer to select Saturday if they want to select Saturday and Sunday if they want to receive their delivery on weekends as well even if the address for the commercial address or office address then the last thing in this stage was to identify the success matrix partner team sign off some of the input matrix was to calculate the defect rate that the customer has provided delivery preference as weekdays only then how many delivery attempts we are making on weekends on Saturday Sunday so this defect rate should be minimum output it should be the customer for the escalation so number reduction in customer escalations and the ultimate cost savings from that stage 4 and 5 is basically the tech development testing and launch in this case so there were three broad level changes which we made so we made also tech team to do the UI changes to add delivery preferences section we asked them to build the backend logic to flow customer delivery preference into the systems databases used by operations team we also asked them to build the capability to restrict restrict the item or order going out on road on or Saturday Sunday if the customer has provided weekdays as their delivery preference then in the testing phase we tested the end to end workflow as a product manager I inputted all the customers all the possible inputs in the delivery preferences and tested the end to end workflow starting from order placement till the time item got delivered then we had the standard go no go meeting with operations team tech team other businesses team category team and asked them to provide an official sign out so this was our our journey in launching in building this product hope this helped nothing else from my side do reach out please do reach out if you have any further questions any clarification required thank you