 Hello, everyone, and welcome to this week's Product School webinar. Thanks for joining us today. Just in case you didn't know, Product School teaches product management, product leadership, coding, data analytics, UX design, and digital marketing courses online and at our 16 campuses worldwide. On top of that, every week we offer some amazing local product management events and host online webinars, live streams, and ask me anything sessions. Head over to productschool.com after this webinar to check them out. Today, I'd like to introduce you to Sushma Rao. Sushma is a results-orientated senior technical product manager with experience in e-commerce, supply chain, and logistics domains. She is a subject matter expert for inventory order, fulfillment, and functionalities at Nike. Sushma has also worked across all platforms and features ranging from infrastructure to UI to integrations. She loves to tackle complex product and business problems in order to deliver value for customers and the company. Feel free to leave any questions for Sushma in the comments and we'll be sure to address them at the end. Without further ado, let's welcome Sushma. Thanks for joining us today. Hi, everyone. Thank you so much for the opportunity Product School. I really appreciate it. I'll start my presentation now. So give me a second while I share my screen. OK, so let's dive right into it. That was a great introduction, Dan. Thank you so much for that. I'll probably introduce myself a little more. I have a very diverse background for anyone who is actually looking for advice as to how to move into a different domain or a different career path or how did I get to product management? This is a tiny shield about it. I actually did my undergrad in architecture, which is building buildings, right? So I was really interested in design and UX from the get go. And but from the perspective of building spaces. And so architecture was pretty awesome in India, but then I wanted to challenge myself a little more. And so what can be more challenging than doing an engineering in aerospace, right? So I came to the US to get my aerospace engineering degree. And once I did that, I finished in 2010. And there was a time when we were in a big bad recession and it was very difficult to find a job. And I luckily fell into the business and analysis space. So I just took up so these are three big career diversions that I had. And so once I came into the business and analysis space, I was in supply chain, really enjoying what I was doing. And I grew from there into a product owner position. And then finally now at Nike as a product manager. So for the past year or so, I have been in the inventory order management and order fulfillment space. And now I've actually gone into the sourcing space. So I'll definitely talk a little more detail about it. A few projects that I've worked on, a few things that we can pick up from there. So how do we actually build a data driven roadmap? So that's actually so from my aerospace engineering background and all of those backgrounds, I mix all of my experiences together and come up with this ideology about how I want to work together with a roadmap. So a few things that I'm going to cover today are first and foremost, we want to understand what an agile product management mindset is about. And then we want to look at early on, even from the get go, what metrics should we look at? We want to look at the process of creating a new product in these five steps. That is innovation and ideation, evaluating desirability, evaluating viability, evaluating feasibility and then creating a feedback loop after you've created the product continuously to learn through experiential learning. So diving right into it again. So what do you mean by agile product management mindset is essentially about the iterative process. We all know what agile means, but when it comes to huge companies or smaller companies, it needs to be scalable. It needs to address the problem at whatever scale, at whatever domain, for it needs to be agile in with respect to even addressing problems that is specific to those companies. So we want to look at this in a hierarchical level. We need to look at it from visioning strategy and tactics. So from tactics, what I mean is say, for instance, a scrum team is more at the tactical level, where they're actually getting work done. Strategy is more at the product manager level, where you are interacting with multiple teams and multiple leadership to understand how you can bring that strategy to the tactical level and direct it. And then the visioning is obviously the leadership that is going to give you the direction. So all of these need to work in tandem and they need to be iteratively working together. So there are times when you may not be aligned with the vision and you need to pull them back and align with the vision. There might be times when the tactical work that your team is doing is not aligned with your strategy. And so you need to pull the team back into strategy. And not to say that there have not been times when we have had to pull it back. The job, essentially, from the agile perspective, I feel from my experience, is always about pulling these strings together, pulling everyone's attention to what it needs to be and align with, align everyone together. So the circle in circle actually is a very graphical picture about how we want to be able to align these. And so what does that mean? So creating of a product backlog for a team may involve tactical work of getting into a room in a backlog rooming meeting and creating those stories. But you need to take time away from that and look at those stories and see if they align with your strategy and product roadmap for that matter. And then furthermore, with the vision boards that's created. So there are things that in your domain that you can control, but there might be things that are not, but it's your job to keep iterating and keep trying to push and get everyone on the same page. So what does it mean with respect to artifacts, for instance, or a mindset? What is it that you will do to get all of these together? So we talk about being customer driven. We talk about addressing the full value of the stream, experimental learning and embracing change. All of these are agile mindsets. How do we create those into products though? So there needs to be a straight line or a connection between how you can connect these customer driven full value stream, experimental learning into your business vision. So what I mean by that is you create a business vision. You'll be very tactical about your own work so that you can connect those dots together. So you create a business vision, you have a potential product. So you can create a one-pager. I think it's a very useful thing right at the get-go. If you can create a one-pager, create a process flow, and then keep looking at it again and again to see where the gaps are and how you can evolve the roadmap to align with it. I'm also talking about defining the right metrics early. Now this is a very nuanced question. It depends on what the metrics might be. The metrics for my company, for my specific project, and for my specific product is going to be very different from what you might need. So a few success metrics that I have seen have helped me is when I was creating a UI-based products, for instance, have been product usage. Percentage of users would take a specific action or a feature usage or a tension or churn rate. But these are all very specific. So I've also been on a project where it's SaaS-based and there's nothing UI-based. I can't really look at the frequency of sharing. I can't really look at a specific action by the user or a feature usage. So for that, I would look at what, even if it's a B2B internal customer, then I would look at customer acquisition cost, what is the lifetime value. All of those need to align with each other. So in a SaaS-based project, I definitely want to give you guys an example. So in the inventory project, what we did was we created a platform. We connected with Jet.com from Nike to Jet. So we are selling our tools on Jet.com's platform. So what that means is we are exposing our APIs to Jet.com and they're coming and pulling our data for all the inventory that's available in Nike. And we are fulfilling those shoes or apparels and footwear to Jet.com's customers. So how would this be? So would I be leveraging Jet.com's metrics to look at my metrics? This is a complex problem. So from that perspective, our metrics for success were about how quickly we could actually provision the APIs to our customers. Jet.com being that in this case. So our metrics were very different from what it would be for a product. Say for instance, which has a UI or say for instance, if it was for Nike.com. So in the business-oriented metrics, what I mean is these are all things that may be very relevant in cases that are different at different times. A few examples for the key metrics that we looked at was, say for instance, fraud prevention. How can we fight bad guys in real time? How can we discover and personalize? Our product, when we are building it. Image recognition, all of this is MI based and identifying best images with the neural networks and provisioning them to Jet.com. Say for instance, logistics, getting ahead of the order, rush by sending extra inventory to the right warehouse. Customer support and supply intelligence. So these are just some examples. So one of the key things when you start building a product, I feel I want to go through the life cycle of creating and the culture of ideation and culture of innovation. How do you build this? Some examples, there are lots of tools out there. And I'm just telling you about this one example of a tool, which is of bearing six thinking hats. What do those six thinking hats mean? So you have, so this is brainstorming session. You don't want to get into a place where you're a product manager for a team. And it's either my way or highway. You want to be more collaborative. You want to build a team which can think for itself, can come up and explain their problems if they come across them. And that's one of the key things is to create a culture of ideation. And how can you do that? You can create that by say, for instance, getting together and asking people to wear different hats from a positive perspective, maybe a negative perspective from a creative perspective and time it. There are immense amount of literature out there telling us that we need to allow people to have their opinions and not feel like there's going to be a distribution about it. So follow the process so that you can create a culture of ideation and innovation. Then how do you evaluate desirability? So this is something that I'm also currently getting my MBA. And one of the things that I've learned at my MBA at University of Washington are these tools that you have, nifty tools that you have that you can use to understand whether it's from the marketing perspective or whether it is from product management perspective if you want to incorporate them in your roadmap. So this is to tie metrics to strategic goals. And if the data shows that you need to adjust your strategy to it, what you do is create, say for instance, the stages of buyers experience. So you look at price, simplicity, convenience, risk, fund image and environmental friendliness against all the utilities like purchase through the life cycle. When the customer purchases, do they feel any, do they have any issues with price? Do they think that it's simple enough, it's convenient enough? And so you go through the process to come up with your buyer experience. You can also hook up to Google Analytics. So obviously everyone knows that, but there are lots of other places where you can get free data from like data.gov or annual reports of publicly traded companies. So it's really useful sometimes for us to look at our competitors, annual reports to find out how they're collecting these data, how are they looking at their customer basis so that we can cater to our customers similarly. Think with Google is another really good resource. Pew Resource Center is another really, really good resource for collecting these data points. So it's important to get all of this data first and foremost, accumulate them and then look at them from all possible. And if you notice there are lots, there's big data, what that means is you have so much data, you don't know what to do with it, but many times you do have to look. So for that reason, time strategic goals with, again, time back those strategic goals with your road maps will be super helpful. You will be able to look at everything, you have all the data, but only what is relevant to your industry or what is relevant to your roadmap is what you need to pick up. Some other tools that I have really enjoyed using is a spider map and a Kano analysis. Say for instance, when you're looking at visibility of your product, you look at all the aspects that you think of and then compare them against all the aspects that you provide these customers. You look at them and you see if you're doing good, if there's a particular area that you might not be doing good in and you want to improve in that area. And if that's even important for the customer, so you ask your customers these questions. A few places where I've really gotten a lot of data is also, say for instance, InSpecTillet is a really nifty tool where you can watch the users go through the entire process of clicking. Are they, in fact, using all of the features that you think are useful to the customer so you get out of your bubble and start looking at them? So Google Analytics InSpecTillet, that's another really good one. Kano Analysis, again, is also really, really useful. There is a lot of literature about this as well. I suggest you read up about these classification of customer base so that it helps you whether they're one-dimensional requirements, what area specifically you need to cater in. Now, coming to viability, why do we want to look at viability? Now, we've done all this analysis. We're so sure about what we want to build. We are diving right into it and building. Again, building that agile mindset brings this thing about coming back and looking at your own solutions again and again. So whether a product meets the customer needs or not. Am I going in the right direction? And so we need to be iteratively analyzing our own product rationale as well. We're going to look at our idea whether it's viable. We're going to address our hypothesis again and again. So it's an iterative process, whether it be for the development team that's building it or for our analysis or even for our product management. Some of the economic value estimation and customer life value also helps a lot with understanding how you may price your product and how you view the benefits of your long run. So if you're looking at revenue as something that you're going to look at from the perspective of metrics, this is also really useful to see if what extensions work best and how much of the cost works there. Whether a particular landing page is actually bringing in the revenue that you want. So all of these things are super useful. So let's look at feasibility. Now we've built it, we've evaluated it, we've been successively evaluating it. Now we want to see if it's aligning with the business and if we are in fact in the right direction again. So if you notice we are reevaluating and evaluating again. So that's the cyclical nature of product management I guess. And so you need to evaluate that. So how would you do it? At this stage you're at a point where you are going to launch and you will encounter a lot of hindrances even during launch. So you will have a conversation and you're constantly during all of this process creating tools and frameworks to identify the optimal path. And you will meet a lot of dependencies that will come across. So it's not just the development team or the solution delivery team that's building this product but also continuous delivery like differences CI CD team in your company or the lease management in your company if it's not on cloud. And you have to work with those teams to align with their release stream or the data management. Did you in fact, are you in fact building the data in the manner that it needs to be built in? For example, if I'm using a data point and I'm recreating it in my database that's not ideal. We need to be pulling it from a say for instance a centralized database domain. Are you incorporating enterprise architectures ideas of what your system needs to be? Are you following the IT governance? Are you following the UX UI standards in the library? So all of these things you would make it does sound very complex but this is a daily life of a product manager you need to keep aligning yourself with all of the systems all of the roadmaps. So each of these teams might have their own unique roadmaps and you need to align with them and be on their roadmaps as well. So then after you've started building the solution you have to prioritize and how do you prioritize? A few things that have been really useful for me in the skilled agile framework has been the WSJF which is Bated Short Test Job first. So if you're using JIRA say for instance or a tool that helps you prioritize one of the things that you could do maybe in addition to using these tools is also have something called as WSJF or the RICE method which is a region-packed effort confidence and what that allows you to do however, whichever one you choose and whichever one aligns with your roadmap it helps you organize your priority it gives you the data that you need to prove to someone that yes this is in fact higher priority and it's not just based on a whim that you think it's a higher priority than the other feature. It is an art it's definitely something that comes from the gut and the numbers that you place are also based on your analysis and so this helps you prioritize quite a bit. What it involves is say for instance a list of all the features and then the duration and the cost of delay and the weight of the weight of the cost of delay over the duration. So this gives you the rating at which you want to place the features at. If this doesn't help you then the RICE method also is another really nice one which prioritizes based on the reach of your product the impact of your product the effort it might take to build that product and the confidence you have in building that product. So whichever tool you choose to use there's definitely ways to manage your priority and I know it might be a little old school but what I do is I pull out all of my backlogs say for instance for a couple of sprints and then I do this so that I have an instantaneous record of who I am pulling things up and down. What that this allows me to do is to not have the features constantly updated but or limits me from moving the features up and down the backlog say for instance if you think about it but it does hopefully all this WSGF but can be incorporated in the tool someday and we can move things around using these data points but for now I'm using Excel. A feedback loop so just like we did the process of creating the product we have to create a feedback loop and that is the experiential learning that you're talking about. A few areas where we have incorporated these experiential learning is for instance Nike Live we have in-store omni-channels who would have expected that now brick and mortar shops are more in demand in some areas than the other. How do we find out about those? In fact we have to create these experiential learnings we find out from other competitors that this omni-channel experience is in demand and we created these in some locations in LA and in New York then we came to know about BOPIS which is buy online and pick up in store. We found out from all of our competitors from maybe from some other industry that this is the in-thing now and so we started building these feedback loops to find out if we can build something like this. Reserve lockers is another example where shoppers select the shoes online they can go to a locker that they can unlock using their phone and then it's called a speed shop and they don't need to interact with anyone in the store they can just go pick up their shoe from there. So these are some experiential learnings so you're constantly having the feedback loop in place so that you keep learning and keep improving on them and keep using your data points to loop back that information so that you can keep improving on them. I do want to finally say that you know this is from the Nike's perspective I really believe that leading, coaching, driving and inspiring is critical to critical to building a roadmap and it's really critical to even working with a team communicating with a team and this should be part of your leadership mentality it should be part of the manner in which you conduct yourself with the team and the way you want to build the team. You should look at the data as a statistician you should look at it it will help you you know gather all that information and pass it on and also communicate with everyone based on your roadmap the roadmap is not just for you it's meant to be open to everyone and to communicate everyone about it and so always remember that you need to encourage the result that you want you know you might be so bogged down on the results and the metrics that you'll you might not incorporate you might be you know unknowingly encouraging the result that you don't want and yeah correlation is not causality and culture of experimentational failure is always needs to be encouraged so I have to say that you have to focus on solving not selling and you have to think about purpose not just profit so I think if anyone has any questions I that is all I have awesome thank you so much Zeus man that was a great presentation packed with awesome insights so um before we get to any questions here in the comments one thing we like to ask all of our featured speakers is do you have any advice for aspiring product managers one piece of wisdom or a quote or something that you'd like to tell someone trying to break into product management yeah absolutely um I actually think that a really good advice that I got from someone at some point is something I definitely want to share with you all is that product management uh is a very senior role in some companies and many times when you want to break into product management it's not a straight path uh many times it's the business analyst's role or the product owner's role or a you know a junior product owner's role that transitions itself into a product manager's role many times we think that a product manager's role is um just out of college it may be it very well may be in some companies but not many companies view it that way particularly in the scale agile um domain and nowadays when the scale agile a methodology is being adopted in these big companies it's viewed as something at the solution delivery level or the program manager's level so I would I would suggest that you all look into our product ownership which which is equally interesting and equally um you know engaging and it's it's just in the tactical domain it is in a shorter range and the product manager's role is in a much longer range of time besides involving and as satisfying to work as so I definitely suggest that you look into a product ownership role to break into product management awesome thank you so we have one question here correlation is not causality question mark can you please expand yeah absolutely uh so many times we assume that as um you know when we are taking out data like say for instance I'm watching in spectacular somebody going and clicking something and leaving off and I see that the churn rate is going now uh going up and I don't see people actually converting enough and I assume that it's because say for instance the UI is not correct or I don't really find it easy to manage so I don't necessarily think that this correlation that I'm making um is accurate we need to be open enough to get everyone's input and to understand that that's not that may not necessarily be the cause of the uh cause of concern cause of the churn that I'm seeing and um I need to address it much better I need to go dig deep and find out what the real reason might be that's what I mean by correlation is not necessarily causality awesome thank you for answering that so next we have from Christina when building out the roadmap how far into the future will you look as far as you can so um absolutely meaning uh if if your company is um in tune enough in well oiled enough and you can look into the future it's always the best idea to look at as far look at it as far ahead as possible from your particular products uh roadmap it may not be possible to do it but if you have multiple products under you then there is definitely a need to look at it from a year's perspective or a two-year perspective so we have uh road maps that are built out at least a year ahead um and that helps us plan much better awesome thank you so next we have how do you control resource management like development hours on your teams is there any tool softwares or add-ons for JIRA or any similar software your team uses yeah so resource management is in this gig economy it's such a big deal isn't it so there's a lot of people who come in and keep uh go and then it's it's difficult to manage resource to manage onboarding to manage the knowledge that comes from the resource and individuals that are coming and going so definitely it's it's uh it's something on our mind we manage hours by tasks we we do not necessarily manage resource from the perspective of the project itself but yes they do get aligned with budget and the budget with the resource and there is that alignment but many times product managers are not responsible for taking care of the budget which I'm really happy about many times because I'm my stream of works if for instance in inventory or sourcing for instance has a signed budget for a year and we use that up and it's called pay to play we don't like to do pay to play from project to project and keep having to abide by the resource cost but yeah sometimes that's the reality of the world cool thank you so our next question is how did you convince the first interviewer about your skill to perform well in your product management role oh wow okay so in a sense it was a very slow transition I I don't think uh like I said I used to be a business analyst before and then I moved on to a product owner's role and then finally to a product manager's role so it was a very very slow transition uh from I mean I it wasn't fast enough for me but if for instance I had to explore and explain my skills to someone today then I think the most critical thing that uh we forget about is communication and the focus on metrics and that's the reason why I uh I'll give you examples of how we use these metrics to influence so one of the things a product manager has to do is influence people without authority right and that can come you can be empowered to do that with with data and with metrics and and how you do that is give them give people a reason to listen to you and not necessarily just do it because you're their boss so uh data gives it empowers you and that that's one of the things you would convince your employer by is that I'm capable of influencing people without having to rely on being bossy awesome answer thank you and for our last question here um do you have a preference on what tool you use to create your roadmap I do I actually really like JIRA it's not just a tool and and I know many people don't realize this but JIRA actually has a scaled agile framework in it and it can scale up to a roadmap and create it and help uh I do like JIRA very much there are many other tools out there I'm too diagnostic for the most part but uh until it uh trap meaning is not easy to use so JIRA is pretty easy to use and build up a roadmap with awesome thank you so much I think that uh about wraps it up today thank you for joining us here at uh product school for our webinar um really quick I'm going to give you guys some more information on our upcoming courses and events so you have the resources to become a product manager our product management product leadership coding data analytics UX design and digital marketing courses are taught by industry experts working at companies like google and facebook in addition to that we offer weekly online and on site events at our 16 campuses throughout the US UK and Canada so if you're located near campus make sure you stop by one of our weekly events every wednesday and thursday and you can also find us on social media at product school and be sure to keep up with the latest product manage product management content at the product blog at productschool.com thank you all for joining today enjoy the rest of your day and I hope to see you next week thank you so much it's a pleasure