 Hello there. Good morning. Good afternoon. Good evening to all of you from whichever part of the world you're listening from. Thank you for being here Today, I'm going to spend some time talking Sharing about building long-term ML driven roadmaps and before we start a bit about myself My name is Kapil based in Seattle I'm driven by passion for products, mostly software products Why they work? Why they fail? I'm extremely people-motivated person And I love traveling. My ideal travel style is many shorter day-long or a few days-long trips with my dog a four-year-old dooder Living in Pacific Northwest offers many destinations that fit into my travel style My favorite city to visit is Montreal, Canada And my favorite nature oriented place to visit is Crater Lake Oregon Now Before we move to the topic of the hour disclaimer The content in the deck is solely my personal point of view and general knowledge material It does not represent any company or my current employment I intended to keep this content for a wider spectrum of listeners New in product industry happy to do a follow-up offline via email or LinkedIn And peers and experts out there happy to learn if something here needs a correction If we gather key skill sets of a product manager's role And create a version of Word cloud it will look something like this By no means this is an exhaustive list But these are the most common yet very crucial skill set of a product manager I place the word roadmap in the center on purpose because that's the topic I'm going to cover Roadmaps I think it's okay to assume majority of you have heard or played with chat gpt If you have not I highly recommend you to do so Chat gpt is next level of ml and ai engine with a goal to make ai systems more natural and safe to interact with You'll be amazed how trailblazing it is That said, I asked chat gpt to help me with my presentation And ask it to respond to my prompt What is software product roadmap? And what I got in a reply is what you see on the screen It's pretty neat In my opinion Let me read it for you A software product roadmap is a visual representation of product strategy and direction Over a specified period of time It lays out the plan features and milestones for a product And is typically used to communicate the product's vision and goals to the stakeholders such as customers investors and team members Software product roadmaps are a useful tool for product managers Development teams and other stakeholders to align on the product's priorities And ensure that the development efforts are focused on achieving the product's goal Wow Those chargons And that's a pretty heavy definition, isn't it? Let's simplify it a bit further for folks new in the product management world In simple terms Imagine you have goals and targets in future Roadmap is a plan to get there And your journey is often divided into smaller but important milestones as you go along that journey Before I move on, I would like to repeat the basics Why roadmaps are important It's about building muscle memory, right? All right, let's begin First strategy and direction Roadmap helps you to align with your stakeholders on the north star state What are the goals and how to get there? Roadmaps are time bound They are built based on the information you have available today And also, they are a plan feature and milestone because without a plan strategy is just a wish Roadmap also helps you align on vision and goals Especially with your stakeholders They help you prioritize things better Because every quarter is like a snake and ladder game Roadmap, they help you stay focused with the limited resources you have And the most important thing about roadmap in my opinion is roadmap is not an execution plan Let's build a roadmap for multi-quarter initiative I'm going to do a prompt pseudo roadmap building exercise As my mechanism to touch base product management concepts as we move along Let's get to it Let's build a hypothetical roadmap for an ML based personalized wellness platform And the prompt is you are a product manager in a wellness solutions company Task with creating a machine learning based personalized wellness program And evaluating its effectiveness on employee engagement and productivity over multiple quarters And the north star goal for the company is to launch this ML based product Outside the company. So we can assume that They have a non ML based personalized wellness platform Currently and they are trying to build a new one Replace the existing one with ML based personalized wellness platform There are many things to consider while building a roadmap Whether it's machine learning based or non machine learning based Whether it's wellness or some other industry first one finding the right balance Between top to bottom leadership priorities and bottoms up tech solutions For instance in this company Building an ML based wellness program may not be the top priority for the leadership Also on the flip side even before proving the MVP and its value The tech team might be leaning towards building a very fancy solution In those scenarios you have to find what is the right balance For your priorities and what are the sequence of the milestones second Building roadmap is highly cross collaborative effort. There are dependencies internal as well as external Depending upon your organization You may be working with all the teams listed here or some of these teams listed here to move your roadmap ahead Influencing other roadmap is a crucial thing in order to move yours forward There's a concept called shuttle diplomacy The idea is that you shuttle between individual Stakeholders and product managers in order to achieve the clarity and rally them towards the common goal We saw earlier roadmap is about prioritizing features and milestones Why is that? We do not have luxury of infinite time and we never have enough resources So building roadmap and prioritizing the milestones and the features Is a constant balancing act between the scope time and resources Next before start building this hypothetical roadmap. I want to cover a quick product lifecycle primer This is typical product life cycle phase gate if you will from An inception to grow to maturity And if you see the roadmap or creating roadmap is under plan And planning in my opinion is the most important phase of your product lifecycle Roadmap leads to prioritization There are various prioritization methods also known as frameworks Why to use frameworks? frameworks create a discipline They help with repeated tasks And they become a validated checklist Especially roadmaps being iterative from one phase to the next There are almost three dozen frameworks And the four listed here are the one I personally have used You may be already have an established process method or framework within your organization For this hypothetical ML based wellness platform building I'm leaning towards using Kano model for my prioritization Kano model helps you to focus or distinguish between What are the must haves? What are the performance features? And what are the delighters? One of the ways I can structure or formulate this hypothetical roadmap Is by following a principle that's very popular in amazon called working backwards The idea is this you start with a customer or user Then you draft a press release about the solution that you have built to solve that customer's problem Then it's followed by evaluation of product market fit Look at the data and to Judge whether to build that product or not Which is then followed by discover solutions and get stakeholder alignment Then it's followed by build a very high level roadmap And identify themes those go into that roadmap Followed by creating backlog well In our hypothetical Roadmap building scenario We already know who is the end user You guessed it The employees of this company Now moving to data In your roadmap building exercise, you may have Data readily available or you may not have it readily available In that scenario, you will need to do market research Some of the companies hire external vendors to gather this data for them And some of them use their internal teams to do market research For the prompt roadmap, I'm using data that I found on internet Fitting this data towards what is the goal? What is the outcome will help me to come up with high level roadmap themes Now I can say that the end goal or the outcome for this ml based wellness platform is Reducing employee burnout is crucial for my organization Because of the impact it has on areas like innovation productivity and retention Now combining the user data outcome and the mission that we talked earlier This is a rough schematic of a long-term roadmap I would devise for building the ml based wellness platform We will cover the details of some of these boxes here in the subsequent slide But if you step outside a bit You would see that it's a very typical way software products are built You start with data Then you plan your mvp Then you test test your mvp You find the signals the metrics from your mvp And then you decide your scaling factor and what follows by the mvp and continue iterating on it Before proceeding further, I want to take a pause here and talk about a concept called storytelling I think storytelling makes our job as a product manager very unique It's an ability to connect the dots in such a way You simplify very technical concept to somebody who is not technical And convey the message of the importance of business priority To somebody whose focus is about building technical solutions In storytelling as you go about building your roadmaps use the material For your story building in your product briefs in your product specs FAQ's newsletter The idea is that to stay consistent in your messaging So I have two examples here One is the market research information that I use In order to feed towards coming up with the goals and the outcome of this ml platform And while doing the research, I came across some Visually appealing video that I can use in my storytelling Up next, I'm going to build a list of themes that I'm going to focus on for my roadmap To cover a lot of ground, I have placed the things to do into different categories Based on the skillset needed and the proximity to the stakeholders Remember, we talked about product roadmaps being highly cross collaborative and iterative in nature I'm not going to cover all of these boxes here But the idea is that having these categories will help me understand The tanglement of their dependencies so that I can sequence The Milestones in a way that the one milestone feeds into the other one That way I can build a roadmap that's very sequential if not parallel Some of the topics will need deeper conversation as I go about building the roadmap from early phases towards the maturity And that's what make roadmaps like these multi quarters sometimes multi-year because of limited resources Budgeting constraint conflicting priorities. There is tanglement of these dependencies Before I go about building this MVP roadmap, I would focus on personas Persona is a group of representation of group of users that you are trying to solve the problem for Focus on the main ones be data-driven use expert opinions before you latch on to the first top two personas that you're going to solve the problem for Here I have a four type of personas that I I would go I would focus on initially but not all of them I would choose one or two out of these four personas to build my MVP MVP minimum viable product For MVP or phase one I would go about thinking asking the simple question to the survey employees If there's a portal or an app to track your wellness using machine learning Which will require a lot of data gathering Will you be willing to use it and as a product manager my area or focus should be Solving for that problem and not necessarily try to Find the best solution by myself because that's where the tech team comes into the play And for MVP, I would go about having a goal of achieving maximum percentage of adoption During the phase one or at the end of the phase one delivery And what you see on the screen is the high level buckets of the work streams or the themes That I will would be working with during the phase one The two main stakeholder during the phase one are legal and human resources Where I need to understand what are the laws? What are the guidelines and policies so that I won't build a feature On this new platform, which is not okay or legal From human resources or the legal standpoint Then the second bucket of the stakeholder or the team that I'll be working very closely during phase one Is the design assuming the earlier design is very Old generation or it's not ML friendly or there is no way portal for the existing solutions Then I would be working with designs to create mocks And then use the power users or early adopters to give me feedback about the mocks For working with designs, I usually follow a double diamond design process Which is illustrated here So if you search on Google, what is double diamond design process? You will find like a variations of a diagram like this But the idea is simple that even before you start developing and delivering a milestone You spend a lot of time discovering and defining concepts and spend a lot of time with designers And designing a solution is never a one team effort Take as you can see on the right hand side It takes product design and tech to come up with a solution a feature that's solving a user problem And the design is very iterative process So choose your sequence of design milestones wisely and try to avoid creating design debts in future Document as you go along and share this document with your stakeholder and users So that you can gather early feedback And it goes without saying include tech because that's where the feasibility Or the Solution comes from Um, so a quick example like how I would go about building mocks So even before building applications or putting system together I would work with the designs to come up with some mocks or proto mocks or prototypes Using tools like Figma or Miro and focus on three main things Like what is the function flow and familiarity of each Navigation or each page or each section on application or web portal, whatever that may be Write down the tenets of these design principles and stick to them and try to be user centric And think about designing the UI or the app UI like Google's french fry movement principle Where we are trying to build user experience even before user tells us what they would like to see on the screen Now moving from MVP to next phase phase two And before I start speaking about the phase two, I will now bring attention to three main points here That first one is be okay to stop. So at the end of MVP or during your development of MVP As you learn more about your problem solving the solution the ROI from it and The investment that you are trying to put into it Once you understand the enough information from that data be okay to stop don't fall for sunken cost bias Second is evaluate your biases with data. So that goes, you know, like that flows into the point above like be okay to stop You build MVP doesn't mean you have to continue going down that path because you know, like you started building MVP based on information You knew in the past and now these circumstances macroeconomics or the priorities Have changed and I think that's the call out here that you need to constantly evaluate Where you are and then how to proceed And also talk to early adopters stakeholders and peer product managers for pulse checks So that you can have an outsider opinion whether you are on the right track or not So assuming that everything of these bullet points Were okay for me to proceed to phase two. So my phase two would look something like this Now in phase two instead of just the adoption I would also go for stability and scale because I would be adding more and more users to this platform But as I said, you constantly keep learning from MVP. So then I would ask myself What is MVP trying to tell me is the wellness product going in the right direction for my leadership What did early adopters like dislike about phase one or the MVP and based on that I will tune my phase two so more or less Similar bucket of work stream as we saw on phase one But if you see the description some of the description under the main work stream have changed because now we are in advanced phase So our requirements are changed. So in for example for data Now we have to Worry about the compliance and accuracy of the data The storage of access of that data by the ml teams Then another change for let's say design is Retreating on design because now we have started getting the feedback about the design So, uh, that's the idea that as you go from one phase to the other phase You more likely will continue to work on the similar work streams, but the Actual tactics within those work stream may differ as you progress from phase one to phase two or the advanced phases So then the next phases is that instead of uh, keep calling phase three phase four And so and so forth. I'm just saying towards the maturity of the product or the platform And again, it's the same thing. Evaluate your biases with your data. Be okay to stop Avoid sunken caused bias talk to adopters talk to stakeholders peer product managers to do the pulse check So fast forward in the mature phase of the platform I instead of going for Adoptions because now we have achieved the adoption that we have targeted for early adapters has been achieved so Now my focus should be on retention or the engagement of these Adopters of the wellness platform that feeds into The customer satisfaction score. So like have a baseline to XYZ minimum satisfaction score about the wellness program Then that ties with the HR KPI retention. So off the uh early adopters or off the Adopters of the wellness how many percentage of employees are willing to stay with the Employer or with my organization because they are seeing value Out of wellness the productivity has gone up and they feel more proactive in their work And while all that's happening Because this solution is a ML base and it would need a lot of infra to be a bit around it I would also work with tech team to establish the cause baseline the run rate and try to optimize the Work with work with tech to have the optimized tech stack with XYZ run rate per month again more or less the same bucket of Workstream themes as earlier phases, but the description and the practical Points within each of those bucket will change as we go from One phase to the other phase. So for example the scope when we started our MVP was for us only because we wanted to Test it for only, you know, like a smaller subset of users and then in phase two I mentioned I think scope extended to international And then you will have to come up with some expansion strategy And then in maturity we are I'm targeting for a global scale of release timeline ganchard So these are the visual representation of how your roadmap is progressing or going to progress. So in my case All the phases that I talked earlier I put them into various quarters using some estimates And this is just for illustration purpose by no means. I'm an expert in wellness industry But you know, like as I mentioned in my previous slide that this roadmap building exercise is just to connect the dots to talk about product management concept. So here um I have these phases laid across multiple quarters starting with mvp q1 Then two quarters for like driving the adoption performance stability Then the two quarters after that increasing the customer satisfaction score and then in the phase three I would heavily work with hr to uh implement the policies And the guidelines so that we can try the wellness program with retention and the benefits to the employees And then as we move towards the maturity Do the cost optimization incentivize employees using the wellness program And in uh next year, uh h2 do external launch As I mentioned in my prom slide that we are targeting uh in in this hypothetical company Build a uh build an ml based wellness program test it out make it better make it amazing And then launch it externally given the company. Uh is a wellness company So the long uh longer horizon tells you like what to expect for 12 to 24 months. Uh, that's uh, uh solid path To represent but you should also have something called as a smaller scale Representation which I'm calling now relaxed and usually it would be q1 q2 for some teams some of the teams do only The current quarter that they're planning or they are under so depending on you know, like how your team is structured You would be uh creating or you should be creating a smaller view like this. So in my uh slide here This is the phase one more mvp details that all the boxes those three columns boxes In terms of action items and just kind of like using some guess work About the uh sequence and the some of the work can be done parallel. So for example As the designs are coming, uh, uh to the, um Fusion we don't necessarily have to wait until every single page or in every single navigation is be designed As soon as we have a concept design, we can start Involving early adopters stakeholders hr legal into getting feedback about those Mugs and lock in the mvp. And so that's the idea behind how you go about building some of these blogs Either in sequence or in parallel leveraging data and ml as I said in my uh prom that I am tasked to build a machine learning based wellness program So it definitely would help me to understand what is data science machine learning lifecycle That way I can Find out the tanglement of dependency. I can call out the blockers. I can find out what are the What work streams have to go in sequence where I can create parallel work streams and so on so forth So this, um Data science and ml is very powerful because these algorithms pipelines They process gigantic amount of data and information to come up with very curated personalized output And understanding this lifecycle is crucial while building ml based roadmaps This diagram is very general guideline But you know, like you get the idea like how it flows from understanding business to like building data pipeline and Building the models doing the deployments And it's cyclical because the data science models just like human brain like they keep learning on data and you have to Cycle through these are training and testing of the models and while We are building the ml based solutions Depending on your industry depending on the problem you're trying to solve There might be additional things you would need to consider such as ethics the laws compliances in my example because The ml based solution is tied with wellness, which is data personalized data I I need to spend a lot of time With hr legal department as I mentioned during my phase approach When I was building the roadmap, but even for the data science model You should ask these questions to the data engineer the machine learning engineers the machine learning scientists How they treat a data how what they understand about ethics and laws What is the model interpretability and so and so forth so that you would build the right solution Using the data science and ml for the problem to be solved How would we know that we are going in the right direction and then I don't know who but somebody said What's the point of running if you are going in the wrong direction? So that is the idea behind objectives and key results. So Here I have just like an illustration that How would I know whether my adoption or even my engagement rate is what it's supposed to be So, you know, like I have a a dial diagram that represents the number of visits the number of Early adopters who follow the routine that wellness program is asking them to follow But they do not log the progress update. So there is a gap in data Also, there's a set of users who follow the routine but also log the data as they progress, right and then As with as I mentioned that in the maturity Of the my I want this product to be tied with the The wellness of the employees the productivity of the employees Well, that's a wrap If you have any thoughts comments questions, please feel free to reach out Be at LinkedIn. That's my handle and once again, thank you very much for being here and thank you product school