 Hi, my name is Isaac Lamont from Johnson Johnson enterprise senior product manager there Really excited about the topic. We're about to dive into Something I'm really passionate about it's a celebrated business outcomes And we believe we have a decent frame with the help teams did it have we've been leveraging within that J&J So at a high level we're gonna just provide an overview of what OKRs are because really if that if you ask No, ten different product managers ten different Product leaders within the industry what OKRs are you may get ten different answers? So we want to just love us that as to How we are defining OKRs be seven teams that we're engaging and then talk through why you want to map outcomes to OKRs And and why we were pushing to do this with the teams that were engaging and then walk through via mirror How to actually do that how to actually tie your outcomes OKRs how to shape your key results based upon behavior changes prioritized What you should be focusing on no prior ties we should be measuring in a different approach to do that That's there may be some instances where you know exactly we should be measuring there Maybe some instances where you need to experiment So we wanted to walk through that and provide some guidance on how to do that as a best practice So some some folks may ask how do you define OKR? So that's what we have here, right? OKRs which stands for objecting to is also folks that they're not aware are an agile Leadership and goal management framework which helps organization to become more adaptive dry continuous learning development And create engagement accountability for company objectives and teams. They really help teams Reach their desired business outcome, right? They provide a clear unified definition of what success looks like They identify the key behavior changes to achieve and drive meaningful impact for both the customer and the business And for us, they're a great way to know that we're on track to achieve the expected outcome that we're looking for Right? So those key results which I'll talk about in a few are our way to do that They help answer why and which customer behaviors are key for us to focus on to achieve the business success that we're looking for Hence, we're such huge fans of them. And then lastly, they help align the product roadmap Features key actions user stories capabilities, you name it with the desired outcome So really driving a customer centric product approach, especially if you're focused on driving the customer-centric approach You're focused on driving the custom behaviors needed that are key for business success So the challenge with what we see around Okay, ours is that sometimes not a lot of time but sometimes We're not too clear on the outcome that teams will achieve as a result of addressing the objectives that make up their okay ours a lot of times We see them as outputs very task focus very deliverable focus very milestone focus like project driven And then they don't always guarantee success. So for us, we wanted to take a step back like they're okay great These are okay ours But what is the outcome you're trying to drive as a result of this and this is what we wanted to map them back to So for us outcomes Drive the desired impact result they achieved the right output They make sure that the output you're focused on are the right ones hence helping you prioritize even better They are measurable. They did demonstrate some level of success Josh Siden who wrote the book a recent book outcomes over output We love his quote because it resonates quite well as to what we believe is that outcomes are useful in the world Where we're not sure that the thing that we that we make will have the result that we want So if you're not sure the thing you make have the result that you want Sure, you're starting with the outcome and that's what we're trying to drive We're trying to make sure that we have a connection to you to so that Whatever we're making and wherever we're focused on that priority is solely focused on achieving the outcome on behalf of the customer in the business So you may say Isaac that's great No Tactically, what does that look like right? So outcomes I mentioned before a meaningful impact on behalf of the customer business and stuff title of the key stakeholders A lot of times they're part of their product vision. They're derived the product vision So if you're looking for your outcomes Go back to your part of vision if it's there fantastic just pull up in there It's not that means you have some work to do some examples of an outcome or you know increase revenue increase top-line sales Securing $3 million in cost savings. Those are examples of some decent outcomes that are that are you know that they have a target They're potentially measurable Objectives we see them as a business need a problem to solve to achieve the outcome So if you if you actually address the business need, what is the outcome? How do how do folks benefit from it? And then typically has or identifies to change in behavior, right? So an example that we have here on the slide is that it increased digital product descriptions by creating a seamless The life full customer experience across channels improving the adoption of our services key results There are a time to the objective We like to think of them as leading a lagging indicators, right the key results Shouldn't always be after the fact the key results should be measures that you're tracking along the way That tell you whether or not The problem you're trying to solve the business need the customer need is going to achieve the outcome that you define So the key results one thing to act that signals and they're a mixture of leading a lagging indicators Leading indicator for folks that aren't familiar our metrics which can inform teams on customer and market behavior to predict And assess traction, right? So once again, they give you some early indication whether or not you're going to achieve the result that you're looking for So you're not necessarily waiting until your product has as fully launched This is actually while you're going through the product lifecycle You're measuring along the way to determine whether or not by the time you by the time you've launched your product You have some indication whether or not you're going to achieve the outcome that you're looking for And the lagging indicators are observable measurable metrics that typically happen after your product is launched So these are metrics that track like the final step the final behavior change needed to achieve that no 10 10 percent increase in top-line sales and a lot of times the lag indicator will be that measure It will be how you will measure that 10 percent increase in top-line sales There are a couple of different frameworks you can leverage and support it is We're big fans of the heart framers and the prior metrics So these are examples of frameworks to help you form some of your indicates if you're struggling coming up but Leading a lag indicators leverage those frameworks and you can use that to inform what your key results look like So just to provide you some examples on further context as to like how these can be structured with with the large organizations like ours So imagine you have a product group that product with me have alcohol may have a vision But underneath that you have individual products, right that are responsible for achieving that vision And then within those different products, they may have their own outcomes themselves And you have individual squads focused on business needs customer needs to Achieve that product outcome sometimes is one to one sometimes as many as there's many to one in relationship So wanted to provide visibility and different dynamics and what this looks like So for example product a they have a specific outcome there And then the squads within that product They may be focused on specific problems to solve to achieve that outcome and they all may be measuring Leading and lag indicators that are instrumental to achieve that outcome So they have clear visibility connection to The stuff that they're working on which is their objectives what they're measuring Which are the key results and the outcome that sits above that so when it provides some visibility as to As to what that may look like in large organizations and how you would structure teams structure squads and support achieve an outcome They're looking for and objectives that they're supporting So one example that's non-changer related it's it's Something that we came up with within our team is the ice is the example of ISIS piece of shop So imagine if I had a fictitious piece of stuff that existed and I was treating as a product No, my vision My vision statement outcome is trying to drive by if I actually own this piece of shop would be to make this one of the Preferred eateries in town right make is make this the top pizza shopping time. I want to have the best pizza And I think if we have the best pizza in town There's a good chance that sales were increased by 20 percent quarter over quarter, right? Fantastic, that's that's my outcome. That's my my mission my vision statement, but If look underneath that I have a couple different squads That are responsible for specific problems and I'm made needs in order to help achieve that outcome So for example the order management squad Their objective is to ensure customers are presented with their orders on time and info So they rate us higher but there's a belief if we solve for this problem There's a good chance that it would actually impact his outcome Same with the customer satisfaction squad Their objective is to increase VP customers so they will spend more in our overall revenue increases If you notice in both of those scenarios There's a problem to solve. There's a many there, but it's also behavior change right in the customer satisfaction That behavior change that we want them to spend more So great. These are our objectives are tied to the outcome Then you know that can then and we know the behaviors that can then inform What are some of the leading and lagging indicators to get there? So for the leading indicators tied to this first objective We want to see a decrease in customer order increase my work as I'm releasing New features as part of my release plan. I should see a decrease in customer orders Through as lead indicator and then if we're seeing a decrease that may be good good indication that the impact is like indicator Of on-time order for someone percentage No, bettering that and then and I've seen an improvement around the Opus table service rating And these are indicators that may take some time to begin realized, right? So if we're looking at customer satisfaction the thinking is if I You know release some new feature release some new capability on my website that would increase the number of loyalty at conversations That's the lead indicator that tells me I may be improving my overall net promoter score And I may be also improving my sales compared against the baseline, right? So these are leading indicators that that and that are behavior driven That show that if I get cheap X there's a good chance. I may achieve why which is my lagging indicator Right. So these are some decent examples You may ask hey, I think this is great But tactically how do I actually do this and we have a mural that we're gonna make available after the webinar that actually helps teams What actually achieving this the mural as I highlighted in the agenda a little earlier? Helps with mapping your OKRs to your outcomes Helps you prioritize what to focus on because once again Teams are being asked to measure a lot. How do we just prioritize and measure what matters? And then when it comes to actually measuring What is their approach in doing that right? How do you actually define a baseline? How do you know what what how do you facilitate a conversation and get teams tracking? And and when there are difficulties and actually identify we should measure How do we get teams to actually live into a mindset of a test to learn experiment To reduce their certainty around the behavior changes you're trying to drive So Um with this when introduced you to the OKR mapping tree, right? So I'm not going to take credit for this This came from uh, Jeff got how Uh, he has a tree called alcoz mapping that we thought it was like no just clicked for us Yeah, so we beg borrow stole that made a few tweaks to it And then applied OKRs into that to help the teams That we've been engaging it lately So we have at the top you have an outcome Which which as I mentioned before Which speaks to the impact the end result once everything goes well like how do we know that our strategy succeeded? That's the outcome. That's the the the tangible impact of the work that we're doing the benefit what we're doing And then we have underneath that the objective So what problems do you need to solve to achieve the outcome? Uh, so what's the unmet need what what's the output what what do we need to solve in order to achieve that outcome? and then we have Two sets of behaviors that we want to identify One is what we call a lagging behavior Which informs the lagging indicator. So these are the final behaviors That we can observe And or measure the indicator we're going to achieve our outcome, right? And I'll talk a little bit that in detail in a few as to what we mean by that But essentially is Where we phrase it and the way we get teams thinking about this is like who does what like what needs to occur What is a customer when a key stakeholders in order? As a final step in order achieve the outcome that you're looking for right and everything before that can be treated almost as like A leading behavior or behavior that form is in a leading indicator So these are behaviors That what that lead to the behavior change that we see above right? So you can imagine this tree can become pretty complicated in detail and either may be a lot of indicators you want to measure So once again, we have a framework to help teams figure it out But essentially what you're doing is you're mapping all the behaviors needed to achieve The outcome that you have only at the top of this tree All right. Um, the one thing I'll say is that before jumping into this I wouldn't immediately go into this I would actually do a little bit of homework around this that there should be a bit of discovery. There should be a bit of ideation You should you know A better understand what your problem is you should better understand who your customers are You should have a good sense as to what that journey looks like for them for whatever product or solution you're standing up Um as if you jump to this without reducing their certainty around those elements, this can become a trap um So it's highly recommended that you get a better sense of of who your customers are what properly you're trying to solve If what you're doing is actually feasible and is what you're doing is actually viable And then you can figure out. Okay, you know, how do I shape my okay? Yars the support of Of showing that I'm learning and making progress towards my outcomes All right, so um as mentioned before Um, it this this does appear as a tree so you can see that here Uh, so you have your outcome statement at the top You have your objectives that support that that outcome statement and you can have multiple objectives You may have multiple squads or you may be solving multiple problems to achieve that objective. It's perfectly fine Now underneath that you're going to Identify the lagging behaviors Like the final behaviors that you can observe indicate you're going to achieve that outcome So these are the ones that are a little in the orange color now that in the below that are all the leading behaviors Right all the behaviors that that would lead to in a form Uh, the ability to impact their behavior above that that ties to your outcome So one example of this that's a healthcare related example Um, that actually has no connection to jj, but we figured might not just use the healthcare related example um, so in this example, we have the outcome of High quality patient care result in the increased customer loyalty Increased in trials and market share. So the thinking is if we improve Um, uh the quality of patient care, this will lead to increased customer loyalty They'll be wanting to try out Um, you know some of our products that we have available result in the increase in market share and then That could be a couple objectives tied to this. We identified two that are somewhat related one is Related the director related directed to our customers, right? so if we if we drive continuous learning and improve The customer satisfaction for health care professionals this an office staff There's a potential that it will Drive this outcome that we have here and then the other items are around the business, right? If we focus on increase the sales of health care products through a seamless experience There's a good chance It would impact this outcome here and then as I mentioned before there was some homework that was done Prior to this to get to this point. Uh, so once again, I wouldn't jump directly into this You should have some recognition You should have validated that at a high level that these objectives may potentially Achieve the outcome that you have here And if you notice these two objectives are actually related, right? So um as they should be So just wanted to call that out So for example, for this objective drive continuous learning and improve customer satisfaction for health care professionals and office staff the behavior The once again the final behavior that we would like to observe In order to achieve this objective is that the hcp Hcps the health care professionals become more satisfied Uh with the education that's being provided the education that we may be providing around our products and procedures, right? That's the final behavior. That's the last thing we need them need to observe in order to show that That they're satisfied We're proving the customer satisfaction We're improving the uh when it comes to continuous learning for both the acps and office staff Which may have the potential impact of of the outcome that we have here at the top right So great, that's the final behavior. So what are some of the leading behaviors that we have identified? Uh in this statistic example to help get to this final behavior, right? Uh three we identified where acps Click more call to action links ACPs visit our practice resources page more often So we're providing this this this education material or are they click on the links that we're providing? Are they going to the the practice research page more often? And then lastly if they're rolling into the loyalty program That's a good indicator that they may be more satisfied Uh with the education that's being provided, right? Uh, and then if we look at The objective related to the business right that the business really cares about they want to see increased sales of healthcare products through the same experience For them one of the final behaviors they want to observe is the patient requests and buys more of our healthcare products That's the final behavior that shows that there's a good chance that we'll be able to achieve that The the outcome and objective that we have here Right the steps required before that and once again these these are all connected the acp trials Uh more patients with the healthcare products and the acp feedback improves via customer satisfaction survey These are things that tell us that Well, no these are behaviors that tell us that there's a good chance that The the patients request and collaborative acps and buy more of our of our healthcare products Right, so these are behaviors, right? These aren't necessary measures yet But these are behaviors that we identified and prioritized I'll be critical in order for us to achieve the outcome and objectives that we have here So you can imagine this tree can these trees can be pretty detailed pretty long You may have multiple steps and multiple behaviors as a result We want to prioritize the behaviors we should focus on right? We want to prioritize the behaviors that drive the most insight Um In support of validator and validating at the work that we're never focused on will achieve The outcome that we have here at the top And the way we do that is that we leverage what's known as a prioritization metrics And the prioritization metrics are based on two elements The concept of insight and effort is a two by two matrix That helps us identify. What are the metrics that matter? What are the metrics we need to focus on in order to achieve? Uh and and confirm that we're on track To achieve the outcome that that that we've identified So by insight uh insight refers to Uh, how much this indicator reflects progress on achieving the objective and outcome For example, um in terms of reflecting revenue increase. Let's make that's the outcome Measuring whether a customer adds an item to their shopping cart would have higher insight than measuring whether the customer clicks on a product page right so for us No adding an item to the shopping cart and checking out if if my outcome was to Um, you know improve revenue Uh to me that's that's a measure. I would like to track a track and measure Um, when it comes to effort effort refers to how much time Knowledge and resources are needed to based on track this indicator So if it's if it's something that's highest high effort go do it right if it's something that's high insight by high effort Then we need to re-evaluate how we approach that right it is possible that squads and teams Uh, do not know how to measure this metric That is high effort. Uh, this means that there's great uncertainty around this behavior change It may be opportunity to reduce the risklessness and achievement and outcome Which we'll talk about in a few Right, so this is an example of the uh two by two matrix. So as I mentioned before so, uh, the the y-axis Focus on high insight and low insight The x-axis is focused on low effort and high effort We really care about the items that are above this line here, right? Uh, so something is high insight low effort. Let's go do it. It's something that's high insight high effort Let's figure out how we can approach and addressing that but ultimately, uh, we want to focus on what goes above this line Right. So as an example from the, you know, okay our mapping tree that we're on earlier Um, so what we've done we've we've taken the sum of the behaviors Now that we've identified from the okr mapping tree and then we started to pot them out with this two by two And then we've identified Basically five items that are truly above the line the five are Once again our combination of leading and lagging behaviors, right that have formed that will inform our leading and lagging indicators Uh, one is around patient requests and buys More of our healthcare products another one, which is a lagging behavior and that goes around ACP or roles in a loyalty program So these are high info high insight low effort behaviors. Let's go measure Then you have high insight high effort behaviors. You have acps click more Call to action links acps become more satisfied with education and acps trials More patients with that healthcare products. These are these are all somewhat difficult to measure And then those in the upper right hand corner or are probably the most difficult to measure And so we have an approach to actually address that So With that being said, that's the example of the two by two prioritization metrics. You may ask is it great What's next? That's that's where we go to our measuring experimentation Section of the mural You know, we broke this into two sections because remember there were high insight low effort Indicators and behaviors we want to track and there were high insight high effort behaviors that we want to track So we provided a way to actually go after both. So for the first one, we're going to focus on high insight and low effort indicators And then we'll talk about how to baseline and track them So what we've done is, you know, which see here before you are test cards Fortunately, I can't take credit for these. Our team can't take credit for these. We stole bag them and took these from Alex Oswaltzer and then David Blaine from strategizer from the book testing business ideas It's the combination like if they're test cards and learning cards We should find extremely valuable and elbow teams to measure to identify baseline and measure their Their key results are leading and lagging indicators. So you what you're seeing here is like a different variation of that so In this example For the high insight lower for indicators. The questions are asking Is we want to make sure we understand teams objectives. We want to make sure we understand the behavior the indicator That's tied to that jacket that we're looking to measure We're going to ask the question. What exactly will you measure? Well, let's ideate and talk through what exactly we're going to measure Let's ideate and talk through how you measure it and let's ideate and talk through How do you know you're on track to achieve your outcome based upon what you have here? So for in this example Now based upon our prioritization metric in the in the upper right um, the objective that That uh, that this Indicate is tied to is drive continuous learning to improve satisfaction for health care professionals office staff One indicator we want to measure is acps enroll in the loyalty program fantastic, right? What exactly will you measure? We will measure percentage increase of acps enroll in the loyalty program each month, right fantastic from there How will you measure it? There's data. There's a data repository that stores the current acps or our role in the loyalty program We will compare the current most number with the previous very tactical You're measuring progress month over month There's a timeline and these are these are early indicators that the work that you're doing here And addressing this objective may potentially achieve the outcome that you're looking for And how do you know you're on track to achieve your outcome? Well, you want to see a five percent increase month over month So you have right now just a baseline Of of your metric and through conversations through ideating with with the product teams you have something to start with Now for high insight high F indicators That are that are high effort The reason our high effort are a couple of reasons, right one It may be a result that they're hard to measure and we don't have something in place to measure, right? So this speaks to the feasibility of what you're building what you're doing It may be because we don't know if we can impact them, right? We don't know if we can actually drive the behavior change if we want once again It's tied to the assumption of making around the feasibility of what we're building We may make assumptions around the behavior change in tracking this right and we may make assumptions around How this indicator reflects progress and inject them the outcome So so once again tied to Desiability and the viability of what we're building right so these high insight and high effort indicators Will inform whether or not you're reducing the risk and uncertainty Around the desirability feasibility and the viability what you're doing early indicators All right, both leading and lagging sometimes But a lot of times early indicators that tell you whether or not you're attracted to the outcome that you're looking for Right, and then if you can figure these out, this is how you can accelerate business outcomes using okrs, right? so in this Part of the mural the questions are a little different in comparison to the other No, high insight low effort Indicators that we identify so some of the questions Are the same like the first two questions like what is what is the objective? What is indicator plot those there and then you're going to ask What needs to be true in order for this indicator to produce high insight Into achieving the objective and the outcome How will you test this hypothesis and upcoming springs? What exactly will you measure? How will we know the assumption is correct and what will we do if the assumption is wrong? So this is our kill criteria this last question. So if our assumption is wrong That means that there's a probably a good chance that we will not achieve The outcome and objective that we identified we may need to pivot. So that's why we're calling that out All right, so let's let's talk talk to an example So if you call we had one objective Drive continuous learning and improve customer satisfaction for healthcare professionals and and office staff One behavior one indicator, which is a lagging behavior that we wanted to Track that we prioritize that acps become more satisfied education pretty difficult to measure All right, we're making an assumption also there. So what is that assumption? Right, what needs to be true in order for this indicator to produce high insight to achieve the objective and outcome And and what needs to be true is that we believe providing higher quality product education Will make acps more satisfied big assumption So how you're going to test that how you're going to validate or invalidate that hypothesis that assumption One way you can do that is by doing a split ab test Comparing the old education page to the new proof education page, right? You gave me do that before you've been launching and rolling it out Just on a simple test with a couple of no acps and then from there you can get MPS feedback from the acps and the two product education page And and that's what we're going to measure we're going to measure an nps score And then we'll know That our assumption is correct is if the nps score of the new product page Is greater than the nps score of the old product page And if we don't see a significant increase or if we don't see if the nps scores are relatively the same That's no then we know we need to pivot. That's not we need to do a little something different With that page so that we can impact and improve the score Because if we're not improving the score, there's a good chance that acps will not become satisfied with the education Something different has to happen there to drive that behavior change Another example is acps click more call to action links So the assumption that we're making there because this is we identified as something is High insight by high effort is that now we believe acps Clicking more call to action links will correlate to acps being more satisfied as an assumption Right, and that's a that's a behavior a leading behavior that's tied to this lagging behavior So to test this Now we're going to have it test acps navigating the site track their call to action clicks with the google analytics And we're also going to potentially even send the acps an nps satisfaction survey afterwards Right, and then the measures we're going to track is And a measure is the number of call to action clicks per acp And the nps score from the survey and then we know that our We know that our assumption is correct Is if acps with a if acps will hire a number of call to action clicks We're more likely to be more satisfied the assumption is validated but We know that we're wrong is if If we will choose a different leading indicator since there is no correlation and dust is not a good predictive future performance so if you realize that We're not getting the measure that we're looking for and we see that that assumption is correct We need to re-evaluate the leading indicator that we have there and figure out something else to measure whether or not We'll be on track to achieve the outcome that we're looking for So I hope that was um pretty clear No, once again the framing that we have here is to help teams The prioritize and measure really matters and use behaviors to accelerate business outcome as typically If you know what your customer behavior is if you know what your business behavior is The typically is that that is going to be the key metric that informs whether or not you're going to achieve Some level of business success thank you for the time and then No, please don't hesitate to reach out always looking forward to support the community and then The material and content that we reviewed today will also be made available made available We're looking forward to seeing how folks leverage this and then once again Thank you for the time. Take care