 It's a real pleasure and honor to have Jeff and Josh with us. I've had the fortune of working with Jeff at a client and we did some really amazing work there. And it's always a pleasure to learn from him. I've not had the fortune of working with Josh. We were just talking about this earlier, but we should try and get him down to India to do some interesting stuff. So this is a very special webinar for me at least. And I'm sure for a lot of folks in the Agile India community, it's been a while since we've all connected. So it was really nice of Jeff to reach out, say, hey, he's up to a new book that is coming out. So Jeff and Josh are publishing their third book together. And this book has got a very interesting title. I was quite intrigued by the title. The title is Who Does What By How Much? Very, very straight cutthroat just to the point. Like that's the question a lot of people have to love the title. And I'm sure all of you would have guessed it by now it's a book on OKRs, but not just any OKRs, right? Like customer centric OKRs and more from an outcome perspective, right? I know Jeff Patton, who's another good friend, has been talking about outcomes over output for a while. And so all of the Jeffs are connected and it's kind of good. So again, without talking too much, I'm just very excited to welcome Jeff and Josh. I want to hand it over to you guys. You're going to run through a quick little presentation about 35 minutes or so. Then we'll open the floor for some questions. We've got a few lightning talks which will happen after that. And so we will welcome folks who got selected for presenting the lightning talks. And then we'll do a bit of a fireside chat with Jeff and Josh. OK, with that, I will shut up and over to you, Jeff and Josh, take it away. Excellent. Hey, folks, pleasure to be speaking to the Agilindia community and thanks, Nuresh, for having us. Josh and I are going to, as Nuresh said, going to share a presentation and then we'll open up the floor to a bunch of discussions. I'm going to start first with the beginning of the presentation. I'll hand off to Josh for a chunk in the middle and then I'll wrap it up and then we'll do some Q&A. So let's jump into it. So objectives and key results is the theme of this. But we're really focusing on this idea of who does what by how much. And so let's start by sharing stories. So this is me, as you can see, four years ago, four years and a month ago, almost to the day. Those are my daughters at the time. They're much older now, four years older. We went to a Barca game in Camp Nou in Barcelona. And if you've ever been to the old Camp Nou in Barcelona, you know that, first of all, it was a big stadium, about 100,000 people. And it was very, very small and compact. In fact, they've torn it down and they are rebuilding it now to make it a bit more comfortable. But we were there on February 22nd with 100,000 people just kind of cramped together. It was a beautiful day. Look how warm it is. Shorts leave shirts and everything screaming at the top of our lungs to cheer on our favorite football team. Having a great time with 100,000 people and everything was awesome. It was such a nice day. And beyond that, my family was in town and we had plans to hang out with them and to travel. I'd work travel coming up in the next couple of weeks. We had all kinds of things planned for the future. And then literally a month later, this was the world that we lived in. Literally, I think it was March 14th, less than a month later. You know, I'm talking to my friends on Zoom. I'm wishing my parents a happy wedding anniversary on video chat. Because the pandemic happened and locked everything down and completely changed all of our plans. Everything that we had going was completely thrown out the window. And in retrospect, especially at the time, it was a bit shocking and jarring, right? But in retrospect, really, it kind of led me to a bit of a realization that regardless of how much we think we can plan ahead and be certain in those plans, the reality is that we're constantly guessing. We're constantly guessing because we can't predict the future. We can only make the best guesses about what we're going to do, how we're going to behave, what we're going to plan based on what we know right now. The only thing we can base any planning on is on what we know right now. And this is a fairly uncomfortable truth of life where you just sort of embrace it. But it's also true at work as well. The reality is that we can't predict the future. We can only make the best guesses about what to build, how to make it, who to make it for, and what we think they'll do with it based on what we know right now. Now, what's fascinating about that is that when we make those guesses about the products and the services that we build, when we think about what should we make? How are we going to plan it? What are we going to build? The reality is that the time it takes us to learn whether or not those guesses were correct or whether those guesses were wrong has improved dramatically. So when I started my career back in the late 90s, this is what software, how software was delivered. It was delivered on top of CDs. And I worked in America online. And if you were lucky enough to live in the US, the UK, France, or Germany in the late 90s and early 2000s, you were lucky enough to receive one of these CDs. You're welcome, by the way. We would work for months on this six months, nine months, sometimes a year, and then we would stop and we would print 15 million copies of these CDs and send them out in the mail. And then we'd wait. Did people receive them? Did they use them? Did they install the software? What did they use? And how did they use it? And the feedback loop for learning if the guesses that we had codified onto that CD were true, that feedback loop took anywhere from six months to 12 months and sometimes longer. But the good news today is that the time to learn, if your guess was right, has shortened dramatically. This is a chart from Gene Kim's amazing book, The Phoenix Project, where he talks about how frequently the tech giants in the world, the Amazon, Google's, Netflix, Facebook's, et cetera, of the world deploy code to production. And you're looking at Amazon deploying 23,000 times a day. That's crazy. But the amazing part of that capability, and I understand that that's the extreme capability, but the amazing part of that capability is that the time it takes for us to learn whether our guesses are right or wrong has shortened dramatically. Literally, it's as fast as we want. As fast as we want to find out, and as a company we're willing to invest, we can find out if our guesses about what to make and how to make it were right or wrong. On top of that, we've got a whole bunch of tools and methods and techniques to help us reduce the risk of guessing incorrectly and wasting efforts. You take the technology and on top of that you layer things like agile and design thinking and lean UX and product thinking and all of those ideas. But in theory, we should be able to reduce the risk of guessing incorrectly and building the wrong products and delivering the wrong services with a significant amount of accuracy. But the reality is, especially with the clients that we work with on a regular basis, is that all of these efforts get hamstrung. They get hampered by one specific thing, this one practice that continues to slow down our success and improving our guessing and course correcting more quickly. And that's this idea of managing the outputs. Now managing the outputs is this idea is basically embodied in this concept of make me a thing, right? This is when someone says, I have a requirement for you to implement. I want you to make a traffic light and needs to have these lights. We're going to install it here and it's going to be beautiful. Make me a thing is managing to outputs. And there's a tremendous amount of risk involved in that because we don't actually know if this is the right thing to build. We don't know if this is the right way to build it and we don't know if it's going to work in the real world. We don't know what people are going to do with it. So instead of make me a thing, what we're trying to do is we're trying to get to a conversation where we're talking about solving a specific problem. There's a fundamental difference between make me a thing which is managing to output and solve this problem. The problem here is we want to get these folks across this intersection safely and consistently on a regular basis. And that is a fundamental difference in the way that we discuss the products and the features and the services that we build. We've got organizations that focus on managing to output, which is trying to predict exactly what's going to work and is a high level of risk there. When instead we want to build organizations that manage to outcomes, which are going to be these measurable changes in behavior that tell us that we've actually delivered value. There's lots and lots of different outputs that may deliver those specific outcomes. Now look, the reality is this, there are situations where managing to output makes sense. If there's a situation where you know exactly what the end product is going to look like, you know exactly what is going to go into it. You know what it will cost to make. And most importantly, we know what most people will do with it. We've got high certainty and low risk. In those situations, we can manage to output. We can just say we're going to build this thing, it's going to look like this, and this is what's going to happen. And there are situations where managing to output makes sense if you're building a bridge, for example. But as we know time and time again, the future is unpredictable and confidently predicting, especially digital products and services is a highly risky endeavor. So instead, we have to base our guesses on what we actually know right now. If we're going to start to think about building successful products and services, instead of trying to predict exactly what people are going to do and what we should make and how we should make it, we've got to work with the information that we have right now. The information that we have right now is made up of at least these four things. We know what people need to do in the systems that we're building. We know why they're engaging with us. We know what they're actually doing in the system. We can measure that, what they're actually doing in the system. If we're lucky and we get access to our customers, we can ask them why they're behaving in that particular way. And we know right now what we need as a business to be successful. Those are the things that we can use to start to think about what we might build. What we don't know is exactly how to solve our users' needs. Trying to predict what products or features or solutions make sense now runs the risk of being wrong in the future when we actually build it. It's risky work filled with untested assumptions that we believe will drive some user behavior in the future, but we can't prove it. And so instead of managing to building a thing, we want to manage to human-centric goals, right? To setting goals away from output and towards outcomes. And that's where objectives and key results as we see them come into play. Now, the way that we defined OKRs in our book is that it's a goal-setting framework for teams that defines a qualitative goal. Why are we doing this? What's the benefit? What's the change we're trying to create in the world for our target audience, right? We want to create the safest way to cross a street, right? Something along those lines. And then we measure success towards that goal by achieving specific outcomes. What are people doing differently to tell us that we've now created the safest way to cross a street? The thing that we're going to move away from is setting goals that are specific to output, right? We're not going to commit to making a specific thing. And at the core, at the core of this idea of objectives and key results is this foundation, the foundational idea that everyone has a customer, right? This idea that we make something for somebody else and that person who consumes that thing that we make, that is our customer. Now, look, if you work in a B2C environment, that's an obvious statement for you, right? Your customer is the end consumer, the person that's buying that from you. So this perhaps isn't a revolutionary statement, right? But for example, if you build systems or services or policies for the vendors for your company, right? Those vendors are your customers, right? So for example, when Noresh and I work for a client, right? The onboarding systems, the procurement systems, the cybersecurity rules, all of those things are products and services that are made by generally the procurement department. And we are the customers of those. We are the customers of the procurement department. If you work in a B2B situation, right? Your client buyer, your client users, those are your customers. And for those of us who work on systems that our colleagues consume, right? So we make systems that other teams inside the company consume. Those are our customers. There's a human at the end consuming every single thing that we make. The job is to try to figure out how to make that human more successful. And the way that we know that those humans have become more successful is we try to measure how they behave. That's our goal. But there's an interesting thing with humans, it turns out, is that even if you think you know them well, and even if you think that you can predict what they do and how they do it, the reality is that human behavior is complex and unpredictable. We don't know exactly what people are going to do, and we don't know why they're going to do it. And on top of that, even if we do manage to nail the behavior upfront, we kind of have a good sense of what people are going to do and they start to behave that way, what's interesting is that over time, that behavior isn't static, right? Just because they started doing one thing first, it doesn't mean that they're going to continue doing that for the foreseeable future. Behavior often emerges through the use of our products. Facebook is a really great example of this. Initially it was just meant to connect folks from the same schools and the same universities. And these days it's used for a whole lot of different purposes that I'm sure the founders of Facebook didn't have in mind. But behavior is a moving target and it emerges through the use of our products. Alright, so Jeff just talked about this challenge that we have, right? When we're thinking about delivering value to the market, we're trying to create value for people, right? And so we're trying to understand what they're doing, what they're trying to do, and we're trying to help them do things that are valuable for them and valuable for us, okay? That's the whole idea behind this idea of outcomes. And so if you've had a chance to read my book Outcomes Over Output, or if you've spent any time listening to either me or Jeff, you know that we use this definition of outcomes. And on the next slide, the definition that we use, and we've been teaching this for five years now to teams, and it solves a really important problem, which is that, especially for agile teams, who want to be more outcome-centric. One of the big challenges is that this word kind of just means whatever we want it to mean. It means results or it means goals or it means targets. And so what we've been trying to do is be very, very specific with what this word means. And so this is the definition that we've been using. An outcome is a measurable change in human behavior that drives business results. And what that does to have that specific behavior, excuse me, that specific definition, is it lets teams start to use this as a tool to plan their work, to deliver around and to measure the quality of their work. So this has been a really powerful, powerful sentence for us. It's also kind of a wordy and geeky idea. And so over the years, what we've discovered is that we can simplify this. So let me explain this definition to you in a little more specifically, okay? So a measurable change in human behavior, this definition contains three questions. The first question is who, right? When we're talking about a measurable change in human behavior, the first thing we need to do is identify the who. What human are we talking about? Sometimes when you work with big companies, you'll ask them who the customer is and they'll say, oh, our customer is the bank. Our customer is the merchant, right? Our customer is the manufacturer. Well, that's not true. The customer is a human who works for a bank. Your customer is a human who works for a merchant. So the first question is who? Measurable change in human behavior is who? The second question is what's the behavior that we're interested in? What's the behavior that they're interested in? Does what? That's our question. So what are they trying to do? How can we help them do it better? How can we help them do things that are valuable mutually, right? So who does what? The third question is by how much? Okay. And that's the measurable part. So we want to be able to use this as a tool. It becomes a valuable tool when we can use it to measure precisely the effect of our work. And when you take these three questions and you add them up, you get this kind of very simple statement that you can ask to try and think about the results. Of your work. Who does what? By how much? This is where the title of our book comes from. Because this sentence is the core, the kernel at the heart of OKRs. And Jeff is going to talk about that in a couple of minutes. The reason though, let me just talk about the reason that we are so insistent on the idea of outcomes, right? We have lots of tools that let us talk about output about the stuff that we make. We have requirements and we have specifications and we have user stories and we have epics and we have all of these tools that are important tools for us in our work that help us be really, really specific about the stuff that we're making. Those are great. We need those tools. But we also need tools that let us be really, really specific about the value that we are creating. And outcomes are that tool outcomes. Give us the tool to be really specific about the customer value that we are trying to create. And so when we can be specific about who does what by how much. And then we can put our solution into the world. Outcomes will tell us when we have delivered customer value. Okay. We can put our solutions in the world. We can see who is doing what and by how much, right? And it will tell us when we're done working. Okay. All right. So how do we use outcomes then in the context of OKRs? This is what we're here to talk about. We're talking about OKRs setting human centric goals. Jeff is going to talk to you about how we use those questions. Who does what by how much within the context of OKRs. Excellent. All right. So that's the foundation. What does this have to do? How do we bring all this together and set goals, as Josh said, with OKRs? So we're going to talk about the two halves of OKRs. Objectives first and then key results. I'll show you a couple of examples. We'll wrap it up and then we'll do, we'll do Q&A, right? So if we're thinking about objectives. Objectives are the goals that we're trying to achieve and our intentions. An objective is a qualitative goal. So usually we're going to see a lot of superlatives in there. The easiest, the simplest, the most efficient, fun and engaging, things like that. We're looking for those types of words in our objective statements that are inspirational and aspirational. We talk about the value that we're going to create in the world and how that value is both the higher purpose of the business and perhaps of our target audience as well. And we want those objectives to be time bound. And I want to be just super deliberate about this because you'll notice that there are at least two things that are explicitly missing from this goal statement. If these are the bullet points that describe your goal, right? There's two things that are missing from this. Number one is features. There's no features. There's no solutions. There's no services that are mentioned as a part of the objective statement. That's number one. And the second thing that's missing from here is metrics. In our objectives, we are not going to put metrics, no features and no metrics. We're just going to talk about the state of the world that we'd like to achieve in the future. And the way that we get there is by starting with the problem that we're trying to solve, right? And remember, we're going to start with what is the problem that we're trying to solve, not the thing that we're trying to build. We're going to focus on, again, what we're trying to solve for, why it's a problem, and what do we know about the root cause of the issue, right? Here's an example. Hey, hypothetical example. We run a continuing education learning platform for lawyers. We've noticed that our customers sign up for the service. They take one course. They leave. When they leave, they tell us that our user experience was the main reason for them leaving. Now, this is bad for us because it drives up our acquisition costs, our sales costs, and it reduces profit. That's the problem that we're trying to solve as a business right now. We look at our problem statement and we ask ourselves, if we solve the problem, what does the world look like? In other words, if we take that problem statement and we turn it into a positive statement, and I want to be clear here, I didn't use the phrase, how will we solve the problem? The phrase is, if you solve the problem, what does the world look like? Well, it could look like this. Build an engaging continuing education product for the entirety of a lawyer's career by the end of the year. What we've done here is we've taken our problem statement and turned it into an objective statement. It's qualitative. It's aspirational and inspirational. It's quality and engaging continuing education product. It's aspirational and inspirational. We want to do this for the entirety of a lawyer's career. That's the value both for us and for them, the inspiration to come to work every day. And we'd like to get this done by the end of the year. It's time box. We've got a deadline that we'd like to achieve this by. And so we've written an objective statement. Now, what's really important, and we see this a lot with the teams that we work with, is we have to set those objective statements at the right level. And so it really depends on where you sit in the organization. And if you're in charge of a business unit or a program or a suite of products or just an entire product, the goals of that business unit or that product or that program can be your objective. But if you work on a team that's in charge of just one slice of that customer journey, let's say it's onboarding or authentication, your objective should reflect the goals for that part only. In other words, only for the part of the world that you can influence. That's where your objectives should lie. We often see a lot of teams sign up for revenue goals or whole company goals. But in fact, their sphere of influence is the authentication part of the experience. So we want to make sure that you're signing up for an objective that you can achieve as a company, as a team. That's objectives. Now, when we think about key results, key results answer the question, how will we know we have achieved the objective? Now, in this case, your key results are metrics. They are measurable milestones. They are quantifiable. So it's just to be super clear, your key results are metrics. Ideally, they are ratios or rates, 25% more of this, 30% less of that, because that gives us a sense of how we're trending. They should be aggressive. In other words, we should be hitting 75 to 80, 85% of our key results on a quarterly basis. If we're crushing our key results every quarter, we're probably not trying hard enough with those goals. Your key results should be verifiable with evidence. So we should be able to look at analytics and reporting to check those numbers. And most importantly, where all of this comes together is that your key results must be outcomes. They must be measurable changes in human behavior that drive business results. You know that you've written a good key result when it answers the question, who does what by how much? And this is what makes OKRs different, and this is what makes OKRs difficult, because we're so used to setting the delivery of a thing as the measure of success. And now we're saying the measure of success is we've changed the behavior of a specific set of humans. And so you'll notice once again, this time we do have metrics because the key results are metrics. But we again don't list features or solutions or products or services in our key results. There are no features mentioned, not in the objective and not in the key results. This makes this different and difficult. The way we get to good key results is that we start by defining our target audience, right? Whose behavior do we want to change? And that could be current customers, prospective customers, as we said before. Like who is your, everyone has a customer. Who is the human that consumes the thing that you make, right? And then we ask the key question, what will those people be doing differently if we achieve the objective? So for example, in my online learning service for lawyers, I want to see a 50% decrease in the number of existing customers leaving the service due to user experience, right? And again, here's that questioning in who existing customers does what leave the service less by how much? By 50%, right? And that's how we know we've written a good key result. And so when we kind of put all this stuff together, right? We can end up with objectives and key results that make sense for our business, right? We want to put it together and make sure that the key results do indeed tell us that we've achieved the objective. We want to build an engaging, continuing education product for the entirety of a lawyer's career by the end of the year. How will we know? We'll know because each user will take at least three courses each year. That's a 300% increase. 25% of new customers will come from word of mouth referrals. Each law firm that signs up registers at least 50% of its staff. Who does what by how much is answered in each one of those key results? Now, are those the right key results? Are those the right numbers, right? Those are assumptions that we're going to need to move forward and validate. But at the very least, we've put together a goal that now allows our teams to go out into the world and discover how to move behavior of our target audience in this particular way. I want to show you one more example of OKRs. And then I'm going to wrap this up and then we'll take your questions and move forward. But this is a great example because it comes from the real world and it comes from healthcare, which is not a world that we may often associate with OKRs. This is the Cleveland Clinic. They're an international healthcare organization. Headquartered in Cleveland, Ohio in the United States, but with locations all over the world. They publish their OKRs publicly every year. This is the CEO's OKRs. And I like a couple of the ones on here. This one, the best place to receive care anywhere. Qualitative. Yes, it's best. What does that mean? Inspirational and aspirational? Absolutely. The best place to receive care anywhere. That's why we go to work every day. And it's time-bound. It's by the end of the year. How will we know? Well, we see the absolute number of safety events dropping. That's an absolute number and the ratio of serious safety events. How frequently a serious safety event happening, dropping as well. Another one I really like on here is the best place to work in healthcare. Again, inspirational, aspirational, qualitative, but inward looking. How do we create a great place to work? And again, how do we know that we've created a great place to work? Because our staff is recommending it to their colleagues, to their friends as a good place to work. And we're seeing the percentage of regrettable turnover go down. Those are people quitting voluntarily and quitting. People that we don't want to quit are leaving less, right? That's our goal. Now, you'll notice again in both of these examples, we don't tell the teams what to make, which becomes a really interesting questions. And honestly, it's what makes OKRs hard, right? They are a goal-setting framework that redefines success away from output to outcomes, right? And setting goals should be difficult. It aligns an entire team or entire organization. But when we set goals towards outcomes, we create a much broader spectrum of possibilities for how we might solve those problems. And it gives us the opportunity to discover the best way to deliver value to our customers at the end of the day. And so with that, I'll say a quick thank you. We've got a book coming out in about a month or so. You can check it out at OKR-book.com. Thanks, Jeff and Josh. That was pretty dense, I would say. I'm sure you've done your 10,000 hours on this presentation. Very well polished and to the point, really appreciate that. All right. I think there's a couple of things I wanted to quickly touch upon. And I know these days, one of the things that everyone's been looking at is chat GPT. So what I thought is, let's test drive chat GPT and see what do you guys feel? How good is chat GPT in helping me come up with OKRs? So I'm going to quickly share my screen. I hope you guys can see my screen. So here's something that I asked chat GPT. There's a new open source product that I'm building called Spectmatic. I'm asking chat GPT. I want you to look at Spectmatic.in and help me come up with some OKRs for the next quarter. We don't really have a lot of tooling in place to measure this. Also, we just launched pricing plans and are trying to improve awareness, branding and adoption of the two. So that's the question I asked GPT. And GPT came back with these. In fact, it actually came up with two options, but I'm just going to put this first option up here. What do you guys think? This is terrible. Terrible. Yeah. Look at all those. Every single one of those key results is an output. Every single one. Not one of those key results is a is an outcome is a meaningful change in human behavior. And those objectives, like what's why do we want to increase? Spectmatic awareness and brand visibility. I mean, you could say it that way, but you know, but again, hosting for industry related workshops. That's an activity. That's not a behavior change. Why? What will people be doing differently after you host for industry related workshops? Part number two to three influencers. Okay. Why? What will people be doing differently after I partner with two to three influencers? You see what I'm saying? But you can ask that about literally every single one of those key results. They're all output based key results. And again, the objectives aren't, I mean, you could argue that they're qualitative. Yeah, so just go through the list, right? I would, I would say they're qualitative in some sense. Yeah. Inspirational and aspirational. Maybe really not. This going through your checklist, right? And are they actionable and action oriented? I would say kind of. Yes. Kind of. Yeah. The tasks. They're all tasks. Conduct. You can see, you can see it in the verbs, right? Conduct, implement, offer, create, set up, publish, host, right? The verbs, right? Who does what by how much is not answered by any one of those key results. Perfect. So all I was trying to highlight is it's done okay in my opinion on the on the objectives, but it's done pretty poorly on the key results. Is that the verdict? Is that a fair verdict? I'm going to, I'm going to object to the objectives. Here's the thing. Okay, ours need to be, and we didn't talk about this in today's presentation, but we talked about this in the book. Okay, ours are an expression of your strategy. And so what is strategy? Strategy is our opinionated approach to solving our most important problem. Right? It's our opinionated approach to succeeding in the market. So we have an opinion about what we're going to do and why we're going to be the most expensive electric car in the North American market. Because we believe that signals, that's an opinionated choice. We believe it signals that we are the highest quality, the most exclusive, the aspirational brand, right? That's a strategy. It's an opinionated approach to solving an important challenge. I have no idea what the strategy is here. The strategy is, it could, this could be a good strategy, right? We want to increase brand awareness. We want to create a community and we want to encourage adoption. Okay. That's pretty generic though. Correct. Because it's a very genetic prompt, right? I mean, there's no way GPT would know what your strategy is or what is, but what's interesting, it did come up with saying, hey, I want to build a community because in the prompt nowhere I've mentioned about building community or focusing on community, but it kind of suggested that. Anyway, that was not the point. What's interesting is the next prompt, which is basically same thing except that I want you to take a look at specmatic.in and help me come up with some customer-centric OKRs as defined by Jeff and Josh in the next quarter and everything else remains the same. Do you see any difference now? There is a little bit of a difference. So I'm just, I'm just scanning the key results. The purpose is to use this word customer more. Yeah, because he's asked. Customer, customer, customer. Increase the number of engaged users on the community forum by 40%. It starts to get into solutioning a bit, right? Active moderation and weekly Q&A and it doesn't really define engaged like what does engaged mean, right? So it's a baby step forward there. But then develop and publish a series of use cases, set up a monthly virtual workshop, create a first 30 days onboarding email launch and really adopt her. It's pretty much the same. So it's like a tiny, tiny, tiny. It's like a newborn baby's step forward. So GPP needs a lot more prompting and learning before they can come up with good OKRs. Is that the verdict? Yeah. Yeah, they need to steal more of our books. Yeah. Once the book is published, we can try it again. All right. This is just something I just wanted to play around because. That's very funny, Naresh. That's the first time I've been asked that question. So kudos to you. I find a lot of people are interested in, you know, can GPT solve things? And then you take a topic and then you get the expert to say, well, how well did GPT do on this? And you always get interesting insights from the expert that you may have not noticed. Sometimes I thought that would be helpful for people. Yeah. Next time at least you go and ask GPT for something. Maybe look at it from that lens. I mean, I think, you know, Paul makes a point in chat says, I think that's why everyone can say they have OKRs, but we're still struggling to make a difference. And I think, you know, I think like OKRs are a really promising framework, but that's, you know, and a promising framework that we both really like, you know, but you can use that framework well, or you can use the framework poorly. And so you don't like automatically get magic when you, you know, pull OKRs out of your pocket. Yeah. And I think the important point that you highlighted and I'm glad you did as because of this conversation was that there is a strategy, in the OKR, right? Which I don't know how many people actively think about when they're thinking about OKRs because unfortunately I see that becoming quite a bit mechanical exercise in a lot of places. And they're not really stitching that into the strategy. And just like how you pivot on the strategy, maybe you also pivot on your OKRs and things like that. I don't know what's your view on that. I just wanted to get curious. Do you iterate on the OKRs itself once you've set them? Yes. I mean, look, the OKRs are based on a series of assumptions, right? We're going to make our best guess about the world we'd like to create and the behaviors that we think are going to get us there. As the teams begin to work towards those goals that are going to learn what's true, what's not true. Is this the right objective? Are we chasing the right behaviors? Did we set the numbers correctly? And we want to pause and reflect that the OKR cycle that we advocate for in the book is a quarterly cycle because that's common. But whatever the cycle time is that makes sense for you, you want to pause and reflect in retrospect, right, if you're thinking about agile terms about what we've learned in the last quarter. Does the objective still make sense? Did the key results still make sense? Has something changed in the world to make us focus on something else? Did we crush all the key results and we need new ones? Are we never going to hit these key results? So absolutely, like we always say that like the longest, I mean, and this is in a generic setting, but the longest you ever have to live with your OKRs is a quarter. Because at the end of that quarter, you can begin to reflect and see whether or not you've actually, you're still chasing the right goal. So the quarter is what would be your time to kind of go back and reflect. Yes. Yeah. If you try to do it too early, maybe it's just too premature. Well, I mean, again, if, again, look, if your Facebook and you've got two billion users, you might learn faster than a quarter, right? Very few of us have that kind of user base to work with. We have some interesting questions. So I'm just going to quickly go through and play back the question. So the first question we have here is from Shantanu. He's asking if you if you're basically providing a service, if you're giving a service to a B2C company, does your don't your don't you think the customers of your customers? So if you're providing a service to a B2C company, then don't you think your customers, customers, that is the B2C company's customer, also your customers? Go ahead. Go ahead. All right. So here's the thing. Why do we have OKRs? We have OKRs so that we can work on the right things and focus on work that creates value. And so to do that, we want to be really, really precise about the work that we're doing and the effect that that work is having in the world. And so in a sense, you are right that if I'm in a B2C universe, my customer has a customer and ultimately we all want to deliver value to that end customer. On the other hand, I have very limited control over what my customer, my direct customer, does between me and the end customer. And so if I don't have direct control or influence, then writing OKRs that have to do with end customer behavior takes a measure of control out of the use of the tool. And so it's more effective, we've found, to focus our user. The example we always give is, well, I'm going to wait on the example. It's more effective to say, OK, look, when how do we help our customer succeed in serving their customer? That's something that we have direct control over. We put tools in front of our customer and we can control that interaction. Beyond that, there's so many factors, so many variables that it starts to become what we call an impact, which is important, but not something we have direct influence over. And so the example might be if you work at General Electric, you're a team of five people at General Electric. Well, General Electric's profits are important to you, but your team of five people don't control them. You contribute to them, but you don't have any control. You can do the best job in the world and General Electric could lose money that year. And so rather than measure the team on an impact, something that they don't have control over, measure them on what they have control over and let the team use this tool to set targets about things they can actually control. So just to understand what you just explained, Josh, what you're saying is if basically I'm a B2B company providing service to a customer who's a B2C company, then the impact, the benefit that the final customer, behavior change that the final customer is going to see is an impact my organization is creating, but impact is outside my direct sphere of influence. So I would not put that as an OKR, as an objective in my OKR framework. That's right. That's right. Exactly. Cool. Appreciate that. The next question shall we move ahead? There's a set of interesting questions I would like to get through. So I'm just moving to the next one. The next question is from Nilesh. He's just summarized that outcome, a measurable change in human behavior that delivers business result. And his question is over what lengths of time are we talking about this, right? Like, does that matter? Look, I hate to give you the answer that everybody hates, which is it depends, right? But it's true. The length of time will vary based on the business and the industry that you're in, the number of customers that you have, the kind of service that you deliver. I mean, if you look at two different examples, we'll use Facebook, right? Two and a half billion users or whatever they have these days, right? You're going to get feedback. You're going to be able to measure outcomes very, very quickly by the nature of Facebook service, by the quantity of customers that they have, by the access that you have to test with those customers or to put ideas into the market with those customers. So that could literally be minutes or hours, certainly with a Facebook type of service. Now, with an example like, you take a pharmaceutical company, right? Where the time it takes to, maybe that's not a great example, but perhaps like an enterprise software company, right? You have a lot fewer clients. You have a lot fewer users generally speaking and it takes time to get updates to those folks. So maybe it's going to take you a quarter to see those outcomes move in a meaningful way over time. And so I think that there's a variety of timelines here and it has to make sense, again, for your industry, for your service, for your customers in that world. I mean, generally speaking, I think for most companies, whatever your deployment cycles are, generally once you get something out, whether it's a prototype or a test or whatever, you should start to be able to see some feedback coming back from it. But I definitely think it just depends and it varies. Josh, did you have something? Yeah, I mean, I think there's a couple of ways to add to what Jeff said. The first is that in terms of OKRs, we split kind of the goal, which is that objective from the measure, which is the KR, which is the outcome. And so that goal statement, the objective contains a time. That's where we put the big time statement. So the best X by the end of the year, the greatest whatever in five years, whatever that is. So we set a kind of a big, gross timeline in the objective. So that's where you put it. The second thing is that in the book, we talk about OKRs being three things. So OKRs are a goal-setting framework. And Jeff took you through that. Here's the O, these are the KRs. That's the framework for setting goals. The second thing is that OKRs are a process. And so we don't just set goals and then go back to doing the same old stuff. We actually have to work towards our goals and setting goals in this way changes the way we work. And so it's a process. And we call that process the OKR cycle. And that process has time boxes in it. So when we're using the OKR cycle, we check in on our OKRs. We check in every week. We check in every month. We check in every quarter and we check in every year. And so we have built-in time boxes. Now, your organization may choose to adjust those time boxes based on some of the things that Jeff was talking about. What's the natural rhythm of your business? But that's the fundamental cycle, a week, a month, a quarter, a year. And so you look at your objective and you look at your results each week and each quarter, each month, each quarter. And you try to make adjustments quarterly and annually. But you measure your progress in these smaller units. And so you can choose to set key results that have time that are time-bound. Our goal for the year is 50%, right? And we think we're really only going to get started in the first quarter. So we'll say 5% first quarter, but we're really going to see some acceleration in the second quarter when this thing comes online. So 5%, 25%, 40%, 50%, whatever that is. So you have the opportunity to set those timelines through your process. I'll just say the last thing, which is not about time, we said in the book, OK, ours are framework, they are process, and they're also a culture. And so in the book, we talk about all three of those things. Awesome. Thank you. Let's quickly move to the next one here. There's a question from Girish. He's saying, I prefer and recommend absolute numbers over percentages wherever possible, which adds more clarity. What I've experienced as percentage without clearly communicating baseline creates varied understanding and confusion. So it's not really a question, but I'll turn it into a question. Yes. Can you agree? I totally, I forgot to say this when I said it. So thank you for calling it out, Girish. Look, yes, percentages are meaningless without a baseline, right? You can't 25% more of what, right? Based on what? And so you need a baseline for those. If you don't have a baseline, then we start with absolute numbers. But over time, the absolute numbers start to have less and less meaning, right? So for example, if you go to a team and they'll say, we acquired four new customers last quarter, OK? Is that good? Well, if you had four customers before, that's 100% increase in customers. Well done. If you had 250,000 customers before, then you added four. That's less significant, right? And so I think initially, if you have zero customers, then adding four is tremendous and therefore absolute numbers make sense. But once you've got a baseline in there, starting to use the ratios or the rates to give you a sense of how you're trending gives you a better sense of whether or not you're moving in the right direction and how quickly you're moving in that direction and gives you a better sense of, like, is that acceptable? Like, are we making enough progress? Are we making enough progress fast enough or do we need to change what we're doing or shift to a different metric? So I guess percentages do make sense in some contexts, especially where you may not really have an absolute baseline, right? There are examples where you may not be able to establish an absolute baseline. Would you agree? The percentages, without a baseline, they don't make sense, right? So if you don't have a baseline, then we're going to start with absolute numbers. But as soon as you start to measure absolute numbers, that's your baseline, right? So over the last, and again, the cycle time is going to vary based on your industry. So if quarters make sense, you can be like, we acquired three customers in Q1 and we acquired five customers in Q2, okay? So now we have eight customers and now we've got a sense of a baseline of how we can manage growth from there, set growth expectations for Q3 and Q4 or maybe you measure it for a year and then you say, okay, annually we expect to see these growth metrics. But at some point, you're going to have enough absolute numbers to define a baseline and build ratios from there. Correct. I mean, so you'd have to wait for that baseline to be established. If you'd try to do that too early, it may be misleading. Correct. Perfect. Cool. Let's move ahead. I think the next question is from Gerald. He's asking, how do you deal with CTPO, Chief Technical Product Officer, on having separate OKRs for business and the products slash tech department? Fire them. Is that the right answer? No, it's a great idea. Go ahead. Give them a better answer than that. So look, one of the things that, so there's this kind of imperfect phrase that we use and we know it's imperfect, we like it. It's pretty good. And here's the phrase, everybody has a customer. Everybody has a customer. And so you can think of your colleagues as your customers. OK. And so my coworker in the next cubicle. What is that person doing with the work that I create? Are they able to use my work effectively? Are they able to get value from my work? And so if you think about that, who does what by how much? OK. Who's the who? Could be that person sitting in the next cubicle. And if I'm doing a good job, what value do they get? And how can I tell that they're getting value from my work? So it's possible with any goal setting framework to abuse it. And so a CTPO can make meaningless measures for the tech department, the number of story points that we completed this quarter. That's an absolutely meaningless measure. Who cares? Or we can say, we've reduced our deployment time from, you know, X to Y. And so our customer, right? What's the value? Well, the customer has, the most recent is able, is you, customer uses the most recent software every day. On a daily basis, a customer is using the latest build. And we believe that that's valuable, right? And so I think it's possible if you think about it really in terms of customer value, it's fine, right? But you do have to think about it in terms of customer value. And so again, it's that same problem that Jeff was talking about earlier, which is we want to make sure that our key results are measuring the value that we're delivering to a human. And it's absolutely valid to have internal humans in the focus of that. I want to add another thing too here. I think this is an anti-pattern that we've been fighting against for the entirety of our careers, right? This separation of the business and the product and tech departments. And you've heard Jeff Patton talk about this, right? Like, are you a service provider to the business or are you a part of the business? Like, are you a waiter, right? Or a doctor, is the Jeff Patton spectrum of things? But this idea that we create these cost centers inside the company that then service the business is an anti-pattern when it comes to modern product development, modern business building, right? All of us are part of the business. And if we can build these unified organizations that don't divide up, well, product and tech, well, they make the software, right? And the business makes all the decisions and tells product and tech what to do. We start to actually build the kind of cross-functional collaboration that makes organizations successful, that drives the kind of agility that we've always been looking for in these companies. And that ultimately services our end consumers, our end customers more effectively. As soon as you start to set goals for the business and product and tech, you're separating the targets for those folks. We're disaligning the organization and we're pointing them in, I don't know if they're opposite directions, just different directions. Why wouldn't we have everybody pulling in the same direction? Like it doesn't make sense to me. And we see this a lot, you know? It's something that we've been talking about seems like forever. And I'm sad that it's still a thing. I think that ties nicely into the next question that Ganesh has where Ganesh is asking a typical situation. The IT services company are ghost controlled by their immediate customer who just want their outsourced task to be done and delivered. But at the same time, they expect value, that is outcomes to be delivered. How does one resolve this paradox? I'm going to say something. I'd love to hear Josh's take on this, but I mean, I think the answer is in the question, right? IT service companies, right? Service company. I think the answer is there, right? I think in a lot of those situations, having run an agency with Josh, a services company, it depends on how you sell those services and what promises you make to your clients, right? I mean, if the agreement that, if the way that you pitch your services and the agreement that you make with your client is, we will make whatever you ask us to, that's the business that you're in. You're a service provider. If the agreement is, if the pitch, the contract, the sales cycle talks about, we will be your partner in delivering a solution that drives these outcomes, that's a different engagement. And so I think the answer is in the question, is that if your company is positioned as an IT service company, your service provider to these client companies to do what they need. One of the challenges that we faced when, so Jeff and I ran an agency together. We were partners in a company called NEO a decade ago. And one of the things that we had to deal with was the way we wrote our contracts. And there's sort of three ways that we saw that we could write contracts, right? The first way was we'll write a contract based around deliverables, right? So we specify everything up front. We build what you tell us to. You agree that we've built it and then you pay us, right? That's a deliverable space contract. You're selling output. You can't, you're not selling value. You're selling output. So that's like, that's the fundamental agreement that you're making with your client, okay? The second way you can do it is time and money, time and materials. And so you say, we've got a team for you. This team costs X per week. We're going to work with you and you're going to pay us, right? It's a really, really flexible arrangement. It's a little risky for the client. The client doesn't like it because they're not sure what they're going to get, but it's flexible and it gives you the opportunity to really collaborate and decide together what you're going to build and what you're not going to build. That was the model that we chose. The third model is to build around value, to say actually we're going to take responsibility for the outcome. And that is a hugely risky proposition for the consultant company. And even though we believe in outcomes, we were not willing to take that risk because our clients would come to us with things that we could not control. Maybe they came with bad ideas. Maybe they came with, you know, we could do a great job, but they could still distribute whatever we did poorly. So we didn't have enough control to say, we're going to sign up for the outcome. So the fundamental contractual relationship that lies underneath the relationship is going to drive what you're able to do. And so where possible, right, the most enabling structure for that kind of thing is to take projects on a time and materials basis and then to demonstrate that you can really deliver value within that arrangement. I mean, just to kind of tie this back in, right, like still there are a lot of ID consulting or ID services company in India whose value to their customer is essentially being able to deliver whatever they're asking for in the most cost-effective manner. And to me, that's value, right, purely from that perspective. And as long as I can justify that I'm able to deliver what you've asked for, whether it's the right thing, whether that's what we should be delivering, is that really going to help you achieve the outcome? At this point, those things are outside the bound of the contract. What we are agreeing is that, you know, the customer has a good insight of what they want and they want someone to deliver that in the most cost-effective manner and something that's maintainable, something that's easy to change. So we could put some of those value parameters on it and say that's value and that's the outcome if you're able to do that. And you draw the book over there and you say, hey, that's what we're agreeing to. And I think that goes back to Jeff's earlier point. You're making a point that I agree with and I think there are times when it makes a lot of sense to say like, okay, look, we're going to manage this project based on the output, right? And it makes sense when you have high certainty that, you know, we've specified this thing and this is the thing we want and this is the thing we need and we believe when we get it, we'll have created value. And if those are the terms of the arrangement, that's great. The risk there is that sometimes we are certain and wrong and it's hard to know in advance. And so, yeah, there are absolutely times where that's the right way to do it, but... That's the inherent risk someone is willing to take when they're signing up for a project like this, right? I mean, when they're trying to get someone to sign up for a project like this. So I think it's okay. I mean, I don't want people to feel like, you know, they should absolutely, it's a sin to do that because there are business contexts where it makes sense and that's the context in which it's set. So it's okay. I mean, but if you're working in a product organization and behaving this way, then maybe it's something you want to rethink. I think that's right. And I think the art here is to be able to assess the risk and to be able to select the appropriate management approach. Perfect. So with that, I think I certainly want to thank you, but we have a couple of folks who had signed up for a lightning talk. It's a just three-minute quick lightning talk. It would be great if you can give your feedback as well for the speakers who are going to do a lightning talk, not a formal, formal feedback, but anything you find interesting or anything you would want to pass along. So if you don't mind, we'll just quickly go through those lightning talks. Okay. As Jeff and Josh, so those senior people are there and you are there. So it is difficult to speak in front of them, but still I'm trying to share something. Go for it. I'm going to set your timer for three minutes. Okay. So dear friends. So today I'm going to share some of the things about the OCR history. This I'm going to omit because Josh and Jeff has covered it in a very nice way. Then we'll speak. I will share something on how to connect our day-to-day work to the OCR, to the strategy using OCR. This is what we were just speaking about. Then the last point, I will share how OCRs are going to make scrum lean. Okay. So connecting our day-to-day work to the OCR, generally at the organization level, we have the vision, then aligning to that one, we have the strategy. Then the organization is coming up with the different objectives and the key results. These objectives and the key results are defined at the organization level by the senior leadership. According to that, there are departmental heads who are coming up with their yearly OCRs. Okay. Then the team is coming up with their quarterly OCRs. Then again, at the lower level, team is forming up their sprint and the monthly OCRs. Okay. So here the beauty is that team is deciding what key result they should be writing. They are trying to achieve so that they are going to achieve those upward objectives. And as Jeff was telling, if something is going wrong, then they can abundant at any point of time. Okay. So at this level, the team is forming their sprints, looking at those key results. Okay. These sprints might have certain development work, certain stories, certain assumptions to test, and the discovery work, and all those things. Suppose team has completed one sprint, then they are going to check back whether the respective key result is achieved or not. And it is adding value. They are helping to move in the right direction or not. Okay. Then it will happen again at the higher level and something is going wrong. Then they are going to stop and then they are going to come up with the new objectives and the key result. Okay. So proceeding further, let's see how OCRs are making Scrum lean. Here the team's goal is not to complete certain tasks, certain user stories, but team's goal is to achieve certain key results. Okay. And the team is building up the product backlog over here. This is very much important. The reason being is in the normal world, when there are no OCRs. 30 second check, last 30 seconds. So the team is building the complete product backlog. Okay. And that is helping them to create the outcome mindset and with this, when they are creating their own product backlog, they are spending less time on the refinement. They are spending less time on the spend planning than the daily stand-up. And there are no surprises at the end because sometimes I have observed in my real experience, the Scrums are mini waterfall. The Scrums have become mini waterfall. And when the team is owning the product backlog in terms of the achieving the product backlog, in terms of the achieving those key results, all those time which is being spent in the ceremonies, that will get reduced. That's it. Thank you very much. And happy to receive any feedback. Thanks, Abdul. I appreciate you finishing on time. I know it's very condensed. And it takes a lot of effort to condense your thoughts into a three-minute short presentation. And it's great. Jeff, Josh, you want to say anything? I think that combining, look, OKRs and Scrum make a ton of sense. The Scrums Sprints are perfect for building in the kind of discovery activities that help us test whether the ideas that we have in our Sprint are actually moving the key results in the right direction. And it helps us prioritize our Sprint planning. The work that we're doing right now is moving the KR forward or it is not. And if it's not, we've got to work on something else. And so you're changing course based on evidence. And if you're changing course based on evidence, you're being more agile. So I agree with that 100% nicely done, Abdul. I just want to thank everyone for joining today. I greatly appreciate that. We are hoping to have more such webinars this year so that we can get together and discuss. So again, it was a great pleasure. So thanks again, Jeff and Josh for your time this morning. We're really, really interesting insights around OKRs and how to make them outcome centric and look forward to your book. And best of luck with the book launch. Thank you so much for having us this morning. It's a real pleasure to be able to talk about the book with you all.