 Hi, awesome. I'm really excited to join you all today and thanks very much to the Product School for giving me this opportunity to come to talk to you, so let's get stuck into it. So I'm Andrew Howe, my mantra, I like a good mantra, I don't know if you guys know Guy Kawasaki, he's one of my favourite people out in the world, but he says everyone should have a mantra, so a number of years back I said what is it that really drives me when I work and that mantra was that products empower in life, so I really like and definitely now will only really look at working for companies where I feel like the product that I'm trying to generate is going to have an impact on people's lives in a positive way and so I think it's really important for me to be working at a company like that and at the moment I work at bookie.com, so I really get that opportunity there. Bookie is a great place to product and it's definitely got that product outcome thinking going on, which is one of the reasons that I've been there for a while now and I can't see myself leaving for a little while, but yeah everything's there is the right way to do things. So just skipping forward now, so again background would be I've worked as startups in the financial services industry and not for profits and obviously now in the global travel industry, not all of that there's been an important role sometime that you know I've started, I've done business analysis earlier in my career and I did commercial strategy and I made the shift over to product roughly about six or seven years ago maybe a little bit longer and it just was a natural progression from all of the different places and different skills I pulled together for me to take a step into product. So what we have today, we want to talk about outcome thinking and specifically we're going to have a chat about product outcome thinking. It's a bit of a large subject, there's a lot out there, if you go out into Google and out onto the wider internet and you start doing a bit of Googling, you'll find that there's a lot of information about product thinking and people have very different views or lenses that they put on there. You might see people talk about technology thinking and problem thinking and outcome thinking. Some people might talk about outputs and outcomes and we're going to talk a little bit about that today. But really those are all things that make up outcome thinking. They are a different type of thinking, they're all part of the same thing and I think in my personal view, it's not the right thing to say you shouldn't think about output. Output is an important part of outcomes. You have to understand the output process and what it's doing but really where we want to do is product managers is in that output outcome mindset. So what does that actually mean? Well, we'll have a bit of a chat about that now. So just to define quickly what an output is because that's most of the thing that people are saying, we should be doing outcomes and that output. So output is the amount of something that you can generate or machine can generate or an industry can generate and it's the volume of things. And that's important because if we want to build something or create something, we have to know how well we can do that and what's involved to do that, right? So we shouldn't completely disregard output as a fundamental part of outcome thinking. So you can be super productive from an output perspective and you can be super efficient but the question is, are you being effective? And this is where we start to get into the round of outcomes, right? So effectiveness is the outcome, right? Productivity, efficiency of the measures around the output. So here's a really great example which I really like about, you know, if you just outputting yourself to the place that you want to be that outcome, have you built it the most effective way, right? So for example, do I really need to build a bread machine within a bread machine to make a loaf of bread? It's a classic Simpson's example of a problem. So what is an outcome then? Let's flip it now, let's talk about outcomes. So outcomes are basically the consequence of something, right? So it's the way that the thing turned out. It's not the process in getting there. And this is where we as product owners or product managers or product leads, whatever, you know, you guys are calling yourselves at the moment. This is where we want to spend our time and I think a lot of our thinking, right? And everything that we should be doing should be about facilitating an outcome, driving the, you know, everything towards that, helping the team, helping each other to make that outcome a reality. So how do we go about shaping an outcome and getting to a place where we can understand that outcome? Well, first off, you've really got to make sure that everyone understands the problem, right? And what we really mean by this is that you guys as a team have spent sufficient time in the problem space. And what you've done there is you've validated that there's a desirability to solve that problem with customers. Now, you can, once you've got that, that understanding, you know, you can do really quick exercises here around things like just jumping up on a jamboard, getting everything that you know about that problem just shoved into one space, and then theming it really quickly so that you can get an understanding of the different subcategories of problems that exist within that. And a really good example of a problem here in the digital realm would be something like that you're losing customers at your checkout page in your e-commerce journey, right? Customers are going through all the steps, they're getting to checkout, and then you'll drop in a lot. And then when you go out into the wider market and you're looking for benchmarks on what conversion looks like, you seem to be 10% down on everybody else, or even worse, right? So you have a problem now, you have a problem, and then you can go out to customers, you can do some discovery around it, and you can say, hey, what is the problem? Try and contextualize that problem now a little bit more with your customers. So we've now gone through that, and we haven't just done that as a single person exercise, we've done that as a team, we've understood the problem is here, and that's really important, because if you're the only person that understands the problem, how are you going to be supposed to generate ideas or ways of solving that problem that's going to generate the outcome that you as a product manager want to create, right? So this isn't about being an individual, product is not, you know, a one person sport, it is a team sport, it's an activity that is team oriented, and you have to have that in your mindset. Really, your job as a product manager is there to facilitate that everyone else's success. So so yeah, so we've understood that problem, everyone's got their head around that that part of the things. So then the question is that ideation phase, how might we solve that problem, right? And this is where we're going to spend a bit of time with with the team. And again, we're going for another exercise of ideation, the workshop, and we're going to start pulling out ideas. Now we don't just want to pull out ideas and say, oh, we're going to do that, because then we're getting into the realms of output, right? So you can have a list of 100 things, and you go, right, I'm going to go make all those 100 things, and that's going to get me my 10 percent back, right? But actually, what we're doing there is being we're not being effective. We're probably not being very efficient. We're probably being very productive, because we got 100 things that we can go and build, and build them, but we're not probably doing that in the right way. And what we mean by that is that 100 things you may only need to have actually done 10 of those to get to where you want to get to. Or it may be only 10 of those are actually the things you need to care about that are going to get you where you want to get to. So this is where we now have to start thinking about putting some rigor around those things. And this is where things like hypotheses can really help us to understand that. So first off, so we've got that how might we, the ideas, and we want to make sure that we can measure those things, right? So have we got a measure that sits around those that gives us that ability to say, Hey, something is this measurable? So I've got an idea. Is it measurable? Can I go into the thing, make that change and see that movement? Because what we're trying to generate here product outcomes of behavioral changes. Okay, they're not. We're trying to change the behavior of a customer to do something more in line with where we want them to go. So in this instance, as the example we're talking about, we want them to convert more. We want them to check out more, right? And so to do that, we need to make sure that we can make a measure between the current experience and the things that we're changing. And we may use things like AB experimentation, but we need to make sure that we have either micro conversion events or other events available data points that allow us to be able to measure that successfully. So we have a measure in place, and we're confident that we can make that measure. So the next step then is, okay, we've got that hypothesis now. So when we talk about hypotheses that I always talk about Teresa Torres, she's really great at defining all of this sort of problem space and also the structure of hypotheses. But she uses this very, and I love this example and it's something we use very similar here, booking. But yeah, I know if I design this, which is the change, this will increase the conversion. So that's the impact for the who here. So online grocery shoppers and by how much by 10%. And also there's a time box on that there as well, because we're saying this is going to happen in seven days. And this comes back to, you know, making sure that we're going to do diligence on our data analysis to say that we have the volume to be able to return that experiment within this window of time. Right. But again, just to flip this, this is obviously digitally orientated. You can use this same method for manufacturing and things like this, because you still want to create an impact. Right. So you're going to change something to create that impact. So it may be that you're doing something that you want to change the color of a t-shirt. Right. And that's what you think you've been increasing the number of sales of that t-shirt for 18 to 30 year olds who shop on your website by 5%. And it's going to happen over the next 30 days. Right. You can use hypotheses in lots of different ways. And so we've got a hypothesis and we've got a set of hypotheses. So we might have more than once. We've got a load of ideas now that allow us to, that we think other things that are going to solve that problem. We put some value around them and we prioritize them. So out of the 100 things, actually there is only 10 that we care about because if we get those 10, we're going to over deliver on what our outcome is, which is we want to create 10% impact to the checkout on our e-commerce site. So now we've got that and stripped out the waste. So we've reduced our output down to what we believe is the thing that's going to create the most effectiveness. And we prioritize that in our backlog and now we're going to go into the realms of going and delivering that. Right. But before we do that, we want to explain that and set us a focus around that. So we want to make sure that the wider business understands it, other teams and also that we as a team have something that gives us the ability to understand that easily. There's a great framework for that and it's basically the OKR framework. And we use this at booking.com and I've used it in other places as well. So we set an objective. We want to make that inspiring. So, you know, we want to make an amazing checkout experience that really invigorates our customers to convert, additional customers convert checkout. We've got a measure that we can measure our key results against, which is the checkout conversion. And instead of just saying we're going to hit, we want to increase this by 10%, which is a very specific number. And what OKR allows you to do is give yourself a bit of range. And especially if you've got more ideas than maybe you might need, you've got a couple of extra ideas to make sure that if something fails, you've got something you can fall back on, right? So for example, you have a 0.3, a 0.7, and a 0.1, right? So when you look at those 0.3, that 0.7, and that 1, this is basically a range of outcome. And what we would say something like is, hey, our 0.3 might be 8%. We're going to get 8% towards our desired outcome of 10 in this next window of delivery. And the 7 may be the 10%, and the 1.0 may be 12%. But the reason why we have that range is because the 0.3 is something that we're really confident on. And that 8% might be a lot easier to get to because the things, there are some things that are just broken in the site or things that we can fix. We know that it's going to improve conversion, right? But actually you make it, so we're really confident on that one, but making then the leap from the 0.3 to the 0.7, there's a little bit less confidence there. We're still in the realms of possibility, but we're not as confident as we were at the 0.3, but we're not so unconfident. You know, it's not completely stretching us that it's not within the realms of us achieving, and we want to aim for that 0.7. That's what we're aiming for, in our OKRs, right? That's what we want to try to achieve a success. And then our 1.0 is, is that, hey, if everything went amazing and we absolutely smashed it and things just happened, then we think that we could potentially achieve this additional 2% on top of that. So what that gives us then as a product team, and especially around our outcome focus, is a range of success, and that means that it's going to be more manageable for us to hit some success and learn from that. Now, if we fail to hit that, that's not the end of the world, right? Failure isn't a final step, it's just an opportunity to learn. So what we don't want to do is fail again through the same mechanism and same mistake. We want to learn from that and improve so the next time we do hit our OKR. So OKR is a really good tool because it allows us to also say, hey, if we under forecast what we can achieve or if we overstretched ourselves the next time we just need to, need to learn from that and ping it back a little bit and it helps keep us focused and also helps make sure that we're not over pushing ourselves out of the realms of what is actually possible within those cycles. So yeah, OKR, really great framework, but then you're in the doing, right? You set your OKR and now you're in the quarter and your team's running and you know you're getting into it. So where are you now? Well, you're still that facilitator and leading up to that place, you should have been doing your planning and identified lots of things like your dependencies, tried to remove as many of those things as a team as you possibly can, but your realm as a product manager in this space is to continue to make sure that the things get facilitated and also the challenge here, I think especially when you get PMs that are or POs that are coming new into the space, there's a tendency to get stuck into the detail and detail is important, but your role here at this stage is to take a step back from that and to be able to make sure that you're continuing that job of doing that outcome facilitator role, right? So, you know, is there a thing? Have you got everything you need? How's the experimentation going? Is there any data problems? Have we, if we underestimating, do we need another skill set we don't have when right, I'll go and get those things so that the team can continue to move towards the outcome success that we want to drive, right? And again, reminding ourselves here that we're trying to drive a customer behavioural change, you know, are we measuring those things effectively? So, sometimes the thing that we need to do in that instance is make sure that we're giving ourselves time to take that condor moment and what's the condor moment? Well, a condor moment is basically when you take a step back, you know, making sure that you take a step back and you're looking at the space holistically and you're going, where are we? What's the confidence like? Are we going, have we spent too much time on this hypothesis? Is it not working? Shall we switch over to the next one in the list or our next one in our priorities? Because if we spend any more time in this, then we're potentially going down a rabbit hole. It's making sure that you're facilitating those conversations to the team as well as making sure that anything that could be blocking the team you're working on because if you get to the end of the quarter and the difference between you hitting your 0.3 in your OKR and hitting your 0.7 in your OKR was that the team got blocked for a week and a you weren't aware of that because you were in the detail of something else or that blocker where you didn't get raised quickly enough because you weren't able to manage that amount of information or whatever the challenge was being. Then there's an element of like we've all failed, right? But we can learn from that because next time you need, the other thing is let's make sure that that doesn't happen. So that's about you taking a step back, make sure that you're having that opportunity to make sure that everything's where it should be. But also that then you're working with the team to make sure that your feedback loops are faster and that you're getting that information that you can do and rather than that the teams get stuck on it that you're able then to go and facilitate a way out of that problem, right? So yeah, so making sure you have those times to take a step back is really important. And also coming back to this planning thing. So planning, people don't put enough time into planning. Plans are only as good as the moment they were written, but planning is a really important tool. And it's important because let's say we go back to that list, we have that list of, you know, five things that we're going to do or ten things we're going to do. Well, what happens if none of those things return, right? But we're early enough in the quarter to do something about it. Or we're going down the track and it doesn't quite work out. Well, this is where our planning comes into effect because our plan isn't going right, but our planning has gone into enough detail to say there are other opportunities here and we can pull another player or a different set of tactics in to be able to try and Oh, sorry. Yeah, so sorry, just a bit of a glitch. And so planning, yeah, so planning is really important, again. So like I said, so if you've got that space and you have all of those potential other tactics that you can come into your disposal around that problem, it will potentially give you that opportunity to still achieve that outcome or part of that outcome without completely failing. So planning is a really important step and you should, you know, one of the things I always say is that we as a team, we always put time into planning and we start that early enough so that it's just a part of the process and it can be managed well and efficiently. And so, yeah, so it does lead to outcome success because those times where you get to a place where you can't go down the route that you originally planned to, it gives you that flexibility to move less than right around those issues to still facilitate back through into that outcome that you want to generate. So the idea here is about the flow of things rather than being so rigid that things just don't happen. And again, what we tend to find or you may find is the things like methodology and things like that can get in the way. They're really important but sometimes we need to be more flexible than that. And as a team working now, sitting down about how you want things to play out and what are your, you know, four backs is a really mature product type of thinking that I think is not always prevalent in all teams. So, yeah, so planning definitely equals outcome success. So just quickly as well, I wanted to quickly lastly talk about this. So again, one thing that can really make a difference between, you know, the type of impact outcomes are generating and the impact that they can generate is the type of thinking that you have. So if you're in a very predictable space, so you're thinking very linearly, you're doing optimization, you're taking it from one optimization to the next, then you're always moving forwards but you're moving forwards that are very predictable and sometimes a slow pace, right? So you might be making very small, minor improvements to the overall thing that you want to try impact and where sometimes if you think more exponentially, what you can do is you can skip some of that stuff. So rather than taking that nice, gentle curve, you're gonna, you're gonna make that curve go up more, more, more erratically, more, sorry, more erratically, more quickly because you're gonna take some risks, right? So if you can get into a place where you're premature around your process and how you're working as a team and the practice that you have and, you know, thinking about how you can be more exponential in your thinking can be a real benefit as well in product thinking because what you're trying to say is okay, this is the sort of customer behavioral problem I have at the moment. These are some of the things I could fix it with right now or potentially improve it with but hey, where did we think customers may be in x time from now and is there an opportunity to jump to that place and pull it forwards and in doing so you potentially have created a bit of a USP or a new curve that puts you in front of your competition and also, you know, makes your customers have a better experience and maybe think that you're a bit more cutting edge around the other things that are in the market. So another thing that we tend to use quite a lot in our teams that I work with is is there an exponential way that we can solve this problem that's gonna get us not just the outcome we want but maybe push us past that as well to drive towards more of the vision or the wider goals that we're trying to achieve. So just something I way to quickly talk to you guys about. And again, a really good book that's worth reading if you haven't read it already I know that probably a lot of you have is Radical Focus. Brilliant book, great, easy read. Really easy to understand how OKRs can be working, how they can be intimately. You can get through this really quickly, really practical ways as well. I think even in the back it's started orientated but in the back it has a lot of product stuff as well. So great, great book to read. And again, I think I mentioned Theresa Torres earlier who's also someone I think has got a real load of good stuff on product thinking and how to think about problem spaces. Yeah, so that's it. That covers everything. I hope this has been a really useful session. There's a lot out there. I'm not a guru on it. This is just the thoughts and experiences of things that I've done and things that work for me. I'm sure that you guys have got a load of things that work for you as well. So don't be afraid to share them with the world. Anyway, brilliant. Thank you very much.