 Hello and welcome to another episode of The Agile Pubcast, a special bonus episode of The Agile Pubcast which is completely free for everybody. This time Paul and I got together with an old friend of the pubcast and a massive supporter of the Agile community in general, Vasco Duat. So Vasco is the host of the Scrum Master Toolbox podcast, if you haven't heard that, check it out, it's been going on for over seven years now, it's got hundreds of episodes, they're all free, and he's also the organizer of a completely free three-day conference that happens every year called the Agile Online Summit. So check that out, like I said, all free, great guy. Today we picked up a question that was asked in my Agile Mastery community around an interview question that a lot of people seem to be getting asked these days, which is how do you measure your team's performance? So somehow an hour flew by while we were discussing the intricacies and the nuances of this. We hope you enjoyed it, settle in, have a drink, here's the jingle. Here we are. I don't suppose you've got your Agile Pubcast glass, Paul, wherever you are, I don't know where you are. No, but I have to give you this merch, Vasco, see this? That's great merch, I should get one of those. Yeah, they're quite hard to send in the post. I imagine you have already a couple of tricks on that. So yeah, welcome everybody back from the summer break. I think in England it went from summer to winter in the space of 24 hours, we had the air conditioning on one day and the heating on the next and that was it. Summer gone. I don't know, it's like in Helsinki for you, Vasco, but still warm? Yeah, we still have a couple of warm days, like today was still very warm, so we're not complaining, let's say it's quite rare to have a warm September here in Helsinki, but we do have it now and we are counting our lucky, what is the expression? Yeah, counting your lucky stars, yeah. Counting your lucky stars, yeah, absolutely. Cool, so have you got a drink? I have a drink, I have La Frog, a single malt scotch whiskey. Wowzers, very, very petty that one, isn't it? Yeah, it's very petty, I love it and also because I wanted to get a British drink, I didn't have anything else that was British here, so. Nice, how about you Paul? Well, I ran to see what I could find from the, I'm in a hotel bar and they've actually given me, it's, so I'm in Bristol in the West country, so they go, they went with something safe, I went with Thatcher's, which is pretty much in every bar and Thatcher's Rousey, but they've given me, you say about your, your glass chair, look at this, this is a lovely Thatcher's glass, look at that. Oh yeah. So it's got a special kind of cider glass to it, I should enjoy this. That looks like something that would fit quite nicely into your bag. Well, you never know, don't it? Maybe, maybe. Don't tell anyone. Well, I'm drinking something very strange. It's called Bundobust. I'll show you the can, look at that. So many questions, so many questions. It's got, it's got frogs and a sort of alligator-y crocodile-type thing. It's a collaboration between my local brewery, Dayer, and who else is it a collaboration with? Vic. I don't know, Vic's secret, that's the hops. I'm sure it was a collaboration. I can't see now. They have pretty interesting artwork on the cover. Yeah, loads of it. So these, it's, it's, it's a sort of cloudy. Can't see through that, can you? It's light, but cloudy. Lends in with your, your wood effect behind you, Geoff and Cod, can barely see it. Yes, yeah. Smells a bit sort of mango-y, maybe? Fruit-y. Oh my God, I hate fruit-y beers. But it doesn't taste fruit-y. Okay, that's good. I like the British beers especially because the, the are, some of them are quite bitter and I, I really like my beers to be bitter. Yeah. Now this, it's, it's, it's really quite pleasant. What's the, what's the percentage? What's the alcohol? I can't see it. Only five, it's five. It's a, it's a normal one, nothing particularly. But yeah, it's got a very slight, very slight tang, but very smooth. It doesn't taste as hazy as it looks. So when people tune in to my podcast, they tune in, for example, for book recommendations, I imagine that in your podcast, they tune in for the beer recommendations as well. Well, we did get pulled up on that once that we weren't giving enough of a description of, of the drinks that we were drinking. So yeah, we had to make a point really of explaining a little bit. And I'm not a particular connoisseur. And Paul's standard line, it's almost become a catchphrase. They'll have on a t-shirt one day, taste of apples. But yes, yeah, your podcast that people tend to tune in for much more serious, much more intellectual information. Yeah, more high brow. So yes, this, we've got, we've got a special guest with us today, Vasco Duarte, Agile coach and host of the scrum master toolbox podcast. And also organizer of the Agile online summit, which is a free, mark that free conference with some amazing keynote speakers and tracks, all dedicated to try and help all those scrum masters out there in the world. So welcome, thank you for joining us. It's a pleasure to be here. So yeah, you guys have been on my podcast and I was like, feeling a bit left out. So thank you for inviting me. Yeah, this is the, the away leg, the return journey. Indeed, indeed. Well, I mean, some of the stuff that that has been going on on my corner of the agile world, we've been working a lot on, on, you know, bringing new voices in the community to the, to the knowledge of everybody. I think that when we started the scrum master toolbox podcast, one of the chief motivators for us was to exactly depict agile as it is lived by the people who do it in practice. Right? Like, all three of us, we talk about it a lot. Right? It's our job. Right? But we also need to give voice to those that actually do it a lot. And some of them know, perhaps as much or even more than we do. And that's great. And others are just starting the their journey and their stories are also motivating and inspiring for the people out there who are potentially also starting their journey. So the scrum master toolbox podcast is, is what I would call a podcast by the practitioners for the practitioners. So we interview scrum masters every week, five questions every weekday. And every now and then we have bonus episodes on which at least Jeff has been, I don't recall if Paul has been in a bonus episode, but Jeff has been for sure. So who are some of the up and coming names that we should be looking out for? And what's some of your favorite episodes recently? Actually, at the week that we are recording, there's a guy also from the UK, Robbie Ross, who has this amazing phrase, which I love the moment I heard it, I said, Okay, that's my phrase from now on, I'm going to steal it, but also give credit to Robbie Ross, who's on the scrum master toolbox podcast. And the catch phrase is, instead of continuous improvement, he uses the phrase relentless improvement. And I just love that because it, it so depicts the ethos behind the idea of continuous improvement in a better way than the original phrase did. Right. So I really love that relentless improvement. And he did it with a smile and, and with a fun attitude and, and that, of course, he even adds more to it. I like it. That's the phrase I was using today, actually was, it sounds a little bit, a little bit marketing, but I was saying, but the scrum master talking to teaching scrum masters today, and we were talking about believing in better, which again, sounds like a bit of a supermarket catchphrase, but it's that it's not necessarily putting a number on it or a, or an endpoint on it. It's just generally having this belief that there must be a, there must be something we can improve on. Isn't that a sky advertising slogan? Is that believe, is that believing better? I think so. It's subliminal marketing at its best. It's entered your consciousness. Luckily, it works outside the UK without that connotation. Cool. So, so yeah, I want to be useful to get your opinion on this because I think we're we were planning on tackling something a bit, a bit tricky today. We've been asked a few, they weren't necessarily asked, they weren't necessarily asking me, but we were having a conversation about interviews because a lot of people have been changing jobs recently and are still changing jobs and companies still trying to hire people and sort of getting some pretty tricky questions, not necessarily because of what, well, I suppose tricky in different ways in that quite often the people asking the questions have a different interpretation of our job than the people who are answering the questions, which makes it quite tricky to give either a good answer or the answer that the interviewer wants to hear. But also, they're just quite tricky subjects. So it's useful that we've got someone else here to help us with this because they look quite tough. So the first one that people ask me was how would you measure your team's performance? That is awesome. So before we dive into that, I don't know if you guys already want to come in, but let me say that I've been asked that question many times as I imagine many of your listeners have had the opportunity to talk about evaluation of performance. Let me just say that one of my approaches to answering questions like that is not to give one answer, is to give many, many, many answers. In fact, that's the whole core of the Scrum Master Toolbox podcast. That's why we call the podcast the tool box, right? Like I got inspired by the story of my dad who's a carpenter and obviously like all carpenters, he has a huge toolbox and whatever problem you face, you pick a different set of tools, right? You're still the carpenter. It's still the same toolbox, but you just pick a different set of tools when you're there in that situation, seeing what's going on, right? So maybe that's something we could play around with as we answer these questions instead of trying to come up with one answer. Maybe we can try to come up with multiple answers and encourage our listeners to also say, hey, you could do it this way or you could do it that way or that way. It depends on the context because it does. And here's a few things to consider in that context. Yeah. Yeah. The phrase that I was taught for them was entrenched expertise. If you only realize you're an expert in one tool, then if that tool is a hammer, every problem you see tends to look like a nail. So having the ability to understand the context and pick the right tool for the job. I think there's also a big difference between what, depending on who you ask, what performance would be assessed as. I think there's a danger of team members. Well, there's a fear element in a tool as well. If you were to ask a team member, would they be honest enough to name something that they'd want to be assessed on? Or rather, would they name something they believe they can score well on? If not, there's a game playing behind this. So it's very hard, I think, to get to something absolute straight away. I think it's very much an experimental thing. But I've seen teams that have basically asked the team and then asked those stakeholders outside the team and tried to compare the two and tried to compare and contrast, is there crossover? Can we have a blend? Can we have some of one and some of the other? Yeah. But it's a potential minefield. Also, the context there. Not just potential, sorry to interrupt. It is a minefield. Very true. Yeah, for me then, so the first thing to pull out there into the context is how safe does that team feel about being measured? So the safer they are, perhaps the deeper the measurement and the less safe they are, the more superficial the measurement, perhaps. Plenty of first-hand experience of teams faking measurements because of the fear of consequences and nobody's winning there. Nobody's winning. So yeah, I'm tempted to say something that I'd like to think would sound clever, but probably won't, which is, for me, a team's performance is whether or not they are doing, they are the best team that they can be in that moment in time. Because there's always going to be competing forces on short-term, long-term, team bonding over focused, what Christopher Christie would call tunnel vision here, that focus on the delivery of the end line, the end of the sprint, that kind of thing. And depending on where that team is and where the organization is right now, optimum would mean different things. Yeah, I think, I was going to add something else then, there's, would you want, because I was always thinking about happiness and morale, right? So you can get an indication of morale and happiness, but you can have a very happy team that's not delivering anything. And you can have a miserable team who's really, you know, achieving basically propping a company up in terms of the delivery that they're making. So I don't, whilst I think morale is one of those things that's easy to measure, I think it is an indication. It can be a complex, a complex metric. I was more thinking about, it made me think about flow. And again, that kind of nice Chicks-Eckmehy's kind of diagram there, that kind of the different aspects of performance and ability against the task and against where we are, and almost plotting where team members think they are in terms of the task that they're asked to achieve and how close to that top right hand corner are we in terms of, because I think flow is a good, if we've got flow, we're achieving. And, you know, we're not necessarily, not directly linked to happiness, but I think it can be, it can be certainly linked to performance. So one other aspect that we need to take into account is that teams are in an organization, we assume, given that we're interviewing for an organization that has its own performance measure. So I would ask, okay, so how are we measuring the performance of the organization and how could we link the performance of the team towards the performance of the organization? Should we even look at metrics that are beyond the team and rather look at the system of team? So system is also a topic. Deming used to say that a bad system will take down a good person every time or something like that, which is, in my experience, a very, it's a frequent pattern that I've observed for sure. And also, one thing that is implicit in performance evaluation, which you guys already alluded to, is why are we doing it in the first place? Right? Are we doing it as a kernel of source of information for improvement? Or are we doing it as a way to measure people so that we can give them a bonus on a bell curve? Right? Because I've been asked this, right? And as an example, one of the teams that I worked with, they decided together that they would give each other the exact same evaluation so that every team member would get the same evaluation. But in some companies, that's not allowed, you have to rate people on a curve. And then the other aspect that comes from that, which is kind of pushed by fear and pushed by these metrics is the aspect of sustainability. We have a great keynote in the Agile online summit about sustainability, pardon the propping up of the summit here. But sustainability is an extremely important topic. I mean, in Scrum, we call it sustainable pace. But it's not just sustainable pace, it's also the level of happiness is part of sustainability in the team. The level of cooperation with other teams is an aspect of sustainability. The value delivery frequency and customer feedback is also an aspect of sustainability. And of course, the topic I like to talk a lot about, which is the death march rate, right? Like how much of our work is actually crunch time versus how much of our work is focusing on value delivery and quality rather than crunch time get, you know, all this stuff done by a particular deadline. So definitely sustainability would need to be a way to measure performance, if nothing else, as a balancing metric. I want to flag that because I really want to dig down into that one. But I can't let this other thing, I don't want to let this other thing go, which is around what you say about crunch. And so many organizations reward the wrong things. You know, I can think back to some teams that Paul and I have been on in the past years and years and years ago, where the organization would reward like monetarily reward and, you know, just publicly Lord and applaud the firefighters. And when you when you reward firefighters, you get arsonists because people know that's what's rewarded. So this idea of crunch time, you know, I'm looking for the teams that are pretty quiet really, you know, that are just going along quite nicely, which I think fits to your sustainability thing quite well. But this, I'm hoping that sustainability is a word that is going to continue in people's consciousness after what was called COP26, this idea from from an ecological sustainability point of view. But if you take that at a generic level, it's basically can we can we not can we avoid overusing resources, whether they be the natural resources of the planet, whether they be the resources of an organization or a team or an individual. So can we make sure that we can keep going indefinitely, not just with our energy, but with the intangible thing, the other intangible things that you're saying there about, well, are we actually putting too much asking too much of other teams or the processes or happiness and the political debit, I would say, rather than, you know, political goodwill. All of those things I think factor into is the team not just performing as a team, but as a corporate, I was just going to say citizen, obviously a team, isn't a citizen, but the metaphorical unit of delivery, they are just part of the system. I think that's also an industry specific thing as well as the idea of the crunch. And the one I was thinking about is the computer games industry, which I know historically has a when you approach a launch, it's just all hands to the pump and it's and to a degree, I'm not going to quote on this, but there's an element of we don't care how many developers this burns through, we've just got to get it done, we've just got to get this thing over the line. And it's similar in other industries as well, like the visual effects industry that time whole thing of burning through companies just because we need to hit a deadline and not necessarily caring about the long term impacts. I think I think churn and I think turnover of team members is also a good indication of performance in my view. Another tool to measure like that. I think that this conversation already highlights the importance of looking at the performance evaluation as a lot more than who's the best on a team or what is the best team on a group of teams. But it reflects the importance of us thinking that actually we are here for the survival and hopefully also for the thriving of a business that serves customers for a purpose. And evaluating one team definitely has an impact on how we serve customers. But it's not the only thing that has that impact. So the aspect of how do we link these two together, the fact that we do want to reward teams that make a great effort versus actually it doesn't matter what effort they make because if the company doesn't survive, it's still a mute point. Like there's no rewards for anyone. So one aspect that I've played with and actually suggested that in some of the companies where I've worked is to think about monetary reward as pure profit sharing. If the company does well, everybody does well and you can do it as a percentage of salary because salaries are not equal. That's fine. We can argue whether they should be or not. But that's not my point. So in terms of monetary value, you would solve that problem. You wouldn't need performance evaluation for the monetary rewards, which is very important. And then kind of pipe the performance evaluation towards an improvement cycle, which could be, for example, surfacing systemic problems. Like in that context, it makes sense to measure performance. If we look at performance outcome as a consequence of how we set up the system, that information is useful and is useful exactly to those that are asking for it, which is management. Management's main responsibility is to manage the system. It's not the people, it's the system. It's how people are treated. It is what kind of boundaries and policies we put in place. That's management's responsibility. And if we do that, then yes, then performance evaluation makes sense. And we could use a number of different metrics for that, like the flow metrics that Paul already alluded to. Oh, something else. Something good. It was good. We can edit out the pause, Jeff, if you need time to think. Do you guys talk about damning a lot here, or you don't usually? Not massively, no. Like I say, we're not as hybrid as you. We're not as intelligent. He's a curmudgeon. I think his avatar would be this kind of old man saying, get off my loan, you young people. But he does have some very, very important insights. You're quoting damning. And this again just gives you an idea of how different I think here, Pascua, I'm not going to speak with Jeff. Jeff's a lot more intelligent than me. But I was thinking about champion manager. So that might mean something to Jeff, and I'm not sure if it resonates with you. But I was thinking in champion manager, which was our kind of football manager simulation game that Jeff and I pretty much grew up in in the 90s. For a player, you'd have, you wouldn't want 20 attributes, Jeff, something like that. You'd be given all sorts of stats and all sorts of trends and patterns and awards and bonuses that they were given. And yet instinctively, if you got good at the game, you'd be for a particular play, you'd look for a certain number of attributes. Everyone would be assessed on a number of attributes, but for certain players in certain positions, you'd be looking for certain things. And I'm almost thinking now, is there something you could do for kind of team profile type of thing that there'll be a set of 20 odd attributes that we all have, but we can pick five that we could kind of specialize in. And that's what we choose that we'd like to be assessed by. Well, you know, I encourage teams to define their own definition of what kind of team they want to be. It fits in nicely with my view of coaching. The team holds the agenda in this case. I assume all things being equal, that a team, when given the choice, would rather be better tomorrow than they are today. Yeah. So what do they want to get better at? And as long as they're getting better, the organization is getting better, all things being equal, not all the time, but most of the time. So operating on that principle, then, you know, I've drawn patterns and found, generally speaking, that if this team has a passion for getting better, if they take quality seriously, if they actually care about one another as human beings, as well as colleagues, they're taking calculated gambles to work outside of their comfort zone and they're delivering stuff, then that team is doing great. And yes, you can get down into more detail and you can break it down even further and so on. But my view is, if they're not, if they're not doing those things, then something in the system is broken because it's taking away their natural motivation. So how can we identify that and fix it? And isn't it kind of a dilemma for us as scrum masters when we know that sometimes the reasons why teams are not wanting to improve is that the managers who ask us how to evaluate teams are the ones creating the environment that leads the teams to want to hide information, as an example. Well, I often tell a story about this group of managers at an organization and one of them said, see, I don't agree with you, Geoff, because people are lazy and people need to be managed. You can't go around letting people define their own goals, define their own targets, picking their work, managing themselves, people need to be managed. And I was prepared for an interesting conversation. It was going to be an interesting day. But I didn't need to really, because one of his colleagues said, well, that's really interesting because my people in my area are pretty good. So you must just be hiring badly or something. But we had this big discussion about actually people will live up to or in some cases down to your expectations of them. And now what was what was took me quite a while to actually figure out I'm talking probably months or even years is that that that guy who stood up and said, don't know, Geoff, people are lazy. He wasn't just saying it to be difficult. He had evidence. He had empirical evidence that backed up his view because of confirmation bias, right? He got what he was looking for. He noticed what he was looking for. And if people thought, well, he doesn't believe in me. So I'm just going to, well, if you think I'm lazy, I might as well just do the bare minimum. He's got more evidence that backs up his view. So it's very difficult. You've got one truth against another truth. But partly that truth was self-created. And I think it meant more that his colleague raised it than if I'd have raised it because I was an outsider. I was a naive tree hugger compared to his colleague. He was another hard-nosed financial person. Yeah, absolutely. I've seen that a lot. One way to explain this is you get what you project on to people. So there's the confirmation bias, which is absolutely true. But there's also something in English, you guys use the phrase self-fulfilling prophecy, right? And because I know that my people are lazy, I'm going to treat them as if they are lazy. And what that means in practice is that you end up creating the conditions on which laziness is what is expected. Even though you might reward the firefighters, actually, you're really looking at a moment when the firefighter has a lazy moment and then you go like, okay, this is what all the people are like. That would be the confirmation bias. But then you create evaluation metrics and you create one-to-one relationships and, you punish people for being lazy. So lazy becomes kind of this narrative that overrules everything, right? Even the firefighters, even the crunch teams, the crunch time teams will be called lazy, even though they were working, if you remember all the weekends a month ago because of a deadline you created that didn't need to be created. But because this week, and I have a story to tell this, I have to tell this story in a second, because this week they took a day off and they're lazy. And this actually happened to a friend of mine. So she's crunch time. She's working for a major telco in Portugal and they have a new billing system to install. And I'm sure all of the people who work with software know how this goes. There's a switchover point, which is probably like 2 a.m. on a Sunday. And you have to do it at that time. And she's working there. She goes home at 6 a.m., gets a shower, gets breakfast, gets back to the office at 10.30. And her boss goes to her and says, why are you here this late? I tried something, took a gamble this week. So I was doing a workshop for a large number of people part of the department. And it was kind of team building on a large scale, if you like. It was strategy alignment and values and things for a large department of a big company. And I got them all playing a game. So we split them up into, I think, maybe six different teams of about 15, something like that. And it was an iterative game. So they had the chance to get better. But they also knew how they were doing. And about halfway through the game, the scores across the teams were really, really, really different from low to very, very high. And I just picked one of them that was in the middle, score-wise. And I said, just in case you're interested, if I was going to put a bet, if I was going to put money on any team in here winning at the end of this, it would be this team. I just picked one in the middle. And lo and behold, that team won. So at the end of it, I asked the whole group, why do you think I picked that team? And they said, you must have noticed something. You've obviously seen it before and they were doing something that you'd seen before. Any other ideas? Did you tell them what to do? Did you give them hints and answers? I said, no, no. I said, genuinely, I've done this many times before. This was the lie. I said, I've done this many times before. And it doesn't matter who I pick, they always end up winning. What does that tell you? They said, so then I asked the team that I picked and said, how did you feel when I said I thought you were going to win? They thought, well, we must be doing something right. Jeff thinks we're going to win. So their motivation went up instantly. And, and they started doing better. In fact, they changed what they were doing and started improving. Whereas the other side thinking, oh, we must be doing something wrong. And sort of lost motivation. And that sense of belief. Okay, in this case, it wasn't particularly genuine belief, but they didn't know that. But they believed in themselves that they, someone was believing in them. And, and they work but equally the opposite. In the other teams, the other five teams didn't do as well. Ugly, arguably, because they didn't think they believed in them. So it's that, but it's that dichotomy, isn't it? Is, in that case, to take it literally, Jeff, is one team succeeding and it's achieving greatness better or worse than five teams that have suffered or five teams that are under the government. And I don't know if this actually was the case in your game, Jeff, but in the real world out there in product development, all of those teams would be interlinked and dependent on each other. And one team winning actually means that somebody else needs to lose for that team to win. But the only way they win is for the others to sacrifice to give them something. Yeah. So this, this was the first game of the day where they were they were completely independent of one another. Yeah. And so we allowed their natural competitive spirit to thrive. And later on in the day, offered another game where the only way they could win is by all teams collaborating and some teams actually sacrificing themselves for for the greater good. Yeah, that's that's the big part of it. And I wouldn't, I wouldn't say that was successful. I wouldn't. And I think, you know, Vasco, you said about the bell curve type thing, that kind of performance management metrics been around for so long. And it generally demotivates everyone because even the people who who are at the top think they will, they could literally all agree to do worse. And there would still be the same results. There's a very famous guy who made a lot of money and was very famous on that bell curve idea. Jack Welch GM manager who said every every year I fire the low 10% like the lowest performing 10%. I was like, you idiot, in order for those guys to be the low 10%, right, they were the low 10% because of helping others. You're, you're, if you're firing the low performance, you're actually telling everybody don't help each other. What do you think that will create? Well, it looked like it created a lot of money for him. I did. There's no arguing with that. Yeah. By the way, there was another point I wanted to point out the moment we start measuring someone that affects their performance that it's in it's impossible to avoid if they know they are being measured or even just observed it affects their performance. There's a very well known well study called or the effect is called the Hawthorne effect, which was about measuring performance in factories and anything you change will affect the performance in that factory. And the same is happening with our teams. You give them a more attentive scrum master, their performance will change. You give them a more cooperative QA, their performance will change. You tell them they're being evaluated, their performance will change. So it really comes to the point that we as let's imagine we are the hiring managers, and we're asking that question. We need to figure out what is it that we are trying to achieve? Not how do we want to measure teams? But what is it that we're trying to achieve? And how do we make sure that the teams themselves can measure themselves, right, and hold themselves accountable to that overall systemic goal? Because at the end of the day, the game of business is won by the company getting better, not by every individual team getting better. I often tell a story that I don't think Ken reads or listens to anything I do. So he hasn't got around to putting a court injunction on me yet, but he can sway with this. He was once asked about teams, performance, and he said, so the person in the class said, I can see how this works with great people, really good people, they're technically skilled, with all the information they need, the environments they need, with management support and stuff, but scrum is not going to work with the average person, the average team. And Ken said, now you don't understand, scrum works with idiots. You can have a bunch of people who've never been to college before, who hate each other, never done software development, can't even tie their own shoelaces, raving alcoholics, doesn't matter. Because you stick them in a room with an ordered product backlog for 30 days, and scrum works absolutely fine. Now at the end of it, you probably have a pile of bleep, but you now know empirically what that team's capable of, and you can choose what to do with it. You can sack them all. If you want, you could send them to rehab, you could provide some training, you could give them a scrum master, you could change one of the team members, whatever, and then give them another sprint and see if they do better. And that is scrum working. So there's a lot of, there is truth to that, in that scrum is about empiricism. So what are the other questions that they wanted us to answer, which we've actually spent so long talking about team performance, probably on our time, is how do you prove scrum is working? Well, scrum is working if you are getting real information really, really quickly. Now, what you do with that information determines, I would say, how agile you are, because it's going to question your values and whether those values are in line with the agile values. So are you going to go and fire the weakest link on the team? Are you going to fire the lowest 10%? Are you going to force them to work overtime until they get, what are you going to do with that information? That's the key, which I think brings us all the way back round to Paul's opening comment around psychological safety. What do we think is going to happen if we do our best, if we are open about our abilities and our performance, and do our leaders, do our managers actually believe that we want to do a good job? I think, and this ties together some of the points we've been making, and I'm in danger of generalising far too much here, but I think, and this is probably our history, Geoff and I come from similar companies here, but I still believe there's a proportion of the workforce for a company that will think, does it really matter? So almost like, regardless of what I say, is this going to make a difference to me? Dare I say, am I going to lose my job? Yeah, just very much just teams, a large number of people that I've worked with and know of even now that are just happy to keep just a little bit underneath the radar and just to keep paying the mortgage, mortgage driven development. If I just do enough, and I don't make a big, maybe it's a British thing, I don't know. The tallest grass gets cut first. Exactly. So I don't know if that's British. I would say that that's actually a global phenomena, but from my perspective, I think it is a natural consequence of how we set up the work for these teams. I mean, you've talked to many teams all over the years that you've been training and coaching teams. How many teams are given the opportunity to be genuinely proud of what they are producing for their customers? Right? I've worked with some teams that, I mean, I worked at Nokia. We were producing very popular mobile phones. That was before the iPhone, by the way. And everybody was proud to work there. And these people, they would do amazing things even without Scrum. I'm not saying Scrum did it. I'm saying the people did it because they had a genuine opportunity to be proud of what they were doing. Right? And if we find that there's a lot of mortgage-driven development in our organizations, MDD, we have to ask, do we give them any reason to be proud? Yeah. Were you both in Nokia at the same time? The question, 2009 to 2011, I was there. Yeah. So Paul left. Well, actually, I joined 2008 and also left in 2011. I think a lot of us left in 2011. Yeah. There was a big, yeah, big message. There was the burning platforms memo. That's right. Yeah. That's when I started looking for a new job when I heard that thing. Yeah. But I think if you were in Finland, I was in Bristol. So yeah, we're probably working on different products at a different time. But yeah, genuinely, yeah, real people that were really infused by the products that they were building. And yeah, they did want to have a say. They were passionate enough to want to be part of it. And the best results I've seen are ones, I'm not talking about Nokia now, but smaller companies that I'm coaching and work with where the CEO and the board stand up and say, we want your input. We want to tailor this stuff, but we want your support and we want your say in how these things are measured and kind of resetting, even as far as like team values, but not just team values or company values. How does, what do we determine as success if we're achieving, maybe not in a specific measurable way, but in terms of the direction and the way that we want to work. And I think it only leads to the other. So I'm going to try and put myself in the position of the customer in this context. So the person who asked this question, right, and they've listened so far, they've listened to us talk for 50 minutes on this on this topic. And all they've heard is it depends. So they're going to come back and say, but I'm in this interview guys. And the interviewer is going to say, so after all that, what's your answer? How do you measure a team's performance? What's your answer? That's a very good question. So what I would say is what's important for you. And then take all of what I said into account and make sure you're not pushing the team to do something that isn't contributing to what you want. Oh, nice. And the question, I mean, for me, this is really, I work with product developers all the time, right? One part of the work that I do is coaching product owners. And it is really hard to get product owners to understand that they are not there to deliver features. They are not there to get more software developed. They are there to solve real problems that and here's the kicker, only the customer knows they have it. The proctor does not know proctor is a specialist on the solution domain, not on the problem domain, the customer is the specialist on the problem domain. And the POS don't even ask their customers. Right. So of course, the first thing, if I was being hired by, let's say, a leader of a product organization that was asking that question, I would say, how do you measure your organization? Because if we can't measure the teams in line with what you measure in your organization, it's all a lost cause, right? It's like shooting in the dark at a target that is perhaps in the wrong direction. Yeah. That's a good answer. And it actually reminded me of what I'd forgotten earlier on, which is so many teams complain to me that they're being asked to normalize their velocity or their velocity is being compared against another team. And I can completely empathize with that. And I actually don't mind teams numbers being compared against each other, as long as they're the right numbers. So I wouldn't be comparing velocity ever. All right. But I would be interested in how much value, business value each team, each product is delivering. So when you say, what do you measure? What do you want? And how much is this team contributing to that? I'll often go in and say, Jeff dollars. How many Jeff dollars is this feature worth? How many Jeff dollars is this product worth compared to the others? And we want to be we want to be delivering that we want to be delivering Jeff dollars. So if one team or one product is delivering more than another, that's, that's got to be our focus. Would I rather have two teams working on that product than one on each? Well, that's an interesting question to ask. So a team could be doing brilliant work. They could be the best team in the company. But if they're working on a bad product idea, what's the point? Yeah, the biggest way, the biggest waste is the work that we do that doesn't solve any customer problem. Yeah, I mean, you could, depending on how brave, if you're in this interview, I'll brave your feeling depending on how much you want job, depending on how much you want to impress. I suppose it's just that the awkward part of my personality would say, you could straight face just say, how would you, if you asked the question, how would you manage to perform it just straight face, poker face, just say, velocity, and just leave it there and then and then see, and then you can test the reaction of the your interviewer. This is brilliant, Paul, because it is true that in a job interview, they are evaluating us as much as we are evaluating them. Yeah, that's the fail fast approach, I think, Paul. A friend of mine went to a bank job interview with sandals and shorts. Okay, yeah, because he wanted to test the culture. Yeah. I think it depends how many I say how much you, you want the job and how much well, yeah, and it's easy to say, isn't it? If you're in a position where you don't need the job, but to, I would want to work for a company that actually, I want to test them to see if this is a place that I want to be at. So how can I find that out as quickly as possible? And I don't think, again, it's easy for me to say, but I don't think it's as risky as most people think. Say that, what is it? What is risky? Putting it back to them. Yeah, yeah, yeah, yeah. And saying, yeah, well, what's wrong with velocity? My other answer to that question when you asked me directly would be, my other, when I thought about it, I would answer it and say, I would want to measure all the things that you think it's impossible to measure. Oh, that's clever. What do you expect them to react? To let them know that I think there's things that are important that you're probably not measuring right now. Or I'd ask them, what do you think it's, tell me something that you think it's impossible to measure a team on? Or just to start a question around that kind of thing. I might not have an answer for it, but that's how I wish to learn through an interview. So here's a way then, right? So I'll try and bring it back a little bit. So for that person who is in the interview who perhaps isn't as blasé as you, Paul. So say, hold on a minute, let me just write something down. So write down your actual answer, just a bullet point or two. Fold it up and then say velocity. And when they say, well, you can't be higher because velocity is terrible. You say, good, I just wanted to test you, but here's my actual answer. Yeah. That is good. That is good. Although you opened up your game now, they know you're testing them. That's fine. I don't think those people are listening to our podcast, but anyway. Yeah, yeah. Well, we do need to wrap it up. I can't believe that's an hour gone already. We only have one question. That's insane. That wasn't the intent. We have to do this again. Well, maybe that's the tactic in an interview, isn't it? Just get them talking about one topic that you really, really like for an hour. Get them talking more, they're more likely to hire you, by the way. That's tip there, gotten from salespeople. Was it called bridging in politics? Yeah, filibustering, isn't it? Is that when you say, that's a really interesting question, but I think the more important question we should be answering is, no, no, filibustering is just talking so long that you use up the whole time. So if she can't get on to the next most important thing. No, no, don't do filibustering. Just get them to talk about how they measure teams, because that's important for you. And then you know how to answer. Hey, Vasco, we really, really appreciate you being here. That was amazing. In return, could we allow you just to speak to our listeners about, yeah, Joel, the online summit? Yeah, absolutely. Tell them why they should come. Yeah, so first of all, thank you very much for having me over. I've been editing and publishing the Scrum Master Toolbox podcast since 2015. So you do the counting of how many years that is depending on what year you're listening to. That's for sure. And as part of the work that we do, which is bring new voices in the community to the common awareness of us all, but also share new knowledge, share stories in the first person. We're doing the Agile online summit where we're going to get, we're going to have up to 40 different talks by people in the community, plus a free coaching clinic. So if you join, that's November 2nd to 4th, if you join at the right time, you will be able to ask questions from other coaches like Jeff and Paul and myself and get their support in your own journey. And this is what we believe in. We've been talking about how important it is to evaluate our employers, but it's also important to gather the knowledge from others. How do others do it? So this job application questions is definitely something you can come and ask on the Agile coaching clinic of the Agile online summit, which is free as Jeff said at the start, November 2nd to November 4th online at Agile online summit.com. Brilliant. Well, thanks for all you do. The Scrum Master Toolbox podcast has been good. I don't know how many hundreds of episodes you've done. They're all free. The Agile online summit is free. It's amazing. People like you, that's what keeps the community going. So brilliant to speak to you again and cheers. Cheers Vasco. Cheers. Thank you guys. All the best.