 300, 400 sogs on my phone for my entire flight, which has saved me a few times. Spotify is a Swedish company, but our tech offices are in Stockholm, Gothenburg, which is the second largest city in Sweden, and New York, Boston, and a small one in San Francisco, but our biggest two are Stockholm and New York. So I was hired to Spotify to be a software engineer. So I was working in data infrastructure. And then I switched to another team that was doing A-B testing, so a lot more about statistics and building analytical tools. And in that team, it was my first time working with Agile, and I just really loved it. I loved the process of breaking things down, and I loved the process of cutting things into smaller chunks and working iteratively to build the bigger picture. And then when we lost our Agile coach because he switched to a different team, I said, okay, this is really interesting, and I want to take this work on. So I started doing some of the coaching. I would run the retros and find the team activities and be the cheerleader or whatever it was. So for me, I started as an engineer and I switched to being an Agile coach. So I guess it's not really a surprise that I'm so preoccupied with wanting to quantify things and wanting to have measurements, especially given my background in statistics. So when I first got really officially into the role of being an Agile coach, you think a lot about what are the things you hear when you read the literature, you go to conferences, et cetera, right? It's a lot about breaking down work into stories and tasks, prioritizing those tasks. And then delivering those tasks according to the prioritization, right? Measuring cycle time and velocity and burden of chart and things like that. And on top of that, we had rules. So I don't know if you're familiar with any of this, right? Task must be less than x story points with no more than y story points per sprint. And if we can stick to that, then everyone will be happy. It'll be perfect and we'll have the perfect product, right? Or our stand-up meetings have to be less than ten minutes long. And if they're even a minute over, then we failed. Who cares if we've talked about blockers or not, right? You hear a lot about this. If you stick to this one prescribed formula for value delivery and for a following Agile, then everything will work perfectly, right? And on and on and on. I'm sure you've heard of a million or so. But what I found is that this gives us a very microscopic view. So we're thinking a lot in terms of moving one story through the pipeline, right? Or just focusing so much on breaking down a task and delivering value that we forget about the people and whether they feel like they have a sense of purpose. Whether we feel like we're engaging, I don't know, all of their viewpoints, etc, etc. And right, again, the end goal is to achieve sustainable long-term value delivery, right? So again, do people feel like they're contributing? Do people feel like they're getting support from the organization? Do people feel like they, I don't know. Yeah, again, have a sense of purpose, right? There are a lot of ways to get there, there are a lot of ways to get there. And sometimes when we focus so much on a stand-up being this number of minutes or having X action items come out of a retrospective, etc, etc. We end up only seeing, right? We end up being stuck in the weeds instead of being able to see the bigger picture. Again, if we focus, if we focus so much in the weeds, we'll miss the bigger picture. And again, that's great and all, but what is it that makes teams high performing, right? What are the elements of this? It could be a lot of things. I mean, for some companies it's different in different countries, it can be different. So sometimes we rely on gut feel. We say, I feel like I'm doing a good job. I feel like people are smiling more and they're happier, right? But gut feel is notoriously flawed. So for example, maybe if you're just in a good mood, then you rate things better. If you're in a really bad mood, then you think things are much worse than they are. Or confirmation bias. You might be looking for, you have a hypothesis in all the evidence you see that supports that. You tend to pay more attention to everything that doesn't, you pay less attention to. I'm sure you've all, you can recognize a moment where you've done that yourself. This is an amazing book, especially around these topics. It's really, really thick, but it's worth it if you can get through it. Thinking fast and slow, I definitely recommend it. To learn more about why gut feel can be such a problem. But again, right? You can't manage what you don't measure. So for us, we've developed a set of characteristics that we think are indicative of high performing teams. And again, this could change based on the culture of your company or the culture you're trying to aim for. So for example, quality discipline. It's not, yeah, if we're optimizing only for that shipping product, then what about taking it from the MVP stage to making it into a fully fledged product that can stand on its own? If we're just ignoring quality, as I'm sure you know, we're building up technical debt and problems for ourselves down the line. Customer engagement, are we doing either user testing or if we're an infrastructure team where I was working before, are we talking to the other teams in the company to say, how is this working for you, et cetera, et cetera? Are there any questions about these? Because sometimes people are confused about some of them. So maybe one or two people have a question, but they're all clear. Cool, these will be online later, or you could ask me questions. But these are some of the things that we look for. So some of the agile coaches at work came up with a matrix to say, right, how can we think about this in a more structured way? So these are the characteristics that we've come up with. But it's one thing to have a list of this, right? But how can we make sure these conversations are happening? It's not that in your stand up, it probably doesn't come up very much. Yeah, I really feel a sense of purpose here. Or when you're having, I don't know, whatever sort of meeting it may be, things about quality discipline, maybe those type of things come up. But it's not always the case. So how can we make sure that we're touching on all these points? And how can we make sure we're doing it in a regular cadence so that we can really be sure that things are going well, whether they're not or where we should be spending time? And that's where squad reflections come in. So in squad reflections, we have a series of 11 statements. And every maybe four to six months, we sit down with our teams and we say, okay, on a scale from one to ten, how true do you think this is, how much do you agree with the statement? And then we have a discussion around it. And from that, we come up with some action points. And I'll lead you through that process in more detailed in just a moment. But before that, maybe you recognize this XKCD, but we're still working with really small teams, ideally, maybe five to eight people. So that means you have a really small sample size. Which if you're familiar with those sort of statistics, it means that it's really hard to draw statistically significant conclusions from the numbers that you're getting. Which is why it's much more about the conversations that are happening. So as much as this is about quantifying and measuring, I do have to give the caveat that, okay, we're still working with really, really small sample sizes in terms of the people. And again, because of the flaws of gut feel, right? Maybe one day you come to work and you're really happy. And so you give everything a ten and maybe some day for someone a five is the same thing as someone else's ten. So the one caveat to not be too caught up in the numbers. So here are the statements that we use, right? It is clear to me why our team exists. It is clear to me who benefits from the things we deliver. I feel that my team is able to influence the world of Spotify. I'm happy with how frequently we deliver value to users. And I'm proud of the value we deliver to users. Reregularly identify the most appropriate way forward. We continuously improve on our way of working. I give feedback, I get feedback. And I feel a sense of purpose when I'm on this team and we get support from the organization. So I'll have these all online for you later so you can see them. But these are the 11 statements. So as you can see and I was mentioning before, it's a mix of feedback, purpose, are we getting support, continuous improvement, et cetera, et cetera. So trying to facilitate conversations around these 11 points. So how does it work? First, we get the whole team in a room. I don't know if you know the teletubbies. But I think it's just for like little kids. But it was the first team I could think of. I'm not even sure they're a team, but they have matching uniforms. I think that's like 50% of the way. So anyway, get everyone in the room. If someone's sick one day or something like that, I try and reschedule it. Cuz again, we're already working with really small sample sites to begin with. It's important to have everyone in their opinions represented. Then you read each statement aloud. And like I was saying, rate from one to ten, where one is really disagree and ten is really agree. You could switch it around the other way if you want to. It doesn't really matter. And then, it's important that everyone writes this down on their own. So that we reveal it at the same time. Cuz sometimes what can happen is that if I show maybe five fingers, and everyone else has ten fingers, maybe I'll say, no, maybe it's a six instead, or maybe it's a seven instead, and we want to avoid that. So it's important that we have everybody revealing it at the same time. From there, we've facilitated discussion around each statement. So for example, I will say, why is it that we feel this way, right? Because the most important part is to see what comes up in discussion and to really drill down and say, okay, why is it that we're feeling this way? And to also, for example, if someone gave it a six and everyone else gave it a nine or ten, I'll say, okay, why is it, what do you feel we're lacking that ever, relative to other people, etc. And the same in the reverse. So these are my three main ways where I try and facilitate conversations. And from there, I mean, usually I have about 60 to 90 minutes for this. So you can't really dig into a root cause analysis of every single statement. But what you can do is you can try and understand, at least to a deeper level, okay, what is it that we feel, why do we feel this way? And is this important to us? Is it not important to us? So then the next thing I do is I take notes during the discussion. And it's important to note that I just write down everything that people say. So I either take direct quotes or a paraphrase. Because if you're in the conversation and you're trying to analyze at the same time, sometimes you're focused too much on trying to make sense of things, as opposed to just taking more of a record of this is what happened and this is what people talked about. So as people are talking, I just try and type as much as I can and write down word for word basically what they're saying. Because at a later point, I'll come back to it and read through it. So it's really important that the team feels a sense of ownership over their continuous improvement. Because otherwise, what happens is that, yes, well, I might be able to get a team as a coach to see through some initiatives or whatever it may be. If I leave the team and people aren't really fired up about continuous improvement and people aren't really on board with that, then, yeah, as I said, once I leave, that kind of all crumbles in a sense. So I've experimented a lot with, okay, how do you get people to really have a sense of ownership over their continuous improvement and not just a sense of ownership, but a real passion for it and wanting to really see it forward. The part that I do is I review the discussion notes and I organize them into the themes that emerge. So again, I reference back to the characteristic documents and I go through the notes and I say, okay, this sounds like it might fit under here, this sounds like it might fit under here. So it's really more of a pattern recognition than anything. But at the same time, some things will come up more than other things. But I don't come back to the team and say, okay, these are the things you should focus on. I might have a strong opinion about it, but I don't let it be the final word, right? So what I do is I put it into a document and then I take it to the team. I share it with them. I say, okay, these are the things you've said. These are some of the themes that they group into. And I talk about it with the team. I say, what do you think about this? Do you think this is an accurate representation? Do you think I've miscategorized things, et cetera, et cetera? And again, the team needs to have a sense of ownership of their continuous improvement. So once I've come back with the themes, what I did this last time was I wrote down all the improvement areas on sticky notes. And then I made a scale. So things that are refined to be really, really high value and things that we find to be low value. But everyone was like, no, everything's valuable. So I had to cross it out and I'd be right, less high value. So there's high value and less high value because everything can help in some way or the other. And so then what I had the team do was I said, okay, take these sticky notes and just put them on the board where you think they land. And then if you disagree with someone, keep moving it. But it's sort of a silent exercise. And then when we feel like it's stagnated, I say, okay, now we're gonna focus on the top two to three of these. Because again, even in our continuous improvement, we have to have a whip limit. I don't know if you've seen this in off-sites or in retrospectives, etc. Is that you have so much other work and you say, oh, these are all great things and we want to work on all of them. But a lot of them kind of fall through the cracks, right? And you only have so much of a work in progress limit that you can have for also your continuous improvement. Unfortunately, it doesn't really seem to be the case that you can do both, which is why I really forced them to say, okay, we're gonna pick the top two or three. So once I've done that, I write it down and we come up with some action items with the team and I get them to brainstorm and say, okay, for example, this last team that I did that with, they said we want to focus on celebrating wins and learnings because we do a lot of great things, but we don't really feel like we're paying attention to it or pausing for a moment to say, wow, this is great work that we're doing. And so we come up with some action items together and again, we pick the top two to three there. And then what I do is I put it in a document and we see how things go for the next three to six months. So every few weeks I try and have a check in and say, okay, how are we progressing on this, etc. And I try and observe them too and say, okay, do we feel, does it seem like if customer engagement is important to us, do I see more emails to the customers? Do I see better interaction in Slack channels, etc., etc. Speaking as someone who works in the infrastructure team, it's a bit harder when you're working in the payment system where I work now, emailing individual users. So yeah, again, I put it in a document and I say, and three to six months later when we do the squad reflections again, I repeat the whole process and I see what comes up. And if it's the exact same things that come up, even though we might feel happy and great, etc., etc., if it's the exact same themes that come up, then I have to pause and say, am I really doing my job as a coach, right? Because even though we might feel great and our gut feel might lead us to think that, oh, okay, I'm really having an impact. If the main pain points are still there, then are things really getting better? So that's sort of my way of benchmarking team performance. Another great thing about this is that, I don't know who in here is an agile coach, but sometimes, or at least when I was starting in the beginning, you kind of hop into a team and you observe a lot and you get to know people and you make friends with them, etc., etc., but sometimes that can take a while to get geared up and to really get going. So what I found is that now when I join a brand team, I run this exercise with them and it's like day one, I can know exactly, okay, these are the areas I should be focusing on, which as a coach reduces my, or increases my velocity in a really big way because I can literally hit the ground running. And as I said before, these are online on my blog in case, and the whole process is written down too in case you want to reference back. And the slides will be somewhere at some point now also. My blog is very MVP. I thought it might be the best way to share it, so it's not the most beautiful thing you've ever seen, but it works. Now it's time for some real-life examples. So, I'm not sure if you're doing well. Yeah. So, I don't know. I think you'll see some of these are grouped together, right? So I think in some sense, we say, okay, well, if we're gonna do this one, we sort of have to do this one because they're so connected together. I think in general, something like celebrate wins and learnings. I'm not sure. There's so many dependencies there. I guess it's hard to get a perfect big obstacle. That's perfectly optimal. Instead of actually, I just need to set this up. So I always have my time to do this. I'm thinking about, you know, let's just do something and we'll do it. We'll see what happens. So it's much more of a 3D algorithm approach. Yeah, maybe we can go with that one. Real-life examples. So, I was running a session with one of my teams. So I work in the payment part of Spotify now, and this team is focused on conversion. So these are some of the direct quotes that came up. I don't always reflect on or realize how much value it is that we're delivering. Or we do deliver a lot of stuff, but once something slows down, it feels like you're not delivering when really you are. So sometimes, you know, when we get caught up on little things, we think, oh, nothing is going and we don't really pause to think, oh wait, but we're actually doing great things. We should be proud of this and have a sense of pride over our work. So again, here are the characteristics. So which one do you think those quotes fall under? Any guesses or opinions? Any other thoughts? Yeah. Yeah, I guess there could be a lot of things, but for us, we said, celebrates wins and learnings, right? We do have the sense that we are delivering a lot of value, right? So it's not the value delivery that we're calling into question. It's more the fact that are we really pausing to think about it? Are we really, you know, giving people the sense of pride and accomplishment for the work they're doing? So it's great to identify that, but how do we fix it, right? There are a lot of different things we could try, but as I was mentioning before, sometimes you just have to pick something and go with it. It might not be the perfect thing, but it's a small step and whatever could help, right? So I came up with the wall of awesome. So this is right near our stand-up board. And so whenever we're doing a stand-up and something awesome happens, I say, okay, maybe that should go in the wall of awesome, which in the beginning feels a little bit, you know, like, oh, I'm the cheerleader, blah, blah, blah, but I'm American, so I think they just think I'm whatever, the crazy American, whatever, so excited about everything. So for example, here we delivered e-cards so people can send gift cards, Spotify gift cards, et cetera. And we'd been working for a long time on that, so it's like, maybe that goes on the wall of awesome. Everyone is pulling for us to win, that's a great thing too. I even put this sticky note, it's a meta-sticky about the wall of awesome reference. So I'd been talking about it for so long and I was always the one saying it and then someone else said, bring the key to our team. So again, I focus on this one. I can just read to you the next one, maybe. So here's another one that came up. We know there are things that we can improve but we don't do them. We don't follow through. It's not that we're not aware, it's that we don't do them. I don't know if you remember the characteristics or if they'll be showing up, but we'll stay agile. Right, so we know there are things we can improve but we don't do them. So we said, okay, that pattern is iterative discipline. So we were noticing that we would put out an MVP, so we have stakeholders such as marketing and business, et cetera and they have a long list of campaigns and ideas they wanna try and once we put out, so once we put out, for example, an MVP of the eCards project, that's great and everything but if we ignore that and move on to the next thing all the time then eventually we're building up this backlog of technical debt that's gonna come back to bite us. So we said, okay, iterative discipline is something that we need to focus on. Obviously there are other dependencies here, right? Being able to push back on the demands of marketing and business, et cetera. So how did we fix that, right? Again, there are things about trying to set expectations with stakeholders and things like that, which is something we're also working on. But something else we did that was quick and easy was the champagne meter. So another girl, our product owner who'd worked somewhere else before had a champagne meter. So basically when we fill the champagne bottle all the way to the top, we celebrate with champagne, obviously. And we decided that for us, we would fill this up based on how many times we went back and tweaked a product. So our saying at Spotify is think it, build it, ship it, tweak it. So we were great at the first three but we didn't feel like in the squad we were coming back so much the tweak it part. So every time we do put something into our backlog and work on improving a pre-existing feature, then we get a fill the champagne meter. So you can see right there that little dot there or that little mark there. We made some improvements to eCards so we were able to fill the champagne meter a little bit. Again, does that completely solve the problem of iterative discipline? No, I don't think so. There are a lot of other forces at work but the fact that it's there in our stand-up, in our squad area, and we can see it as a concert reminder of are we thinking about this? And even the fact that the level of how filled up it is is a way of holding ourselves accountable to the fact that are we really following through on this? We said it was important. So making it your own, right? These are things that matter to Spotify culture. And I guess there's probably a lot that overlaps with the culture in your company but I'm sure there are some things that are unique to where it is, right? What elements are unique to your culture? What do you value? And shaping either characteristics or shaping statements around that so that you can have these conversations. Because even though this is a talk about measuring team health and a way to try and benchmark it by comparing where you were three to six months ago to today, yeah, it's still a way to facilitate having these conversations that wouldn't come up otherwise, right? So maybe if you're thinking back to where you were, where are things that you feel like you're lacking or areas you want to get better and find some way to incorporate them into this and then have a way to have conversations around them. So right, squad reflections, I guess if you want to sum it up, is that they help us build a more comprehensive view of a team's health in a way that even strict agile adherents can't, right? So even if we optimize our agile process, there are still things that aren't necessarily getting talked about, that aren't necessarily getting, yeah, focused on, but things that could be really important to the overall health and happiness of a team. And I think this about, oh, it's not going. No, is that going? No, that's so weird, it worked before. Anyway, basically, this guy's so focused on fishing and the sheep comes up from behind and it, or whatever it is, and it runs in and it pushes him in the water, right? So if we're still focused on the task at hand and we're ignoring our external environment, sometimes it might come back to bite us. Any questions? I'm a coach right now for two teams, but there are some other teams in my area that don't have a coach at all. So I guess in some way you could say I'm a coach for all of them, I'd just like to. I'd say it's most, well, in some ways it's unique for each team, right? Each team is a unique set of individuals. But in our product area in general, I think this iterative discipline kind of an overarching theme in our product area in general because some of it has to come, or some of it comes from relationships with stakeholders and pushing back on some of the demands they make and also, et cetera. So in that sense, there are some things that overlap, but in general, things that come up can be very different. It's never the same set of two or three for any team. A squad is just a cross-functional team of engineers, a product owner and an agile coach. Sometimes chapter leads are working as engineers as well, but I would say most teams do scrum, but some do Kanban. So it's not necessarily that it's a scrum team. Yeah. So I'm in the Iron Bank tribe with payments. We also have customer support. So it's sort of, you could say, like a product area. So there's another tribe that I was in before called infrastructure and operations. So we do, you know, we have your SREs, your security, your data engineers, people keep continuous delivery, et cetera, et cetera. So those types of things are grouped together as a tribe. And then we have, yeah, the individual squads make up that. And if our tribe is really big, we might have product areas. So I'm in subscription platform. So that's sort of a subset of the tribe. So it goes the tribe, a product area, and then the individual squads. You know, they have something completely different than in the community. So how do you take those points? Do you make it less value, how do you decide? In terms of discussions about statements, I'd say that happens almost every single statement, right? For several reasons. One person's four is another person's seven. But I think for me, that's a useful tool because that's a way for me to facilitate a discussion pretty easily. Because if everyone gives it a seven, it's hard, you know, it's like, okay, I guess we're all feeling pretty similar about this and we agree upon this. But when someone else feels really strongly the other way, then I find that it gets the discussion going more. So I almost find it a useful tool. And again, you know, we do take these numbers and we take averages and then I also keep track of those. But again, the team composition can change over time. Some people go and join new teams. Some people, sometimes we get new team members. So it's hard to really look at these averages or even if you want to look at the mean and the variance, it's hard to look at that as statistically significant science. But yeah, I think that's a perfectly okay thing. Yeah, that's the... This and this, but I, you know, you can rate it whatever you want. It's really about the conversation that's happening. Yeah. Yeah, but again, someone's utopia is different from someone else's utopia. That's why sometimes, I mean, every time teams get really caught up in the numbers because they think it's about the numbers, but really it's just about what we uncover and the type of conversations we have. It's the numbers are a means to an end really. Some people look at them. Sometimes I'll look at them. It's just, again, it's hard to say. Sometimes I will look at them and I'll see what was the weakest link here. What was the weakest link here? But I don't find that to be as helpful and informative as the conversation itself. Yeah. So how long does it take for a squad reflection? I usually say 60 minutes is too little. I aim for about 75 to 90 minutes. And then maybe, I don't know whatever it is, five to seven minutes a question. Maybe every four to six months. So that's the freak, she was asking me the frequency. It's every four to six months. Because otherwise, if you do it every month, it's hard to have had enough time to really get to develop some of these things to get better in some of the areas. So it's more like a bit longer term than say a retrospective. The whole process. No, so the 75 to 90 is just for squad reflections. And then as I'm doing it, I'm continuously improving the process. So I had another meeting, that was maybe 15 minutes of one where we said, okay, which ones are high priority? Which ones aren't high priority? And then when I come back, I'm gonna have one just brainstorming action item. So that might take 45 or 60 minutes. So yeah, I guess in total, it takes a bit longer than that. But the most important part, the conversations. So does management ever try and collect these numbers and compare? That doesn't really happen at Spotify. I'd say, especially having our background as being a Swedish company, like the trust runs through the organization, right? Even to a point where sometimes it's like, surprises me even, but no, it's not really how it works. But I mean, that could be a pitfall somewhere else. Yeah, so teams also do retrospectives. So our cadence for that is about every two weeks. This is something supplementing that. Cause sometimes you come into a retrospective, you're thinking about some of the things that happened in the past, I don't know, two or three weeks or even the past week. And again, it's not always so structured around these sort of topics. I mean, it can be, but yeah, they compliment each other in terms of, of this or just in general, as the don'ts or the dos, maybe the dos. Yeah. I would say, what I was trying to explain to people who are interested in switching over in Spotify or who are just curious in general is that it's great to know frameworks. It's great to know how to facilitate this with that retrospective in a certain way. That's all great. But if you can't build trust with your team and if you can't have people like have some sort of confidence in you, just none of it really matters. I think when we find people are being a little bit pedagogical and they're trying to say, okay, we have to do this by the book, et cetera, et cetera. It doesn't really fly with our teams. So I guess that's the don't. And then the do is, you know, I just go in and become friends with people really. Yeah, at the end of the day, it's a people job working with people. It's about trust and confidence and yeah, transparency, et cetera. So you're asking me about individual performance? How do we engage that? Well, we have one process that we do for performance reviews. It's called loops and we go and we ask everyone we've worked with or even other people that we might have interacted with for feedback. And so we get all that feedback maybe every, I don't know, 12 to 18 months is the official cycle, but I try and do it as much as possible because it's always good with more feedback. And so that's the process we use to get feedback from peers or maybe from the product owner to the chapter they did, et cetera, et cetera. And I think that that feeds into how the manager evaluates things. Because again, our managers aren't working directly with the people they're managing. That's not how our teams are set up. So we rely a lot on what the peers are saying with the other people involved. Feel about it. Any other questions? No, okay, I'm gonna hop over. Thank you. And these slides will be up somewhere and the website will be up. Yeah, thank you.