 You're on hello everybody. How are you? I'm Kristen lands, and I'm a member of team practices group here at the media foundation And I will be talking to you today about this notion of Team health there we go team health. What is it and what the team practices group is doing to help teams learn about and improve their overall health and also what we're seeing as trends across the team so my goals for this talk are for you to understand the health check philosophy and the process that we use to Gauge team health share what we've learned so far and also hear what you think I'll leave time at the end for questions and discussion Also hello to all the non SF people out there. I'm usually remote So it's nice to be on the other side of the camera So how many of you know what the team practices group is quick show of hands? All right, quite a few people and there's two members in the audience anyone from IRC. I can't see so I don't know But I'll assume a few so anyway just in short the team practices group is a Team here and we exist to help teams find practices and processes that work for them either through Dedicated resourcing so for example I'm a member of the mobile web and app team here as well as a member of the team practices group We also offer coaching to people who are just interested in learning a bit more about specific processes And also workshops by request for teams that might want to hone in Improve the things they're doing or even kick off new processes and practices We're also doing some org level stuff We've been involved in streamlining the quarterly goal setting process And we do some not just team level stuff But cross-cutting stuff like these team health checks that we talking about today clicker issue. Sorry. Thank you So just to be clear Team health I'm not a doctor. I'm not going around taking people's blood pressure What the team practices group is talking about when we talk about team health is actually a Set of qualities that it's a it's a range of different focus areas that make up a healthy team So Process First actually before I dive into the process Why do we even care about team health a core belief of the team practices group is that Healthy teams build better products so And it's not like we're making widgets here or whatever We have like really awesome products to make in a seriously awesome mission to fulfill So we want to make sure we have the healthiest happiest Most effective teams that we can have And I want to pull the thread on that a little bit too and say that Great organizations care about healthy teams I'm grateful every day to be able to work with the teams here and Do the type of work that we do in helping them reflect on processes and practices and Just their overall health and I think it it speaks to our leadership and to the values of the organization that We make time to reflect on how to be better and how to have healthier teams so the way that we start to learn about team health is through a Team health check survey so We kick this process off September of last year and this was really our pilot survey to Explore like is this idea even sound does it make sense? We just started with a handful of teams. There are Five teams and the idea was to do this health check with teams at or around the quarterly boundary And it's in a survey format That's the actual team health check that we use We shamelessly stole this survey from Spotify They've been doing this for a few years at least But we tweaked the survey in these focus areas Oh, there we go The examples to be appropriate to our culture here at WMF and to make sense for The things we care about Mike switch Jack Jack Okay, that one was this is better good. Good. Okay Where was I we stole a survey from Spotify? We tweaked it This is the survey you can see our focus areas over here and we include examples of awesomeness in these areas and crappiness in these areas and We have a rating scale the original scale was ranging from crappy to awesome So it was three points crappy meh and awesome It's evolved since then so the way that the survey is administered is Really important part of the process It's a group conversation So we don't give individuals this survey and they go and fill it out It's a team conversation with a member of the team practices group and we go through each of these focus areas team health focus areas and Converge on one rating So this is a really important part of this exercise getting to that single rating Raises a lot of really interesting information and a lot of questions come up and maybe even surprising things Like I didn't know you felt that way about food quality. So Given that a big part of these health check surveys are also Taking contextual notes as we go through the process. So we have Pages and pages of notes from the health check surveys that really inform Where the team was at at that point in time and helps Helps us to unpack like what they were talking about when they said that their Sense of value was mediocre for example And another nice part of this process is that it builds a lot of trust in the teams So we don't always get a chance to reflect on what we're doing in the course of doing business throughout the quarter from sprint to sprint from release to release and It's nice to be able to pause and talk through some of these things Sometimes it can be a little awkward or uncomfortable or even dare I say emotional But all of that builds trust and and helps the teams to Learn how to communicate honestly and openly with each other So we piloted that first version of the survey in Q1 we got a ton of feedback on How it went integrated that back into the process in the survey rolled out our second version in January and We just wrapped up our third version a couple weeks ago. So quarterly process We improve every quarter One important caveat before I dive into what we've learned is This survey is not meant to be an indicator of a team's performance. So it can be a Temptation to look at a team That rates themselves as crappy and say they're underperforming relative to another team But this isn't an apples to apples team. It's a qualitative assessment of each team's Sense of how they're doing. So there's a lot of reasons why one team could rate themselves crappy in one area One man's crappy may be another man's awesome So it's important to keep the context in mind and to know that these can't be side-by-side comparisons so what have we learned from Learn from the team health check process. I See it as there's three dimensions of learning We've learned a lot about the survey in the process We've learned a lot from individual teams and the teams are learning a lot about themselves And we've learned about some patterns and trends across engineering So what have we learned about the survey in the process? We learned about the timing and the pace so We've tried an hour. We've tried two hours. We've landed on about 90 minutes. That's about the right thing It's a bonus if we can get in a short retrospective at the end of the survey It's tricky to have meaningful conversation around these 11 focus areas in a time constrained environment, but 90 minutes is okay for that one thing we're going to try going forward is to change the sequencing of the survey areas So there's not so much context switching. So we might be talking about fun One moment and then jump to code quality. It's a little well It might not be a shift for some people. Some people think code quality is really fun but to group things together in a way where There's more of a natural flow and not in less context switching We learned a lot about the focus areas themselves and the examples we use so We got some feedback that some of them were confusing. So we tuned up the language there for example I think one of our original focus areas was speed We we changed that to pace because speed gave people like this impression of we have to always be racing and like killing ourselves and getting to the destination and Speed in and of itself isn't always a virtue. Sometimes it's more important to slow down so we changed speed to pace to give more of a to frame that focus area as We want teams to have the appropriate pace not necessarily be super speedy We learned a lot about the rating scale. We started off with three points People kept over a couple rounds of this Hacking that scale and saying, oh, this is like crappy plus or awesome minus So we gave in and made that a five point scale. I think where we're heading with it is one to five That seems to be what people want and this Word became controversial. This was the middle anchor in our scale for a while, but there was a sense that it wasn't That that it didn't convey the right thing to people it kind of gave the sense of I Don't care. I don't know. It had some negative Negative responses to it. So we changed meh to mediocre Again, the context is incredibly important not just for understanding individual ratings better But from quarter to quarter, it's great for teams to be able to look back on their notes and See where they were. It's like a snapshot of the previous quarter And it helps them kind of connect the dots to how they got to where they were in that current quarter We learned a bit about scheduling of these surveys We've tried for the quarterly boundary and after I think in the organization that we're in it makes sense to do it after the quarter ends we had an experience this time around of People saying I would have rated our value high But we didn't have a chance to release yet because we're not releasing till the end of the quarter so I think we nailed that we also want to turn around our survey results a little quicker and Communicate them back out to the teams and to the team stakeholders One of the biggest lessons we learned from the first round of the survey was and we learned this from like really bombing one of these surveys and Sort of failing it failed for a few reasons, but the big one was not establishing the right expectations at the outset of What the purpose of this thing was for when you get into reflecting like this Sometimes it can feel like finger-pointing or blaming so it's just really important to establish that this is a tool for learning And not for blaming so we've changed how we frame this going forward and most importantly We established that this practice and this process of measuring team health is valuable to the teams Through feedback from the teams and from the people that support them Which is good because if it wasn't valuable we'd just stop doing it so What have we learned on the individual team front? I don't have time to go through Every teams results right now. We're up to 11 teams that have taken the health check survey How many people here on the hangout or IRC have taken the health check survey? All right For So, yeah All these results are available on office wiki. You can Go check them out right now. They are not public yet. We're still working out How we want to present this info And it's important to note again that The individual Survey results are not the whole story and textual notes are available You can talk to the individual visual teams if you have questions about how they rated themselves But it's important to remember that that isn't the entire picture These are the teams that have participated so far in the team health checks and Analytics research and community liaisons are bolded because they are actually not purely engineering teams So we're expanding this process out beyond engineering once again the caveat So The moment we've been waiting for what have we learned? What is the state of team health across engineering? Sounds so dramatic the state of the Union Spoiler alert. There is no definitive state of team health across engineering, but I will present some patterns and trends that we found A warning we're about to see some very colorful slides So these are the rolled up results from our first round of team health checks So we averaged Across the teams and came up with a single single rating across the teams So what do people notice? Here Anything stick out? Lots of yellow things were feeling kind of mad Q1 Support was red So people were feeling pretty crappy About support so we rolled up these results and we also pulled out some of the salient points from the notes from the contextual notes that we took so teams are Feeling difficulty in escalating problems. They felt like they were always waiting Too many external dependencies things could go forward Anything else that we notice? Mission fun and learning are looking pretty awesome Does that sound like WMF? So that was our proof of concept initial Survey Ultimately, it was just four teams represented here. Oh, that's a different view. I told you it was colorful So our second round we doubled the amount of teams that was nine teams So things look a little bit different here What do we notice? That's different here. No red more green some more awesome More teams giving input Anything else? Yeah, so again, we rolled up the results came up with an overall rating and Hold out some Some trends and highlights that we observed different view of that so After the second round of doing these health checks We really wanted to you know, keep keep the learning inside the teams but also Pick out what we saw as trends across teams so the big ones that came out for from not just the ratings themselves but from the notes were tech debt and Cross-team dependencies were the big pain points so This is where those contextual notes came in super handy again because these were both meh yellow in the survey results But as we looked through the notes, we noticed that tech that was impacting Value is impacting speed. It was like hitting all these different areas same with cross-team dependencies So here's another visualization of All of these results. So I anonymized this and jumbled up the data. This isn't even real data I just wanted to show an example of How another way that we can visualize the trends across teams so you get that horizontal view You can see what's happening across a focus area and then you can look In the columns and see how the teams are feeling about themselves Team nine is feeling pretty darn good about themselves so we looked at these trends across these nine teams and And asked ourselves, okay, what do we want to do about these? So our plan was Team practices group. That's TPG Was going to take ownership of improving the situation around tech debt Cross-team dependencies we wanted to present to engineering management to see if they would take ownership of Helping out helping resolve some of those issues oops sneak preview So team practices group has met with release a subgroup of the release engineering team to talk about what we can do to improve the state of code overall across themes and came up with a provisional plan Plan of attack to Gather a tiger team of people who really care about code quality test coverage to to try to come up with some solutions To ease tech debt and standardize on some practices share practices and try to take Try to ease the pain on this one of it Arthur who's not here and he's in the Mojave desert has bubbled up the cross-team dependencies stuff to engineering management and I don't have a latest update on how that's going But I heard the conversations were going well and there's some ideas out there about how to ease this pain So I mentioned that we just finished up our latest round of health check surveys and this is My interpretation I haven't had a chance to Go over this with the team practices group but From round three and this is probably not terribly surprising, but the things that are coming out in this past quarter are kind of a hit decrease on on how people are feeling about mission and goals and destiny so in light of our Organizational changes reorg stuff I think people are just wondering How do our team bowls aligned with the organization goals? They are wondering that like several teams ask that very question People are worried about their autonomy. There's this call to action. Sounds exciting. But what does it mean for me? What does it mean for my team and how I work? People mentioned a looming reorg and there's generally a great hunger for clarity about this stuff So what's next for the health check survey? We're going to keep adapting and iterating. We keep getting feedback from the people that take the survey We're going to include more teams including more teams outside of engineering We want to continue to measure our effectiveness. So So far this has come in the form of direct feedback from teams through retrospectives and from managers But we want to want to keep honing in on how effective we are and Think of the best ways to measure that and we want to I think obviously we want to evolve this. So are there other Ways of measuring team health that we could think about are there complementary things we could do for example I've been talking to some people about Maybe doing more like of a team 360 talking to teams you collaborate with and getting their input and their thoughts on their perspective on what your team does and Yeah, what's next for team practices group? This is our good friend. Okay our objective key results This is our objective for next quarter and it's not directly related to team health checks, but Our goal is to develop repeatable best practices for engineering teams and It's sort of like a almost like a menu of practices that people can look at where we describe the best practices that have worked well for people here and We could certainly feed this back into the feedback we get from health check surveys so the challenges that I see us facing is Continuing to identify the best strategic interventions for People that support the teams and for the team practices group So what's going to get us the most bang for the buck? What areas can we focus on to keep improving team health? And we want to measure how well What kind of impact these these interventions have whether their process changes or whatever But there's a balance between like measuring stuff and Well, you don't want to be creepy about it and be like oh our fun has dropped below three. So now We got to do something. So there's a balance of measuring stuff and Continuing to make the qualitative dimension The survey is visible and making sure we have plenty of context around those and a big challenge that I see is balancing standardized practices for teams with what works for the team individually. So There's stuff that we want to share and Standardize on but also there's practices that aren't appropriate for certain teams. So we want to strike the right balance there That's just attribution. Okay. I think I'm done I'd love to hear questions comments and thoughts and also I just want to say thank you to my fellow team practices group members for being great colleagues and developing this process To our leadership who support this process and the team practices group and to the teams themselves who take this survey and teams we work with It's always really a great honor to spend time with teams and hear what they're up to and hear what they care about. So thank you And if you have questions, could I please ask you to use this microphone over here? Thanks Burning questions about team help We'll close your mouth Pull it down. It's on a gooseneck There you go. I'm curious if you have had any This particular team practice team practices group has worked with any other teams besides engineering We worked with we did our first well Research and data team have been on board since the beginning actually and I think they do a little bit of engineering But I don't think there's strictly an engineering team and they were actually Instrumental in helping us to evolve the survey because their feedback drove us to change it to be Generic and not just engineering specific. So we changed releasing which was one original focus area to delivery So delivery, what are we delivering? We're delivering maybe publications or reports or we're delivering services So the survey is now ready for primetime with non-engineering teams We also did the survey this past quarter with the community liaisons And I'm sure you probably mentioned it is the survey posted Yeah, the survey itself is on media wiki.org and the results are on office wiki, okay Quick question from IRC. Where can one read about these results technical bet? Good question Nowhere yet, but note to self next check in we do with release engineering We shall make a note to get some documentation up. Hi. Yes, sir. Thank you for your presentation So you mentioned at the beginning that the process was done through a group discussion that I was Fascinated by the TPG group And you mentioned that sometimes people were discovering Feelings from other members of their team. So it feels a bit like Like couples therapy where you know just having people in the same room and talking about Meta, you know meta awareness and discussion, etc. Just can improve communication between members of the of the group. So Is that your feeling as well? And do you think that maybe just Having these discussions is a net benefit Yes Yeah we explicitly Part of we explicitly exclude Like next steps from this process. So we don't go in saying we're going to find all these problems and and Identify next steps. Sometimes we do find them, but I think we really wanted to keep it as the value was in the conversation itself and You know, it also sets a baseline for that quarter, but outside of the process people can talk about What actions they want to take afterwards? They can take this information and do what they want with it But I absolutely agree that The process in and of itself is valuable. It's time that we take as a team It's a special thing that we normally don't get to do in the course of day-to-day business So it can be really illuminating to hear What is you know, what are the pain points that the person you work to next to every day is experiencing? It provides that space to listen and talk Be curious to hear from anyone who's taken the health check survey about their experience While someone's waiting to give that experience, there's another question from IRC Which is are there any blog posts on the results? No That would be a good one. I Don't think so. Grace or Kevin. Do you guys know? All right, we'll put it in our team practices group backlog other questions about process the results team help Anyone who's taken the survey All right, thank you everybody. I guess I'll point out to we have team team practices mailing list Join up on that. There's some interesting conversation there and resources being shared and Team practices IRC channel as well. All right. That's it. It's a wrap. Thank you