 All right, it's 45, and I'm gonna start right on the nose because this is gonna be long, so bear with me. Before I start, completely unrelated, can I get a show of hands? How many people here know what a maple bar is? Yeah, that's what I thought, okay. We can talk about this afterwards. All right, so welcome to my session. My name is Stephanie, I'm gonna talk about agile things. If you looked on the schedule, you'll notice this is an intermediate session, so if you're expecting to be taught what Scrum is, I apologize, that's not gonna happen. But what I will use this talk to talk about is some pretty honest things about how we do things at Amazee Labs. Some things that we're not particularly proud of, but that's what happens when you're growing and you're trying to become better. You share the things that you learned the hard way so that other people don't have to experience those same things the hard way. So another thing that we're gonna talk about is we're gonna go through some of the how we did things and there will be parts during this session where you'll have to do the squinting game to see what I've put on the board. Don't worry, afterwards I'm gonna post the slides and you can come and talk to me and I'm happy to show kind of the back-end of how we do things in JIRA if you have questions. So no panicking. So the first thing is, again this will, sorry, who here knows what Scrum is? Okay, fantastic. Who here does Scrum or wishes they did Scrum? Okay, cool. So there's some of you who haven't raised your hands. Buckle in, it's gonna be confusing. So again, this is not gonna be a how to do Scrum talk. This is the how we did it at one point and how we do it now talk. This is gonna be a little bit of anecdotal. It's gonna be a little bit of real-life example so bear with me as I try to explain things in tiny piecemeal of things that are much bigger. I'm gonna talk about what we do, what we've learned and how it's worked for our team. It's gonna be a little bit abstract in parts but hopefully my examples will explain themselves. Before I go into too much detail, I'm Stephanie. I work at Amazylabs as a project manager. We are based out of Austin, Texas. Hello. Some of you might also know me because I used to be Amanda from the Drupal Association running DrupalCon. So if my name sounds familiar, that's probably why. All right, so a bit about Amazylabs. We are an agency that started in Zurich, Switzerland and we are now located in Austin, Texas and in Cape Town, South Africa. So we're a little bit all over the board. When people ask how we pick these locations, I joke that we just took some darts, threw them on the board. It's kind of what it looks like. Recently, as of last Sunday, actually, we all got back from Amazy 10, which was the Amazylabs 10-year reunion celebration that we had down in Cape Town, South Africa. This is a bus of our 50 closest Amazy buddies and it was a really great time for all of us to get together and see the faces that we associate with Slack names during the week and just get an idea of what people are like in real life. It was really great. This is sort of the culmination of what I will be talking about. So it's a little bit Tarantino'd. You're seeing the end at the beginning. So a little bit about Amazy Scrum as it once was. Amazylabs is located again in three different locations and with this comes a couple problems. We tried to be kind of a uniform team so not just located and distributed but also working together and independently and this causes quite a few problems because time zones, locations, cultural differences, work style differences, Zurich started first and so they had their own kind of established rhythms and rules, Austin started and sort of did their own little Wild West approach to all the things and then Cape Town showed up and they just wanted to play. One of the problems that we had was that each team sort of worked independently and each team worked with a project manager and a tech lead at the beginning and this caused a lot of problems. So what would happen is all that information was being bottlenecked in those two people and it wasn't getting to the team until much later. So when we would approach projects, we would have issues with specification coming from one person so the team couldn't actually begin work until tech lead has said here's what we're gonna do, here's how we're gonna build, here's the list of all the things that you need to configure in Drupal, go ahead and get going. So that caused one bit of concern. Another point is that it caused a single point of failure. That person who was the tech lead could never go on vacation because if they went on vacation, everything on this side broke, which is a bummer. Same thing with the project manager, if they weren't there to speak to the client and the client message never made it to the developers and nothing happened. So tons of inefficiencies there as well. The other problem is that the tech lead and the project manager had a great relationship with the client but the team who was actually implementing it did not. So there was no tribal knowledge, nothing to pass down and if you passed a project to another group of people in the team, it was either no knowledge was passed and so you were looking at this build and saying I wonder why they did that and talking to 17 different people to figure out who was actually responsible for making this do that and that was incredibly inefficient as well. The other thing is that when we would finally pass a project to someone else in the team, it took so long to get that person up to speed of what was happening that it was almost not worth it to pass the project on. So these were all the little things that were problems. Another problem with the way that we used to work is that we tried using Scrum, we thought we were implementing Scrum, we were quite proud of ourselves but we didn't really have a baseline for how we would approach projects. So we went from saying okay well right now we have a tech lead and a project manager and let's just do Scrum instead. We didn't really do any training, we didn't really bring the whole team up to speed, we just said here's a project, now estimate how long that project is gonna take. The team kind of looked at each other and we were like seven, I don't know. So that was very problematic obviously because we didn't know how to communicate things with the client because we were all like do we do it in story points? I don't know what a story point is, people keep saying things about t-shirt sizes, I just know how many hours I think it's gonna take and so there was a lot of disconnect there as well. Another problem with the way that we Scrum'd was that we had multiple clients, we don't have the luxury currently of having just one giant dedicated mega client who allows us to do Scrum in the way that Scrum should be done. And so we had clients who we would do Scrum with where you have the Scrum ritual of a start and a stop, but we started and stopped with each of the client on a different day and so what that ended up doing is creating a sprint start almost every day of the week and a sprint end almost every day of the week which made the team very sad. It also made it really difficult for planning because the clients that we had at the beginning of the week got obviously the bulk of our time and the clients who came at the very end of the week we just got really good at apologizing too. Sorry, you get like three, I don't know. So that didn't work, obviously not a great idea. But to us in the moment, we thought we were doing Scrum okay, the clients were happy, we got projects done, nothing caught fire but looking back on it, it's a little bit like your 20s where you realize that maybe it wasn't so great and maybe you were sitting on the floor of your house because you didn't have a couch and maybe you were using a Frisbee as a bowl so it was time to change. We now had these three companies, we were working on improving our brand, improving our team happiness, improving our client happiness and we figured it was time to do something. So at the beginning of January, 2016, management got together and said it's time to change. So what we did is we brought in Superman, no we brought in a professional. So someone who understood Scrum and could explain Scrum to our teams and actually give us the proper training to become an agile team. The good thing about this is that the management didn't just say surprise, new thing is happening, they actually said surprise, new thing is potentially happening, we're gonna approach it in a very tiny way, we're gonna see if it works and if it works then we're gonna move towards it but we're not just gonna surprise you and expect you to get it right away. So what they did is they found a client who was open to learning agile as we learned agile which worked out really well. It's kind of hard if you have a client who is very set in their waterfall ways and wants to continue in their waterfall ways to say, hey, we wanna do training wheels with you, how does it make you feel? Especially the Swiss clients, like teaching the Swiss clients how agile works was not a Swiss thing to do. So fortunately it worked, the client was happy, the team that they used was very happy and all the teams after having seen this said, okay, cool, we're bought in, we wanna be part of this new experiment with you, let's do this. So Michael Schmidt, our CTO put on his training hat and then flew to all of the different locations and brought the good word of agile workflows to the colonies and it worked out quite well. One of the things that was a big change is that we went from having teams by location to having teams within a location. So Zurich had enough people, I think they had 24 at the time to actually split into two different teams. Previously their standup was 24 people standing in a circle and doing rounds which took half an hour, surprise. So they split that and so each team became dedicated and each team named themselves and each team became used to having their own independent standups that took obviously less than half an hour every day. One of the biggest things about learning scrum that we had to digest was that we thought we knew how things worked. As I said before, we made projects happen. People were happy with our projects. The teams were happy-ish with our projects. So we just had to take all the information that we thought we knew about our workflow and just sort of had to throw it away. And that was both refreshing and terrifying. So when we all went in there after this new rebirth, it was a bit of a baby step for all of us. So out of this, quite a few things changed and that's what I want the rest of this talk to be about. I didn't want to focus on all the negatives and I wanted to focus instead on things that you guys could also do or potentially learn from for your companies as well. So some of the things that we changed is that we restructured meetings and I'll go over that in a bit. We involved the team in all of the things, maybe to the point where they're not so thrilled with it but it works out for the best. We added a Scrum Master. We enforced story points rather than estimation by hours and we really, really harp on team testing. Super, super important. And it sounds like a lot to be like, these things are new and different and it totally was. But at the end, I think that it worked out. The other thing is that we kind of played it forward. We didn't just surprise our clients and say, hey, this is the new way of doing things. Be on board or don't. We approached it the same way of, we're gonna do this new thing. We would really like it if you were on board with us. Are you on board with us? And they were. So after we learned the Scrum ways, we went to our clients and said, now it's time for you to learn the Scrum ways and they learned Scrum with us, which I think worked really well. Had we come to them and said, surprise, here's a new way of doing things, I think it would have ended a lot differently. But in the process of doing it, we didn't just come to the clients and say, hey, there's a new, strange, new, fun thing, let's do it. We shared with them, like here's some of the pain points of why starting and stopping sprints on different days are not good for you. Here's why this thing is not good for you. Here's what you'll benefit from by joining us in this change. And after the second demo meeting, I think the clients start to get at it. We started our Scrum rituals early and we've stuck to those Scrum rituals since and since implementing this, existing and new clients have all kind of helped hit that rhythm with us. So one of the biggest changes is we changed all of our client meetings from being kind of scattered all over the week to now all starting and stopping or starting on the same day. So for us, this day is Tuesday. So we call it demo day hell because this is the day that we're gonna be in meetings, we're gonna be talking with clients, we're gonna be demoing things or we'll be doing backlog grooming. So Tuesday is just kind of dedicated as you're probably not gonna get a lot of work done but make sure that your brain is turned on. So it worked. The other thing that we did about this, again, keeping in mind that we have two other offices is that we synced our Scrum, our sprint weeks so that we all start and stop at the same time. I believe one of our Zerk teams starts on Tuesday while we start on Wednesday. But for the most part, our two-week cadence is all the same time so we can actually pass tickets back and forth but I'll get into that also later. So this is now what our sprint week looks like. It looks like there's a lot of things on there and there are. But keep in mind before this was being done multiple times all throughout the week because we had multiple clients all throughout the week that we were doing this for. So now we actually had dedicated times and client scheduled meetings to perform all these things in. So while it looks on paper like it's very complicated and time-consuming, it's actually more efficient than the way we were doing it before. So I'll just go through these really quick. So I don't have header labels on this because I don't want Monday, Tuesday, Wednesday to influence what you guys do. It's really how your team works best but for the sake of the two-week sprint, here's what we do. So on Wednesday morning, we start the sprint. Immediately afterwards, we do ticket planning and I'm the project manager on the team so this means I walk away and I leave it up to the developers to huddle and go through the tickets and figure out between themselves how we're going to approach this week. To cut ahead a little bit, they're also part of the sprint planning so they shouldn't be surprised by the tickets that they see so this is more of a technical like, okay, if we're gonna do this, this and this, we need to start over here with this ticket and this client wants to do some kind of a launch by Friday but not on Friday so we need to make sure that this ticket is done a little bit earlier and it's their way of kind of getting ahead of the work before the work actually snowballs. The other thing is that we do daily standup but if you notice daily standup is not on every day. We don't do daily standup on the same day that we do sprint start because the dev team sitting around talking about what they're gonna do kind of already is that. Again, on the Thursday following and I'll get to that later, that's also a super busy day and so people kind of know what they're gonna be doing on those days. So daily standup, Thursday, Friday and Monday and then Tuesday is client day so we do backlog grooming. We do not do backlog grooming with all of our clients. We only do backlog grooming with clients who have a project that's active enough to actually meet so some clients we meet every week, some clients we meet every other week. They're cool with it, we're cool with it. It's very clear about when the meetings are gonna happen. On Wednesday, just like we do with Tuesday, every Wednesday is a meeting for the sprint. So the alternate Wednesday is sprint estimation. We take all those backlog groomed tickets. We do estimation. This gives us a little bit of a barrier before we start the next sprint to ask questions, see if any of the technology has changed, make sure we're on the same page with the client and with the team before we start the next sprint. Another thing that we do is on Thursday before the sprint ends, we do a board review. So we pull up the board and say, is there anything on this board that scares us or won't get finished or is blocked or whatever. It's a little bit different than the daily because this is like the panic final review before the weekend. And then Tuesday, that's the last day of our sprint. We meet, we do the retrospective. The retrospective we do as a team we don't do it with the client. And this is our chance to sit down, talk about the week, things that we like, things that we didn't like, things we want to do better for next time and to get those things all laid out. So the cool thing about the way that we've implemented Scrum is that each of our teams do the retrospective differently. So our team takes a Google Doc because everyone can write at the same time. We have columns, what we liked, what we didn't like, what we want to change and then each person spends five minutes just filling out their name under their section and then we review it. We have a team in Zurich who takes a Google drawing and they use GIFs and drawn things and smiley faces on the spectrum of smiley to frowny to indicate how they feel. And for them, that's good enough. And so that's just one of those examples of how we've all embraced Scrum but how we all kind of flavor it differently. So it works for the team. Another thing that has changed is that everyone attends meetings. So before you had the project manager and the tech lead attending the meetings and now on big projects, everyone who's going to be working on that project attends the meetings. And the reason that we do this is because we were losing a lot in the game of telephone so tech lead and project manager would go. They would come back and relay the good word from the client. Team would say, but what about this, that and the other thing? And the PM and the tech lead would have to go back to the client and things would just get lost in the middle. It would take too long. The team who was implementing it didn't quite get invested in the project because they were just being handed tickets to do, which was fine, but it's just, it's better overall if the client, if the team can feel like they are a part of it and have some choice in the matter. So this has actually helped us with a lot of things. Before when it was just me and the tech lead meeting with the client, or sometimes just me meeting with the client and the client would ask a question, I would sit there and think, should I respond to this? Or should I say, I have to go ask the team because I don't really know this because I'm not a developer. And that kind of pause sometimes doesn't help with building trust with the client. And so it's actually nicer to have someone there who's experienced with that, who can actually say, yeah, I know the answer to this or no, I don't know the answer to this, but I'll go find out and let you know. Constantly having to be the person who says, yeah, I don't really know the answer to that just sort of makes me look not efficient, which I think is a good improvement. Because I want to be there to do budgets and keeping the team happy and all the things that the project manager should do, not the, I'm going to come back to you later. The other thing is that technical ears hear things differently than non-technical ears. So I might hear from the client that they want to have something that moves on the homepage. And to me that means, oh, a slider, a slideshow. But to the tech team who might understand this differently, it might be, I want to have an animation. And so having the tech team there to actually interpret these things makes it so that there's a lot less left on the table. Another thing that we learned how to do better, remember that bull of M&Ms from before, is that we learned how to estimate as a team. We learned how to take a pile of tickets and understand the amount of work that we expect to go into them and then also how much of those tickets we're able to deliver at the end of the sprint. So this actually helped us continue to build trust with the client because we would say, hey, we're going to get these four tickets done for you because they're X many points. And then at the end of the sprint, we're actually able to deliver them because we're used to being able to produce this many story points in a sprint. So the good thing is, is that we figured it out. The hard part is, is that it was really difficult because going from t-shirt sizes, hours, whatever, and then everybody's own interpretation of how long it takes to get something done. Like if you ask a friend and person how long it's going to take to do a backend task, you're going to get a little bit of a mismatch. But over time and after doing a lot of estimation as a group, we've all sort of hit the rhythm. And as we do that, it becomes sort of a game. Like how many times can we all hit the same number during story poker at the same time? So we've turned that from something kind of painful into something kind of fun. So one of the ways that we learned how to estimate together is that we, what we call estimating in ANAs. So if you look at the Amazing Labs website, you can see that there is a person page for each of the employees. So each of the person pages looks like this, person sitting on the couch and then a description about them below. The way that we started our estimation was to say, how long do you think it's going to take to implement this? And at the beginning, when we were learning how to estimate, we were not fast. So these estimation meetings probably were about two hours until we kind of got the hang of it and now they're closer to half an hour or an hour, but that first month or so was pretty painful. So the way that we figured this out was to figure out what it was that we needed to do on this page and all the different elements that went into it. So it wasn't just how do we do this, we were actually talking out how to write the acceptance criteria for a ticket at the same time. So if you're looking at this, you've got a hero image, you've got an H1, ignore the top thing because the header is already part of the website. And so we started to learn to talk through the different elements on a page and not just say, give me this thing, I'm gonna take it and then when I'm done building, I'll throw it over to the front end person. Now we look at it as in order to do this, we need to be able to do all these things, have it tested and out the door by the end of the sprint. So it was a little bit different way of looking at how you would approach new work. Another thing that changed is that we took on writing tickets, but like I mentioned before, we do scrum, but we kind of do scrum in our own flavor. Our team decided that writing stories, user stories wasn't the best way for us to understand and interpret tickets. So you've probably seen the phrasing of blank so that blank, whatever. For us that was too up in the air and it left too much open to be decided and at the end of the day the developers were like, but I don't know what to do and I'm glad that you're giving me this freedom but I would rather be told some more structuring and so based on that, we structured our tickets a little bit differently. So now if you see one of our tickets and we'll go over the tickets later, what you'll see instead is the task kind of short-handed up in the title and then acceptance criteria that helps lay out what this is. In the acceptance criteria or above the acceptance criteria, what we'll do is we'll try to figure out what the business value is, what we're hoping to accomplish and what must be part of delivery which is essentially a user story but written in words that we like. So a little bit about backlog grooming that might be a little bit different than how you guys implement scrum is anyone can make tickets. And when I presented this before at Bad Camp there was like a shock and awe happening in the crowd. The reason that we do this is because if you're a developer in there and something breaks or you notice a thing and you wanna flag it, you might be able to articulate that thing better than coming back to me and saying, hey Steph, this thing happened. Now I need you to make it be not technical. Go do that. So now if a developer finds something, they'll make a ticket, they say, hey Steph, I made a ticket and I'll be like, great, cool, let's add it to the backlog. We can figure out if it's an on fire ticket or if it's a great to have ticket or if it's just a ticket that if we don't end up fixing by the end of the story or by the end of the sprint or by the end of launch, we as a team will feel like we failed. So it's just a nice way of being able to bubble up problems or tasks in real time rather than having to create a parking lot task list at the end. The other thing that we learned to do is clear acceptance criteria. So even if a client writes the ticket, it doesn't mean that the team cannot go in and edit the ticket. What we try to do in that situation is to make a comment that says, hey, I'm changing this acceptance criteria because the way that you've written it makes it sound like we should launch on the moon but I don't think that's really what you meant. So we're gonna change it to something that we think is a little more realistic. The other thing that we do is a shared definition of done. We went through the exercise of figuring out what those done were and we're happy to realize that those are not gonna be things that can last on every ticket. It just, it becomes too burdensome and heavy to have it be part of every ticket. So some examples of definitions of done items are browser testing, functional testing, tech review testing, code review testing. You're not gonna do a code review on a design. So it's a little bit unnecessary to have all of those things applied to all tickets all the time. So when we do a ticket testing or when we do a ticket, we make sure to kind of outline what those done's are gonna be. But as we go through the process of doing tickets week after week, we get a shared vocabulary of what kinds of things we know are gonna be expected out of a ticket. So for example, if we have a design ticket, that design never sees the client until the tech team has seen it. Because we don't wanna say, hey, a client, here's this pretty thing. And then after the client says, yeah, I totally want that thing, the tech team goes there, there's no way that's gonna happen. So we're getting a lot better at communicating early and being able to present things that the team is already comfortable with so that the client doesn't get shiny things taken away from them. The other thing is that we've enforced the freedom to split tickets. So for example, back to that Anna example, if we find out that the Anna picture needs to rotate into a different picture halfway through someone being on that page or some kind of thing that's not really clear when you look at it, we have the ability to say, okay, we're gonna deliver that piece with Anna as the header, but the animation will come in a second ticket. So being able to go through and split the ticket so that it's more manageable and can still deliver something usable, but that still identifies that there's more to be done later. The other thing is reinforcing prioritizing and it's not just for the client that we do this, it's also for the team because our team loves making shiny, pretty beautiful things. And sometimes we see a thing and it makes us sad in our heart to not be able to focus all of our attention on it immediately because other things need to happen. And so again, creating tickets allows us to say, I realize that this font is not the ideal font. Maybe if there's time at the end of the project, we can just get that done and maybe we'll just eat the cost of getting it done because it's not broken. The client thinks it's great. I don't think it's great. So now that we've talked about some processes with clients and with the teams, I'm gonna get into some of the juicy technicals. And this is probably where you're gonna be sad about not being able to see clearly some of the examples, apologies in advance. For those of you who don't have JIRA, just maybe pull out your phones for a few minutes. For those of you who do, I'll be able to show you what the guts of this look like afterwards if you'd like. So this is our backlog. Hello. And I'm just gonna kind of go over how some of it works. This obviously isn't the whole thing, but this is the digestible part of our backlog. So it's broken up into three sections, the current sprint, the upcoming sprint, and the backlog. So the current sprint, as I mentioned, we have four different teams in amazing labs, kind of all over the world. Each team has named themselves. Our Austin team is Scrum and Coke, which yes, we think is super clever. And this is, as you'll notice, multiple projects being listed in here. So one of the big changes that we made was to take all of our client boards and merge them into one mega board, which we're very excited about. On this, you can see a couple things that are useful. You'll see there's epics listed. There's people assigned to tickets. There's story points. There's a priority in there, even though you can't tell that there's a priority in there of things that need to get done. And then you have the upcoming sprint. So we handle this upcoming sprint probably a little bit differently than some of you would. I know that Scrum tells us that we should be in real time doing all the things. We're still people and we are still a little bit OCD and so we'd like to know kind of what's coming up next sprint. And so depending on the project, we might actually have one or two upcoming sprints listed out just to help us keep track of the tickets and the order that they come in. So what you're seeing here is once the sprint is done, in theory, we're gonna tackle this next sprint in this order. And this sort of help keeps track of stuff. This doesn't mean that what's happening on here will happen. What sometimes happens is that maybe green 59 will end up in the prioritized for sprint multiple times until we get tired of seeing it and then go send it to backlog land where it will live forever until the project is over. And then we look back on it and say, hey, remember that one time, that that one ticket was there? We don't think it was so important. Yep, so another thing about this is that we try to implement priorities with the client. But there comes times when the priority won't be able to be implemented because for example, if the client would like there to be a tire first and we need to make sure that there's an engine actually because further down the engine's gonna be more critical, we'll take the priority and actually change it up. The priority with the client really comes into play when we are discussing, do you want it to happen this sprint or next sprint? Once it's in the sprint, we sort of take control of the tickets themselves. Again, the backlog. Another cool thing about this is, again, we've got four different scrum groups going on at any given time. And so we have this cool quick tabs across the top. And so depending on which project that you're working on, you can just click on the quick tab, it'll just show those tickets. But if you're in a different project team, I don't want every other project to show on my board because that would make everyone super, super sad. So instead, we've implemented this thing called, we've implemented labels and added that to our JIRA search string. And so now if Zurich says, hey, I want you to work on this project, all they have to do is add ATX to their ticket and it will show in our backlog. So it makes it really, really easy to pass tickets between the teams without overburdening each of the teams with all of the projects that exist in the AMAZY ecosphere. Another thing on here that you'll notice is, so obviously I've hidden the client names and changed them out with colors, but the ones that I haven't changed are actual internal projects. So one of the things that we do to sort of envelope scrum into all the things that we do is we incorporate internal stuff, group-wide stuff and client stuff. So on this board, you're actually seeing internal work. I think DrupalCon and client work and Zurich work. So it should be a board of doom, but it's actually quite nice. One thing also to mention is in the scrum, in the sprint, we of course take in tickets that we think we're gonna tackle, but sometimes things happen mid-sprint. So the client will turn their website off or I don't know. But depending on what the issue is from the client, we'll take that, make it into a ticket, and I'll check with a developer and say, hey, you got three seconds, I need you to look at this ticket. This thing happened, do we think that it's critical or do we think that it's a super easy fix? And depending on the criticalness or the easiness of fixing it, we'll sometimes take in smaller tickets or super critical tickets into the sprint. But for the most part, we've gotten pretty good with our clients of saying, is this broken and will you die if we don't fix it? Or is this broken and can you wait until next sprint to incorporate it into next week's sprint? And for the most part, they're pretty cool with that. All right, so that was Jira, I blew through Jira pretty quickly, but I'll mention a couple other things I believe later on. Retrospectives, so this is one of the things in the relearning of how to scrum that I think the team struggled with the most. At first, we were like, I don't really get it. We're obviously talking to each other every day at the daily stand-ups. We all kind of are aware of what's going on, right? Surprise, when we actually started doing the retrospectives and being honest in the retrospectives, we realized, oh, these are useful, let's continue doing them. So I'm gonna share a little bit of our learning from the retrospectives. The first thing that we learned is that the retrospective is not a stitch and bitch. We have an hour dedicated for the team to talk about how that week was. If the entire time is dedicated to just moaning, then everyone will feel like it was a waste of time and feel very negatively towards doing the retrospective. And so by enforcing those three columns of what did we like, what did we not like, and how can we fix it, that actually forces us to say, yeah, this thing really, really sucked, and here's how we're gonna do better next time. And so going through the process of actually verbalizing these things have actually helped the team feel like they're actually beneficial. So just kind of embracing that as a useful thing has helped change our mindset about the retrospective. The other things that we've learned out of the retrospective is things like when you have a ticket and you are done with that ticket, you put it into testing. And previously, when tickets went into testing, they kind of went there to die. And that was really lame because you would get to the end of the sprint and no one had bothered testing any of the tickets and suddenly it's Friday and everyone realizes that they have a day to test and fix anything in the tickets and that ended up stressing everybody out and everyone felt like it was maybe someone else's fault or maybe the PM's fault for not testing everything. And so realizing that testing was important early and often, we embraced the mindset of treating tickets like a hot potato, that's a potato. So in the morning during daily standup, someone either mentions, hey, this thing went into testing yesterday, can someone test it? Or we use Slack to say, hey, this one's ready for testing, someone go and grab it. And we have now a process of as soon as a dev person is done touching a dev ticket, a dev person will pick up that ticket and then test it. And then depending on whether or not we need design work, then our designer will go and pick it up. And then once that's all done, I will pick it up and try to break it. So getting that done early and often and then sending it to me saves everyone the heartache of having me break it early but late. Another thing is that story points are gonna mean different things to different people. We once mentioned to our client who does Grum that a 13 by Thursday is one of the goals that we have. And he looked at me and said, there's no way that we could do a 13 in a single sprint. So 13 obviously means different things to different people but for us, a 13 would be a pretty big implementation that we could get done during a single two week period. For us, when we do that Thursday board review, if that 13 is not in testing by the end of the day, we have failed. So that's one of our goals, get that 13 out the door so that we can test it and find all the bugs associated with it and get that ready for client demo. The other thing is testing. So I'm gonna repeat this again because we learned it the hard way, test all your things. Of course we're gonna miss stuff because that's just what happens when there's all the moving parts of a website but if you test, you shouldn't be too surprised by really big things. So some of the things that we've implemented is code review. Surprisingly, we didn't always do code review at the beginning. Sometimes we would do code review at the very end of a project and then you would be surprised and sad. We would now do white box testing which means Stephanie goes in there knowing what the functionality should be and then try to break it, which is fun. Including deployment instructions which seems like a yeah, you should totally have that was actually not something that we had written down as a you must have. So now all testing includes deployment instructions and testing instructions if the thing that you're testing isn't super clear. And then also browser testing defining what the browsers and the mobile devices are in the ticket because not every client has the same requirements. So making sure that we have those queued up to go. Always part of it. Just a quick view of our Jira board. This is what I talked about again. So if in the middle of you see that we've got internal testing and review. So again, like a design ticket, you would really test the design ticket but you would review a ticket. And with the client, depending on the client, not all of our clients are technical. And so that column just becomes demo because not all of them are gonna go in and do actual client testing. We do have clients who love to do testing and so they kind of go into that same column. So internal things that we haven't started yet. In progress, things that we're working on actively. And what you'll see in here is that there's no two faces. No one's face is on two tickets while in progress. Unless you have like two laptops open and you're actively doing something, it's impossible for you to do two things at once. In theory, so we try to encourage someone to not have their base on two tickets in progress. And that also helps it so that if something needs to get done, like keep it on the to do side and leave it blank so that if someone finishes their work on another ticket, they can go grab that. Otherwise, if you have your face on two things, like people are gonna think that they're being taken care of and in actuality, like ticket B might not be touched at all. That just sort of helps kind of keep everything together. And then one of the things that we do but I'm not really gonna go in here is that we try to keep deployments out of the same sprint that the work is being done unless we've communicated directly with the client and said, hey, this ticket must be deployed as part of this sprint. We try to keep them separate. So what we'll do with the end of a sprint is we'll, it'll go through the motions of going through client testing. And then after the demo is over, we'll create a new ticket that says deploy this ticket. And depending on how many tickets that we have for that project, it'll say deployment for a sprint, whatever the sprint is. And then it'll list all the tickets for that project that are associated with the upcoming sprint. So this way it makes it so that the client has time between the demo and the deployment to actually do some heavy testing. And so that we're not rushing it like Tuesday at two o'clock to like deploy all the things really quickly and hope nothing breaks. So it just sort of helps meter out the work and it meters it out so that the client's not on fire and that the teams aren't on fire. And it's actually helped us get a better control on deployments in general. So along with this team communication changed. I didn't think it was going to. I thought our communication was actually going pretty well, but the things that we're gonna go over are some of the things that were surprises to me. So one of the things is that now that we're in a more all ticket all the time mentality, Slack actually became a great way of passing those tickets to each other. So our DevOps people that be have actually integrated Jira with Slack. And so if you mention a ticket in Slack, the ticket will appear in Slack. And it's really easy to say like, hey, I'm looking at yellow 52, please go take a look at this. And then the team uses that to just pass tickets back and forth or say, hey, I have a question about this ticket. What do I do? And that way like email becomes a place of real business and not just a place where information goes to die. Another thing that we do is that each of our Slack, each of our projects has its own dedicated Slack channel, but depending on the client, our clients sometimes would love to be in Slack. And so they are in Slack with us in that channel, actively going through tickets with us and responding to questions. Or they're in Slack and maybe we don't feel comfortable having some of the conversations about the project with the client there. And so we've got two. And we just labeled that our deployment one. So in all ways, we're still allowed to have communication with the team and we're still allowed to have communication with the client, but we build ourselves like safety barriers so that everyone is comfortable with what's going on. Let's see. The other thing that we got really good at is making comments. And again, this feels like a duh. But one of the things that we would go through is that a dev would pass a ticket to a dev and maybe in Slack they would say, yeah, good job, looks great. But in the ticket, there would be no history of any communication at all. So you would go from having a ticket being worked on and maybe some deployment instructions and then suddenly I would come in and be like browser test, boom. And there would be no history of anything that happened between, start at like point A and point B. And so that was problematic for a variety of reasons. For example, if you close a ticket and two months later have to go back and look at that ticket, you go, oh, I guess this magically happened and everyone was super happy about it except for it broke. So now we're super good about going in and saying, I have a question, I want you to review this, I need some feedback and then documenting that. Even if we have a meeting to talk about something, someone is responsible for taking the notes or the outcome of that meeting and documenting it. That way it lives here. As a result, we also use confluence, but as a result of getting better at using the comments, we've stopped putting so much in confluence that's real time and using it more for just client meetings and like dev environment information. So it's very static and JIRA on the other hand is very vibrant. Another thing that we've implemented is more meetings, but in a good way. So now meetings don't suck. It's not always me as project manager calling the meetings if one of our devs has a question that we either pull in all the other devs or they'll grab the designer or they'll grab me or we'll grab the clients and do an ad hoc meeting. And we actually encourage these now and people actively do them on their own because now they feel more in control of what's actually being discussed and they're not just sitting in a two-hour meeting trying really hard not to touch Facebook. The result of this is that devs are happier, clients are happier because they feel like they're more involved. We're not just seeing them two times a week and then just being like, hope you're surprised, here's the thing that we made. They feel like they're more actively engaged in what's being built. Another thing is that because we are a remote team and the team Austin also works from home on Fridays everything that we do has to be treated as though you're not gonna see the person that you're talking to in real life. So we've gotten better at doing remote collaboration and this can mean a variety of things. So one, if you're gonna have a Slack discussion and the Slack discussion is gonna become a book call that person. Cause like everyone's super fine with typing a book but the other person maybe doesn't like reading the book and things get lost in just text barf. So get on a Slack call. We have Zoom integrated into Slack. I feel like I'm pitching a bunch of products but it keeps us integrated with have all of these different tools kind of talking together. So I ping one of my devs, I say I've got a question. This thing looks like, oh wait, hold on, let's have a call. I'll do slash zoom, start the call. Five seconds later we both know what's happening and we both can walk away like knowing that the ticket is under control. One of the other things that we as a PM have gotten really a lot better at is stop emailing your clients, which sounds kind of funny. Pick up the phone if there's a problem. Pick up the phone if you need something right away and maybe your client is prone to travel or you know they get a lot of email in their inbox because even though you as a team are involved in the project all the time your client isn't part of your team in the fact that they have their own business happening. They've got their own people they're reporting to and they've got their own things they have to do every day. They're not in the project all the time, most of the time. For you to think that they're gonna be available all the time is just not realistic. Give them a call, get on a zoom with them, put a meeting on their calendar if you know that they're super busy but just constantly have communication open with your client. If they're in Slack and they're active great this maybe doesn't apply but if they're clients who maybe need a little bit more coaxing or are more likely to say yeah just go figure it out like whatever, get them on the phone call. It's a lot easier, it's a lot harder for them to not answer the phone or to not answer your question if you do that. So now we're getting into the fun stuff and again I can talk about this offline cause it's a little bit difficult to take like a 40 page document and put it on a screen slide. But we also started talking about Scrum with clients going into new projects so we didn't want to just surprise them by sending out proposals in a waterfall way and then when the project kickoffs be like surprise Scrum. So now we have it in our proposals of how we do things. We include our project management approach. We include estimations and we enforce almost to the point of being annoying. This is an estimation. Here's the definition of an estimation. It totally is not a promise. We're not gonna get you 75 things for X number of monies because things change. And we reiterate it in different ways so that it's not just if you change a thing I will charge you more. We try to make it clear so that projects are a living thing. If you change this it might not make the budget change but we might need to get some things off the table in order to deliver by the time that you need it. So reinforcing the same idea but in different ways throughout the proposal just reinforces over and over again that Scrum is not just one part of how we do it it's the entire how we do it. The other thing that we do is we outline specifically that the client will pay for Scrum rituals and surprisingly people who are new to AMAZE are pretty cool with that. Like if we're clear which is one of the things that we try to do in all communications with clients if we're transparent and say here's gonna be a thing that we do they're cool with it. If we surprise them with the invoice and say here's a line item that we didn't talk about then they get sad. Who pays for Scrum? So you saw our JIRA board before and I mentioned that there were not just client things on the board. Because of that it's not just clients who pay for Scrum. We acknowledge that we're gonna have other things going on and so it doesn't make sense for the clients to bear the burden of us doing all of our Scrum rituals. So this is a quick and dirty example. At the end of every month we do a Scrum report. As you can see there's three sprints listed at the top depending on the month sometimes a Sprint will fall kind of between a month so we kind of have to fake it a little bit. My favorite months are the ones where the two Sprint cycles land perfectly between the first and the 31st doesn't always happen. But what this is is it's math that hopefully doesn't have a typo in there. But what it is is we take all the story points that were completed during the Sprint. We don't count ones that were started or in progress or anything like that. This is only completed tickets. We calculate them so Team Ticket Pink had 12 story points over the course of these three Sprints. In the number of things that was 2% of the Sprint so they're gonna carry 2% of the burden of our logged hours to these events. So to go back a couple of slides we had daily stand up, we had Sprint planning, we had Sprint estimation, we have Sprint retrospective all of these things that the entire team builds against. So in JIRA we'll log, spend an hour in Scrum estimation, spent 10 minutes in daily stand up. All of these hours are logged and they go towards the calculation. So one of the things I didn't show here was just the count. So like in a typical Sprint or in a typical month we might do between 30 and 40 hours in Scrum rituals which is a lot. And again it's also a lot to put on just clients. So this number will say, okay you had 47 hours logged to these Scrum rituals. Now let's see how much is actually billable. And so in the adjusted column what I'll do is I'll take out all the internal work. So I'm not gonna obviously charge myself for them but I'll just remove it so that only the things that we're actually gonna pass on to the clients show in the calculation. So for this one Team Yellow they had 3.56 hours of time and that shows in the work log, the JIRA work log that I pull for all my clients as a SNC strategy and planning. We don't write out Scrum and Coke because not everyone thinks it's funny. But that's how it looks. They'll see it as a line item in their work log. All right, so my takeaways. These are just hopefully recaps of things that we already mentioned. The team as a whole is way more efficient. Quick plug for AMAZ-IO getting the team onto new projects is super easy. It used to take half a day to get a developer onboarded onto a project just from a technical perspective. Now it takes an hour. So that really helps. Clear communication and clear expectations both for the team and for the client makes everyone a lot happier. You go in knowing what's expected. You come out hopefully on the same page of marching orders. We have better testing which means the client gets a better product and it means at the end of the day we feel much more pride in the products that we're delivering. The team is really confident. The client's really happy and the team is super invested in the final project. One of the things I didn't put in here was the team launches. So every time we do a website launch the team kind of throws a little party because for us it's a really big deal. We spent a bunch of time building this thing. We're really excited about it and so we're gonna celebrate. So now all of the AMAZ-Is are on the same scrum cycle. We're all doing scrum properly now instead of our own broken variations of it and we're all really happy with the variations that we're using that are all working. We can pass tickets between each other and communication and relationships with all the other offices have actually improved. And in conclusion, everyone's super happy. We all have happily ever after. Does anybody have any questions? Cool, I think there's a mic if you wanna go to the mic so they can record what the question's gonna be. I alluded in passing and then moved on about how it was really hard and painful to convert the team's two story points. And I don't know if you can expand on that a little bit because on our teams, when I'm getting from the devs, they don't really see a use in that and they're like, I don't understand story points. I just know how long I think it's gonna take me to do something. So how did you kinda work through that? So one of the hard things for us is that our team is really small. We're three developers, one designer and me. And so to have me sit there and maybe give a story point estimate for how long it's gonna take to build a skyscraper isn't maybe the most efficient. But what it does is it kind of gives you a shared language so that when things come through you all start to grow a shared understanding of how much time certain things take to do. We do acknowledge that ours do play a part in story points but we also acknowledge that it's not just the getting those people in the room or getting that person in the room to actually deliver the product. Sometimes it's a matter of how many different pieces are we gonna need to pull in in order to get this done? Are we gonna be able to do this all internally? Are we gonna have to go find someone else? Do we need to have a third vendor come in and do something? And just having those discussions of what we think the ticket means is helpful. Does that answer the question? Okay. Just in case it didn't know, I just wanna drill down more into that same question as it were. And so there's always this tension in Scrum where it's like this ideal world of Agile where we just push business objectives into work and then let the developers figure out the implementation. We're trying to get away from telling the developers to say, do this so that they just do it and it's done. And then later the client says, well, it's actually broken and the developers say, well, this is what you told us to do. Right, we're trying to get away from that, right? And so we wanna leave that stuff open but then you have the other tension of saying, we actually wanna get the stuff done, right? So we wanna know what it is we're doing before we go into work, right? So there's those kind of conflicting goals. And estimating is kind of the ground where all that kind of the tension actually comes out. So if you're estimating hours for the purpose of like saying, how long is this actually gonna get done? Or so really the question is, can you really talk into this question of what you are trying to get done when you are going through the process, particularly like in the backlog refinement, especially, but then also in sprint planning. In your mind, what are you actually trying to accomplish when you're talking about estimates? What we're trying to really do is figure out how complicated something is. Because if we identify all the moving parts of delivering a ticket and we realize that we don't know how to do a ticket, we'll estimate it extra high. So if we're handed a ticket that says I want a Ferris wheel, we're like, well I think I know what a Ferris wheel is and I think I saw something on dribble.org that might be a module for a Ferris wheel. I need to go look at it. We'll estimate that ticket high and at the end, if we're wrong and it's actually a three because the module did all the things we wanted it to, then we'll change that story point from like a 13 to a three. And it's just the matter of having the understanding that we can be wrong when we're doing story points really helps. For the most part, once we're in the project, we have a better rhythm of getting the story points right. But usually it takes a month maybe, like two full cycles of a sprint to realize, yeah, we gotta handle on the project. We understand what a client says when they say X and Y. But for each client, it's different. And I actually do a report where I pull all the hours worked in a sprint against all of the hours, all the story points estimated on a project and come up with an average of how many hours it takes to do those deliverables. And for the most part, like if we have a project that's existed and we kind of figured out and know how it goes, like a story point per hour ratio will be like an hour maybe. And a new project will be somewhere closer to like 2.7. So there's definitely a range. But we do try to stay away from hours because then the developer feels beholden to that hour. So like if they say it's gonna take four hours but they can't do four hours because something exploded or they just didn't understand, then they feel really guilty if they're not able to deliver under four hours. And so we do that to help keep the developer from feeling that guilt and also to just communicate, like, I don't know how to do this. It's super complicated or I know how to do this and it will be super complicated. So I have kind of like a two-part question. So one is I know ScrumRules, you're not supposed to share resources across multiple Scrum teams. We work at an agency and the nature of our work that's like kind of impossible. So I was wondering, do you guys share resources across teams or no? Yes. You do. Okay, so then my follow-up question to that is how do you prioritize the client work then, like internally? Yep. So what we do is we actually treat the PM person on the other team. So we've got another team in Zurich called, no, our Cape Town team is Death by Burgers. And so their PM for their team will act as our client. They will be the client intermediary, go between to us and we'll put them on our Tuesday sprint cadence and say, for example, Victor, you're gonna meet with us on Tuesday, you're gonna be the client, we're gonna go through all the rituals as if you were the client and we're gonna take all of your tickets for this project as if you were a client project and just add them in. So depending on, we won't add them if we're obviously super busy with our own client work, but if we have space, we might say, yeah, we can help you this sprint or we'll take all your information this sprint, but maybe we won't be able to implement until the following sprint or two sprints away and we've done that. We've actually launched maybe like three or four different non-Austin projects in the last, since 2017 started and before that would have caused like a big break in the wheel of things, but now that we have our thing, we just kind of move them in, they're part of the machine and they also understand how it works. So at the end of the day, the client's super happy, they don't know the difference, well, I mean they know that someone else did it, but they don't know the difference in quality or routine so they know what's expected of them and the team is able to just absorb the new project and kick it out like they would something internal. Okay, you said that you heard an Scrum expert, so how was the process and what recommendations can you give us to hire the right one? I actually can't speak to that. It was all done on our Zurich side. If you run into Michael here, he's gonna be doing a session in this room, I think at two, he'll be able to give you more of a rundown on how that worked and what they actually did in the office. We only got the kind of residuals of that. Okay, thank you. Okay, yay, I can talk to you afterwards if you have questions, but we are out of time. Thank you all for coming. Yeah.