 Good afternoon, everybody. Welcome to the last session of the day. Or as some people know it, the nap time. Hopefully we won't put you to sleep. To set the stage for today's presentation, we've borrowed some inspiration from a popular children's book, If You Give a Mouse a Cookie. This was written by Laura Numeroff and illustrated by Felicia Bond. And for those of you who are unfamiliar with the story, here's a brief taste. If you give a mouse a cookie, he's going to ask for a glass of milk. When you give him the milk, he's probably going to ask you for a straw. When he's finished, he'll ask for a napkin. And then he'll want to look in the mirror and make sure he doesn't have a milk mustache. When he looks in the mirror, he might notice that his hair needs a trim. So he'll probably ask for a pair of scissors. When he's finished giving himself a trim, he'll want a broom to sweep up. He'll start sweeping, and he might get carried away and sweep every room in the house. He may even end up washing the floors as well. And when he's done, well, you can probably see where this is going. It's kind of cute, but in the software industry, there's a term for this, scope creep. And scope creep is not cute. In fact, it stings. It hurts because it means you're going to be over budget. Your work is going to take longer than you were originally estimated. And you're likely to miss your launch deadline. And you know what that results in? This. Upset stakeholders, team members, and angry boss, frustration with yourself. So how do we avoid scope creep? In this session, we're going to look at three different perspectives on that question. The perspective of the product owner, or otherwise known as a client. The perspective of a project manager. And the perspective of a developer. Before we dive in, let me tell you a little bit about us. James Mohan, Chris Walker, and myself work at Astonish Design, which is a software development firm here in Austin. In our work together, we've learned a few things about the important and collaborative roles played by clients, product owners, project managers, and developers in the making of great web software. We'd like to share some of those insights with you this afternoon. As I mentioned, we'll focus on the roles of client, in this case, product owner, project manager, and developer. My name is Rob Lee, and I will be sharing the client or product owner's perspective today. To start, I'd like to outline the example scenario we're going to use for today's talk. We thought the, if you give a mouse a cookie, was a good metaphor for the innocent and innocuous way that product scope can begin to creep. You notice in the story, it started with, you know, give him a cookie, he wants some milk, the milk, he wants to check in the mirror, and it's simple. But for further discussion today, we're going to look at another, maybe less silly metaphor. For today's discussion, we're going to imagine that we have a school administrator responsible for a bus full of kids embarking on a museum field trip. Sounds pretty straightforward. The objective is to provide a snack for hungry kids on the museum field trip and keep them happy. So our school administrator has several responsibilities as product owner. They've got to keep the main objective in mind, keeping those kids happy with snacks. But they've also got stakeholders who have concerns and agendas of their own. They've got constraints, often budget and timing are at the top of that list. They also have their own concerns about having an enjoyable field trip and continued employment. They don't want something to go wrong. There are a whole bunch of constraints to look at beyond just budget and timing. A few important ones are the school cafeteria doesn't have the bandwidth to make snacks. They are just focused on school lunches and that's it. So they can't help me out as the school administrator. We're not gonna have refrigeration on the buses so I can't have a snack that involves being kept at a cool temperature. I need to make these snacks in advance, obviously. I've got a limited budget from the school district. In addition, there are a whole bunch of other constraints to be concerned about and you can see some of those outlined on this slide. The point here is that as product owner, you're juggling a whole bunch of priorities. Your key role is to keep the focus on what is essential. And you do this by understanding and then being able to clearly articulate the why behind your objective. Now, as a creative, visionary product owner, I might be thinking, hey, I've got this school bus full of kids, I wanna give them a snack, keep them happy. This is an educational opportunity, right? So the snack could be the periodic table of cupcakes. That'd be really cool, they'd learn something, we'd keep them happy, that's awesome. Then again, there's the childhood obesity epidemic and maybe I wanna keep these kids healthy, so maybe broccoli is the right snack. I'm juggling all these different constraints and ideas. As I think about it, I decide, you know what? Everybody likes brownies, kids especially like chocolate. So I've gotta figure it out, I'm gonna have brownies. So this is an easy decision. Oops, I'm gonna write my RFP for brownies. I'm gonna shop it out to three to five bakeries around town since my cafeteria can't do it for me. I'm gonna go with the lowest price and I'm all set. Maybe not. Remember all those constraints? I haven't done my job well if I haven't incorporated expert input and insight into my constraints. I'm missing out on a greater degree of creativity and knowledge than I bring to bear on the situation all by myself as the product owner. So what I wanna do is find a collaborative, consultative process where I might learn that brownies aren't the absolute best choice. It may turn out after discussion with my external resources that cookies are the best choice. In my conversation with this outsourced bakery, they might have pointed out that cocoa prices are volatile and that if I'm planning to fit in this budget over the course of the school year, the fluctuating price of cocoa might result in me blowing my budget. They also may point out that brownies have a limited degree of substitutions you can mix into them, whereas cookies have more flexibility. I can substitute chocolate chips for nuts or raisins or toffee and I can vary the volume of those mix-ins based on budgets. If I'm tight, I can maybe put fewer chocolate chips in this time around. Brownies may fall apart more easily than cookies, et cetera. There may be a whole list of reasons why cookies are the better choice. I might not have thought that through without talking to this outsourced team. So great, I'm sold, let's bake cookies. Therefore, it's time for me to partner with the project manager at the outsourced bakery and facilitate the creation of my batch of cookies for the school trip. Where my priority as product owner is to focus on what's essential, the project manager's priority is to prevent pain. And to talk about that is our own project manager at Estonich, Chris Walker. So preventing pain is a very lofty objective and we take it very seriously here. In our experience and my personal experience, we found that the best way to manage scope and to manage that pain is to really empower the product owner to do that themselves. And we do that in two main ways. One, we manage assumptions and not just manage assumptions but we also want to excavate. We want to explore those assumptions. We want to manage them, we want to mitigate them. Sometimes we even need to combat them directly. The other thing we need to do that's equally important here is to provide the resources and the tools that the product owner needs to actually make those decisions and to manage scope themselves. So to talk about how we do that, let us continue to torture this cookie metaphor here and let's talk about what scope creep looks like in the baking world. It can look like being over budget. This is a very easy thing to do, especially if you've got a product owner that knows exactly what they want. They might know exactly what Godiva chocolate they want to use. They may have these great chocolate silver sprinkles that they saw that just have just the right look and intricate illustrations as well. The problem is they may have a budget that can only afford them two to maybe half a dozen of these cookies. So that's great, we've identified the perfect cookie, but we've got a bus full of children to feed and I do not want to deal with the ones that don't get the cookie. As a project manager then our role here is to help expose, to help understand the budget here and make sure that we're all on the same page about it. Scope creep can also look like taking longer than expected. And this analogy that means that we have to take our cookies out before they're done. And while a lot of us like licking the spoon while we're baking, the last thing you want to do is to give a bus full of children some cookie dough, uncooked cookies. And it could also mean missed deadlines. In this scenario that means that we do not have a product to launch by the time our bus takes off and this is not an option either. We've got lots of stakeholders. We've got upset parents to deal with. We've got a school board to answer to and I do not want to be on the bus with screaming hungry grumpy children. Simply not an option here. So as you can see, baking much like project management is not easy. There are lots of ways that a project can go wrong just like baking can go wrong. In fact, the easiest thing in the world to do is to manage a project imperfectly. What we've found in our experience is that these imperfections and these opportunities for scope creep and trouble come from assumptions. And assumptions while almost always are innocent, they almost always come from a really good hearted place. They can almost always derail a project. One of those assumptions is in the form of technology. It can take the form of an assumption about technology. If we have a product owner for example who may do a little bit of baking of their own, they may have an oven at home. They may even win local pie baking competitions. But that does not mean that they understand exactly how our oven works. We may have a very high end convection oven that we work with and there are lots of subtleties that go around that. We may also be at a higher altitude on the other end of town where we've got drier air and there's a lot of intricacies and subtleties that go along with technology. We can't always assume that our product owner understands it perfectly. Our job as the project manager is to help expose these assumptions and give them an opportunity to breathe, make sure they're all in the table so that we can transform those into knowledge about the technology, not just assumptions. There can also be assumptions about roles. There are many product owners out there. I've worked with them. You may have worked with them. You may be that product owner that thinks that it's your job to make lots of decisions and very often, often over the shoulder of the team that you're working with. Equally dangerous and just as likely is a product owner who may be unwilling or unable to engage in the decision-making process. Our role as the project manager is to help our product owner understand what our rituals and what these roles are and not just to communicate those but to communicate them continuously. There are assumptions about possibilities and these are really fun. You'll recall that Rob is a very visionary product owner in our scenario and he comes up with these grand blue sky ideas about these cupcakes that can be educational as well. We love working with product owners like this because there's a lot of fun that can come out of it. However, as project managers, our role is to be comfortable talking about and reminding our product owner of what the constraints that they have outlined for us are. The other opportunity is with product owners who may think that nothing is possible or they may not understand what is possible and they may come to you with a list of requirements that are half-baked. They don't have, oh, I continued the metaphor there, didn't I, that are not complete. Kind of the bare minimum of what's required and there may be opportunities to give them even more than that. So our job then as a project manager is to encourage some thinking, to be a little bit of an instigator to bring out some of those ideas but also be comfortable being the wet blanket sometimes to bring them back to reality. We have to be kind of the angel and the devil on each shoulder, helping them open their minds and also to remind them of what the constraints that they've identified are. And assumptions about scope. I picked this cookie because I think of myself in this scenario where I once thought it would be a great idea instead of baking a dozen cookies to take all of that dough and make one single big-ass cookie. What I didn't account for was the fact that that's gotta spend a lot more time baking and it's a lot more difficult to get a cookie that size baked and not just burned on the outside. Going back to our scenario with Rob, he may decide that he has got pecans. He knows that pecans can fit in his budget. No problem there with scope. He knows how easy it is to drop those pecans into the batter. Again, no problem with scope there. What he hasn't accounted for is the fact that our baking team has to chop up each of those pecans. So again, an assumption that's really easy to make that's not going to extend our deadline. Our role as a project manager, of course, is to help unpack and unroll these assumptions. And often, there's a lot of communication that takes place there too. My assumption is probably like Rob's. I think, okay, pecans, great. Yeah, they're cheap enough. We can afford them. We can drop them into the batter, no big deal. But we really need our developer's expertise when it comes to some of those intricacies about chopping them up. And James is gonna talk a little bit more about that in just a moment. So as you can see, if you didn't already know it, the relationship between our product owner and our project manager is a crucial one. And it's one that we take very seriously. In addition to managing those assumptions, there are three main things that we as project managers strive to give to our product owners to help them make these decisions themselves. We provide these three things through a methodology called Agile Scrum. We're big fans of it and we find it helps out quite a bit. But I think the principles here remain the same. And the principles are that we've got three things we wanna offer. And that is flexibility, a context for providing our product owner with directions and advice. And lastly, indicators. So starting with flexibility. Flexibility is extremely important. Flexibility is what gives us creativity, for example. Without flexibility and creativity, we wouldn't have delicious baking treats like this, which is not quite a cupcake, but it's a meatloaf kind of cupcake with this beet frosting and these chive sprinkles. We can debate whether or not we wanna eat this, but we can agree we like creativity, we like flexibility. In the baking and in the project management world, there may be a scenario that maybe, see how this sounds in the baking and the project management world. Chris, I know we initially approved white chocolate, but now that I see it in there, I'm not so sure. Or Chris, I know that chocolate sprinkles are what was approved, but it's just not the competitive advantage anymore. Or Chris, my brown sugar budget just got slashed. In these situations, our product owner is going to need flexibility as well. Agile Scrum is something that provides this right out of the box for us, where we take all of our user stories and we put them into a backlog that our product owner is able to prioritize. And that need to reprioritize is not just an arbitrary one, it comes from real business needs. For example, Rob may learn that one of his students on the bus is allergic to nuts. It's not just a random decision that we need to accommodate, it is something that is a very serious business need. So providing that flexibility is very important for our product owners. Great bakers need direction and they value advice. They use recipes, like the ones they cut out of magazines, the ones that you print out from the internet, and ones like this that they are used and they are valued and they are carried around for a long time. You've got a long-term relationship with recipes like this. Because we love them, we value them. Product owners need direction and they value advice as well. So our job as project managers is to provide this for our product owners. Again, Agile Scrum is something that makes this very easy for us, but there's lots of ways that you can do that. What we do is every two to three weeks as part of our development cycles, our product owner is having a conversation directly with our, excuse me, our product owner is having a conversation directly with our developers. Where our developers, and again James will speak a little bit more to this in a couple of minutes, speak directly to the product owner to help give them real knowledge and technical expertise that they can understand. At the end of this process, we really do want to become that well-worn recipe. Used, valued, hopefully not too stained or worn out. And lastly, great bakers need indicators. You can see here when we're making cookies, our baker has got the ability to see exactly what temperature our oven is set at. Our baker can see how much time is left on the countdown. They can look through the window to watch the cookies rise and they can even take the cookies out and poke them with a fork to know exactly where we are in the process. It is no different for our product owners. There's a phrase, there's a phrase that we use in baking, which is at 400 degrees for 20 minutes or until golden brown. Until golden brown. That's a phrase as a project manager that makes me extremely nervous. And it's a hard one to define with baking and it's an equally hard one to define with projects. What is golden brown? What is done? And how do we know when we get there and how do we know when there is? Agile scrum does not provide this so easily. We've found that we've had to use a couple of modifications to get there. But the goal is here to give our product owners a view into the oven and to give them a view into exactly where we are in this progress of making cookies or making a software project. Not just that, we wanna give them indicators as well as how that relationship of where we are in the total process, how that relates to budget, how much money have we spent and what's the relationship between that and what we've got left to spend. We've given this a lot of thought. We've come up with some visualizations and some dashboards that we use on a regular basis with our product owners. You guys may use different methodologies than Agile scrum. You may even work on a fixed fee basis, but there are lots of ways that it can make this process visual for your product owners and it's well worth the trouble. So baking, like project management, is not easy, but it can be extremely satisfying. It's part art, part science, just like baking. And like baking, it can be very warm and gooey when done right. If we're doing our job right as project managers, we are not really managing scope ourselves. We are really just providing the product owner, the tools and the resources they need to manage it themselves by helping them manage their assumptions and helping them to have the flexibility, the context for advice and the indicators that they need to do that. Another important perspective in this though is from the developers and James is here to tell us a little bit about that. Thanks Chris. So in active development, the role of the developer is to take constraints and these requirements and translate them into stories on the backlog. That means the developer needs to be able to understand business needs so that he or she can help prevent present pain or future pain by understanding the so that behind each story. Also, a developer needs to be mindful and thoughtful about decisions they make because they have to help control scope and because implementation can cause future pain if not done correctly. So how should developers make these thoughtful decisions? So, sorry I skipped ahead. So we have a new requirement to continue to overcook this analogy. Our product owner comes to us and says, I want broccoli in our next batch of cookies. So I would like to say real quick, I wanna become a geeky developer for just a minute. So a couple thoughts could come to my mind. First, oh my gosh, this guy is crazy. Who would want broccoli in their cookies? I mean, does this sound crazy to you? I often, as developers, we hear these kinds of requests and they sound just as crazy as putting broccoli in cookies. My second thought could be, wow, this is so cool. I could be that guy that puts broccoli in cookies but I must have self-control and I must think about the implications of doing something like that. So I need to know, based on the current scope of project, whether or not I need to ask Rob some more questions about his request. So the developer, as the balancer of context, needs to keep in mind, again, this business need and the software implementation to help keep it in balance and they're really in the best position to help inform scope decisions. So how do we handle this broccoli request? I think we can ask a few more questions. To understand the so that of adding the broccoli. How will adding the broccoli affect the target users? Can we actually add broccoli to these cookies? So again, is it possible to add broccoli to these cookies? And will the addition of this broccoli cause pain in the future? So when we think about the tools that we will need, will there be a need to add broccoli again in the future? What kinds of preparation are necessary? Additional cook time? These kinds of questions allow us to think about the solution both now and in the future. What if the broccoli was a hit? And we didn't prepare for a second round or increased demand. Also we didn't talk about how this would affect the shelf life or again, the cooking time or the togetherness or the taste. So when evaluating scope addition to a project, I like to keep two things in mind. One, the end user and two, the business need in light of the overall goal of the product. So let's put on our user bib for a moment. Would we eat these cookies? In software, this could be more user experience oriented. How hard would this be? What can I do to make this easier? Is this part of the goal of what I'm trying to do in this application? And back to the analogy, why broccoli? Is there nutritional value? What do we get from adding broccoli? And I really can't think of a child that would want to eat broccoli cookies. There may be an edge case person that would want them. Mr. Hipster may decide that he wants broccoli cookies because he thinks they're cool or not cool in this case. So we evaluated this addition from a user's perspective. But what about the business perspective? Can we understand business need as developers? I think we can train ourselves to be able to do this. But I think it requires an understanding and desire for empathy with a client and empathy with a user. And adding broccoli doesn't really make sense at the moment. Maybe we missed something, something that we don't currently know. So let's start some dialogue. We can ask our product owner some questions about how important it is to them to add this broccoli. And this may sound something like, what is the purpose for adding broccoli to these cookies? Can you tell me more about why you added and you wanted to add broccoli to the cookies? What do you hope the end result of adding broccoli to the cookies will be? And did Rob just have an awesome plate of broccoli last night and decide that it was good to add broccoli into the cookies? The hope is that we can get the client talking so that we can understand what they are wanting and be able to give advice. So after some questions and discussion, it becomes apparent that the real purpose for adding the broccoli is to make the cookies healthier. What kinds of ingredients make cookies healthier without drastically changing the cook time consistency or taste? What is easiest to implement? And what could be the cost of making one of these changes? So we have some options on the table. What about these ingredients? Coconut oil, applesauce, raisins, flaxseed. We could evaluate each one and see how they fit into our requirements. So this process happens and in our example, we conclude that cutting out half of the sugar and replacing it with applesauce will reduce the sugar content and the overall calories and so would swapping out chocolate chips for raisins. We're gonna assume that we've communicated any additional steps and costs to the budget and the team, everyone involved, is okay with that. We've communicated that effectively. But wait, wait, wait, wait. This talk's right, it's about scope creep and change and we just change scope drastically maybe. Is that okay? Can we do that? I think scope change is okay when it's an outgrowth of a collaborative process where the product owner, product manager, project manager and development team use the tools and indicators, like the ones Chris talked about, to arrive at an intentional and informed decision about changing parameters. And the developer is in a key position to help inform this process. So there you have it. Three perspectives on managing scope in a project. First, we talked about the product owner's key role, keeping the focus on what's essential and knowing to understand and clearly articulate the why behind their objective. And second, the project manager's key role to prevent pain by empowering the product owner to manage their own scope. And third, the developer's key role to balance the context and help inform collaborative scope decisions. So there you have it. Three perspectives on managing scope and scalability, but there's more. We brought with us some cookies since we've been torturing that metaphor all afternoon long and perhaps they'll wake you up as you head out the door this afternoon. Thanks very much. They are not broccoli cookies. We'd also be happy to answer any questions. Actually, there is an attribution on that slide. That'smyhome.com. Up here on the table up front, we have several types of cookies and if you've got any questions, far away. Just a quick question. This has been to two of these now in a row talking about how to kind of manage client expectations and defining requirements and kind of moving people through the process. And as a small shop, all it's done is filled me with guilt that I'm not doing these things properly. And I was just wondering if you had any thoughts in kind of the growth of development from smaller as a smaller company moving into larger companies, how you kind of integrate the best practices of some of these things, keeping in mind that we're working with clients that have smaller budgets and that there may not be the time to be as exhaustive, but having said that understanding that not being as exhaustive essentially ends up reflecting on us in terms of lost opportunities to do other projects or lost profit margin because we're spending more time kind of implementing and just wondering if you had any thoughts, comments, suggestions or tips, tricks for kind of getting the best of both worlds, I guess. I would say from personal experience that going through that exact transition with our shop, some of it is fake it till you make it and it's being bold enough to say, hey, I'm gonna try this and I'm gonna see what works and what doesn't. You will make mistakes and the Agile Scrum methodology itself kind of is built around that idea that everything's iterative and you're evaluating what you just did, learning from your experience and then applying what you learned to the next cycle. And so I would say there's kind of a meta thing going on there. In our implementation of Agile Scrum, we had to fake it a little bit. First, in fact, we're still taking the Scrum methodology and tweaking it to get it to work for our agency. My guess is the way that we do it will be different than the way you do it or what works best for your business and the way you work with your clients. You like that methodology is kind of a starting point to at least look at and kind of implementing something a bit more rigorous and kind of define Agile Scrum. Yes, it's really helpful to start with an established methodology and then see how it's supposed to work and where it breaks down for your business and then make the decision, do we need to change something in our structure of the way we run the agency and produce our deliverables or is there something in the methodology that could be tweaked that gets us closer to the mark? I would also say that one thing we've learned and you heard Chris talk about it and James bring it up as well, communication is key and pulling your product owner, your client into those conversations and talking about, well, let's be real frank here, what we've just discussed might take us outside the scope of the budget. Where does that put us? How does that make you feel, is that okay? Do you have budget that we're not aware of? And having candid conversations is the key to all of this. Excellent, thank you very much. You're welcome. Let me add, it doesn't eliminate awkward conversations but it helps you have them sooner and with a better relationship with which to have those. So I've also been a part of now, this is a second conversation like this and the issue that we're running into with our team is that the communication between a project manager and that person's technical understanding and a developer and their technical understanding, it's not always very clear. So the developer doesn't necessarily know the right questions to answer and the project manager isn't necessarily asking the right questions. How do you guys go about making sure that communication gap is covered? Well, so I'm in a position as a project manager that I don't have a lot of technical expertise and I find that's actually kind of an advantage in the room when we're all speaking together because it allows James and the development team to speak very frankly and very honestly about what they want to say because they don't have me as a filter. They're speaking directly to the product owner and also it allows me to really kind of sit on the product owner's side of the table and ask maybe the dumb questions or the questions that they're afraid to ask or the questions that just haven't been simply stated to make sure that we're all speaking the same language. So that would be my point of view on it. Anything you wanna add to that James? Yeah, I would say as a developer that it needs to become, and this is maybe easier said than done, but it needs to become a mandate that we have this ability to empathize with our product owners to understand that business need. I said that several times, but it's so important. Why am I building this application? What's the ultimate goal for it? And so if I can get my head into that, then I can understand why people would ask more general, less technical questions. And then I have to be able to understand in the Agile Scrum methodology, we have as a person, or as a whoever, I would like this so that this, right? I need to understand that as a developer so that that makes sense both to the product owner and the developer, and then I clearly flesh that out with the team. And then I communicate the metrics that I need to communicate back up to Chris and say, so this is the estimated points or hours for this, which he can then say, this is the cost for the thing that you're asking about. And so that transparency of those metrics aids significantly, I think. So then as a developer, when you're doing something new that you haven't really done before and you estimate and it doesn't go very well. Yeah, great. Who ends up, what happens then? Ultimately, you as a developer are not the one that is taken out back and beaten, hopefully. And if you are, then come talk to me, and well, no, I'm kidding. But we have to be able to be honest about new things. And our product owner has to understand if that's the case. So not quite faking it till you make it. I think more recently, I've come across some situations where I would have liked to have done a lot more research prior to estimation, so that I felt more comfortable about those estimates. If I didn't do that research, then I would be arbitrarily sizing things to come to some general thought that I may know of. Now, when I am creating technology that's never been created before, then I think the best way to handle that is to get to a very specific level when you estimate. So we're digging in so that we know all the intricacies of what actually needs to happen to get that application off the ground. And when we do that, then we can understand what each one of those little tasks, because I bet you I've done those little tasks, but I haven't put everything together. And so there's a mixture of information and the minutiae, I guess, of the nuance. Thank you. Yeah, so part of some of the process could be prototyping or figuring out how could I build this thing that's very close to what needs to be done. The question, then I have a real question. But something that we've introduced as well is a midpoint. When you get midpoint in the time that you've estimated, it's probably a good, and you don't feel it's gonna happen. It's a good time to check in because maybe the requirements weren't clear. Maybe there's more complexity and that can be communicated to the client. Hopefully midpoint through the sprint so some adjustment can be made. And that can be really helpful for making at least an adjustment in larger scope. You can have something fall off, move it to next sprint in order to spend that time because the issue is really important to resolve. And everyone knows things change, you know. Thanks a lot for your talk. I had a question about the, you mentioned you had some dashboards, some techniques, some reports that you put in front of clients that spoke to budget or progress reporting. And I was hoping that you could elaborate a little bit more on that, what those things look like, how successful or unsuccessful they've been to help you keep things on budget and where you've made improvements along the way to those reports. Great question. It's one we think about a lot. Our dashboards are not perfect. We iterate on them constantly. But they essentially what we're trying to communicate here is the cost per sprint. So our product owners have an idea of what the cost is and we can lay that out weeks or months in advance depending on the project. So they see week to week what they're going to be paying for development. And then the other important thing you've gotta give them is we're spending all this time in our scrum meetings sizing stories. So I know that that's an eight and I know that that's a 13 and I know that that's a three. But those numbers don't do me any good unless I have an idea of how many points equals how many dollars. So we do some basic averages, some basic math to do based on like a three weeks, three weeks, excuse me, based on three sprints. What's our average ratio for points per hour? And those are the kind of things that we try to consider and try to build into some metrics to predict based on our velocity and based on the anticipated hours we're spending moving ahead what the project is going to cost you. I think just real quick, one of the other things is you have to have a fully fleshed out and sized backlog for that to work as best as possible. Yes. Because if you don't, you don't have a perspective on when that's going to end and what that's going to look like. Right, so you're using the JIRA dashboards and tools and exports or do you make something like specific in another application you put in front of them? Yeah, we've got something specific that we've created. I think that answered my question as well. Was it what? That was my question too. I'm curious if that screenshot you show, the thing that had the nice shiny graph of how much money had been spent is if that's part of your dashboard and I just wanted to, yeah, I guess I was curious what other metrics you give them. Do you also include a percent of the project completed along with percent of money spent? I assume so. We've got a handful of tools that we use and they're not all appropriate for every project but things like this, this is a good facsimile of the one that we use and we'll use typical burn down charts. So there's a lot of tools that we have in our arsenal and we use one depending on our product owner and determining on the project with which we're working. Great, thanks. This is a question that probably doesn't have a good answer but how do you deal with clients that come to you and take more of the attitude of we just wanna pay you to take this problem and make it go away. We don't wanna be investing 10 hours a week as a product owner to be involved in this project. We've run into that as well and we haven't found a way to solve it specifically because that type of product owner probably isn't compatible with this type of process. And so honestly what we try to do now is screen those people out before we ever get engaged with them because we're getting into a relationship and the key component of this relationship is collaboration and a key foundation of collaboration is open and honest communication. So if you've got someone who doesn't wanna participate, how can I collaborate? It takes two. So we've got some client relationships where maybe due to other demands on their time, et cetera, we've got a product owner who just can't be as engaged as they would like to be. And we try to work with that and accommodate it but if it's an attitudinal thing it's probably best for us not to engage in the first place. Would you add to that? Yeah, absolutely, although some people sometimes they're like, because we do a training, right? And we sit there with our product owners and we say this is agile scrum, at least the way that we do it. And we walk them through the process and we'll spend more than an hour having a conversation without even touching how your specific project relates to it. And at first there is some eye rolling and some discomfort, what are we doing? I'm learning a new vocabulary, you're giving me a glossary, what is this? But two weeks later they're like, two points, no, we don't need that story. Let's move that one out of the ice box and let's tackle that one next. Well, that's a five, that feels like a three to me guys. Like they really start to use the vocabulary and get really engaged. I think we all know the ramifications of stuff like that, right? If I had made those, or if we had made those broccoli cookies we would have potentially a lot of upset end users or terrible buy-in. So if we're not collaborating, because the basis is that the problem that someone comes to you with is rarely the real problem. And so agile scrum allows you to have those communication to come to this collaborative decision. Because we've had clients that will say we have a staffing problem and the staffing problem wasn't there, they had plenty of staff to handle. And then we ended up adapting a business process to solve their need, which enabled them to get their staff where they needed to be. And so if there's not that process, then you're destined in a way for failure. That's waterfall and that kind of thought process. Here's a goal and meet it. It's hard and almost impossible, I think. Thanks a lot. Any other questions? There are lots of cookies up front. Help us out, folks. Thanks very much.