 We're good ahead all right, so this is the collaboration game time tested techniques for Agile release planning I am Andrew Lindsay the VP of product at New Mote Canadian company that does advocacy type working mainly with NGOs and Far to that I was at the Linux Foundation Open Media and McGill University and in all those contexts I've been working on Agile teams and True of that got a bunch of different letters attached my name that I have something to do with Agile and Scrum development and Yeah, keep running into problems and creating presentations as I find solutions for them And slides on bit.ly I had to collaboration So just a quick question is everybody in this room gonna working on Agile teams already and familiar with the Basics of all that is is there anybody that's not All right Maybe a little I'll go through very brief definitions just to make sure that we're all in Roughly the same page, but I'm not gonna Double into that stuff so much if anybody has any questions or anything. It's clarification. Let me know and I'll be happy to To do my best to clarify So Agile is a principled collaborative approach to Developing software products of some sort Scrum is an iterative framework for for building things quite often software and a sprint within Scrum is an iterative undertaking of building an allotment of software, I guess Usually two weeks long sometimes one sometimes three sometimes four User story often used in Scrum is a way of describing Something that we're going to to build usually takes the form of as X I want Y so that I can Z doesn't have to it's not meant to to To Completely utterly describe what it is that you're you're building or undertaking It's just meant to be a sort of a bookmark or placeholder for a conversation about what it is You're building something to remind you of the specifics of what you're undertaking so that you can have that conversation with the stakeholders with other development teammates that sort of thing a product backlog Giant list of all the things that you want to build over time use your order from top to bottom bite by business value prioritized by what you're going to take on first what's going to give your organization or stakeholders your customers the most immediate Value and things that are you know more debatable at the bottom of that list and then sprint backlog every time You go into one of those iterations in Agile. You're sitting down with the development team and you are you are agreeing to a sort of contract of the the things you're going to take on in your your sprint to your iteration your two-week period whatever it might be and and making sure that they're brought into the sprint with an appropriate level of detail and The whole thing looks roughly something like this. So I'd say in two weeks. You're doing a big circle where the start of that circle you're You're having a sprint planning meeting and you're going through the things you're adding to your sprint backlog making sure that they fit in with the Empirically determined capacity that your team has in story points or whatever, you know the measurement you've chosen to work with and Doing cyclical daily Developing where you're usually doing stand-up or something like that to check in with your team and make sure that everything's on track and at the end of that doing sort of a Demo where you're saying this is what we built and this is how it sort of meets what we're aiming to build and usually also something like a Retro where you're Coming in looking into your processes and saying how can we be more efficient about what we've done here and to improve it as we go into our Our next iteration so we're constantly improving All that makes sense to everybody the norm So Who here is doing something along the lines of agile release planning currently? It's not everybody. I so I got my my scrum master certification Think darn near ten years ago, and it was a three-day Processes that often is in the first two days covered pretty much everything in scrum and the third day was like this tacked on Release planning workshop that I couldn't make it because I had some sort of social conflict or something like that and And so admittedly for the first few years that I was working in agile release planning wasn't something that I gave a whole lot of Attention to probably because I missed that day of the work from But I think I've come to realize over time that it's it's something that if employed correctly solves a lot of problems That came up for me without having it as a tool in my toolbox so to speak So it's it's more Longer or medium-term planning. It's it's typically three to six months increments Place like currently where we do it at a quality level quarterly level. I think that's fairly common it's badly named because In a day in an age where we're doing continuous integration continuous deployment a lot of us or at least You know maybe deploying more frequently than once every quarter It doesn't make a lot of sense in terms of being tied to when you're actually releasing software or whatnot but it does make a great deal of sense in terms of being able to have some sort of medium-level opportunity to look at what you're you're building out and to To develop a picture that your sprints in a given quarter or a given release can flow from So how does release planning differ from sprint planning? Sprint planning as I mentioned one to four weeks often to release planning three to six months often three Sprint plans typically we're trying to set those in stone, right? Because it's a it's a contract between the dev team and and stakeholders the business side of the organization about what we're going to achieve and say to be period so we Occasionally you know something comes up very urgent We haven't built an appropriate padding for it or something like that and then it has to be undertaken But we resist as much as we can typically changing that sprint plan so that we can Accomplish what it is that we set out to accomplish at the start of that two week period that was Release planning on the end is kind of the plan that doesn't survive the battlefield so to speak it's Something that you look at at a medium level maybe in three month increments, but Recognizing that the agile is about fluidity and that things are going to change and that It's about being able to Be versatile and to react to different needs that come up as you're building out a product that sort of a thing So it is something that's it's not necessarily looked at as a sort of Unchangeable as is a set in stone granularity It's kind of interesting with this plan with with sprint level planning when we're trying to plan out a spread We want to make sure that we've got an exceptional level of granularity ideally anything that comes into our sprint when we're going through that Sprint planning process. You know we've had everybody in the room or On the zoom caller where it is where you're doing your sprint planning Pitching to make sure that that would sort of consider it from all angles that we detail what it is and we're building out that That that any sort of unanswered questions are answered so that when we actually start doing the work We've got as much information as possible. Not that we're not going to have conversations and and go back and reconsider things This is we go through the sprint, but we want to have granular small tasks or tickets that we're taking on in this print so that it's Allowing us to do that detail level of work Whereas with release planning it's it's more at the epic level The picture high level although there are opportunities as you're doing release planning to get more detail and to Start to get more granularity as you're putting that stuff together Any questions on any stuff so far make sense So our product backlog if we're not doing release planning, this probably still looks pretty familiar It's it's more granular at the top less granular at the bottom But we can see here, you know our latest release has a granularity at the top and Our releases are typically you know lined up linearly against that back. So I asked the question what sort of problems Do we think we might encounter? If we're sitting down in a room full of stakeholders or customers or whoever really is involved in these conversations in your Context trying to to move a conversation forward that sort of medium level about What it is that we're going to do in the next three months and what our priorities are going to be and and What we're going to undertake anyone have any ideas around that or or Go back some stakeholders will only exist in the perpetual now and for them everything is now and My experience Some of them will try to jam their idea of what should be real now into Into all conversations. And so if you sit down with them and try to talk long term Sometimes people are very focused on various specific outcomes and not willing to divert their attention to with their eyes to the horizon and see a little further Sounds accurate, and I think that the perpetual now part makes it sound very sort of zen as well Did you have something to add as well I See that for sure and and I think again with Agile as opposed to waterfall It's you know people may still be used to sort of getting promised this this very strict set of things that were good build out Not that we necessarily regularly cheap that what we were doing waterfall stuff in the past anyway, but Yeah, any other thoughts Yeah, I think that's what brought me to this myself I can remember In previous organization, I worked for where there were three very distinct departments and and we had Development resources to serve really only one dev team And then so there were often Conversations about you know, where should we split our focus over the course of the next three months? Should we undertake this large-scale initiative for? For department one and the expenses of the department to department three I think especially small organizations, you know, if you've only got one dev team If you've got a limited amount of money to spend if you like an awful lot of small organizations like an awful lot of startups if you've got really big ideas You know Company I work for now. We have Absolutely no lack of ideas about things that we want to do it. It's it's a very substantial list so Oftentimes it's it's trying to figure out, you know, where is that business value for the organization as a whole? As opposed to you know, this one particular initiative the one you're taking or things like that And so I've kind of think of it in terms of This is a little bit sort of tongue-in-cheek but this sort of disgruntlement Triumvirate and that You've got multiple stakeholders. You've got divergent needs and you've got limited development capacity and so I went searching for solutions on on how to address that and Really it was was in agile release planning that I I Found answers that have worked for me And then those center around things like ensuring that all stakeholders have an opportunity to share their viewpoints. You don't have One particularly vocal Person a group of persons sort of out chatting everybody else in the conversation considering all the sort of candidate Opportunities or bits of work that you can undertake from a variety of perspectives Empowering the stakeholders themselves to make these decisions and in a collaborative process and not a competitive process And and so in most respects where I found the most successes is looking at at Things that fall into the category of games and in agile planning. I suspect a Lot of you folks probably do like Scrum poker story point estimation that sort of a thing so things that extend from that tradition and that that have a similar benefit I Did my My P&P before I started getting into which is like a fancy project management certification before I started getting into Scrum and agile in a big way and and There was a period of time that I understand it in the 90s where it was really in vote to if you were like a Big organization three had a person IT team to to bring in consultants To do something called a wide-band Delphi estimation, which was you would pay this consultant group to come in and they would like Grab each developer and isolate them in an interview room and ask them all sorts of questions about estimating How having a particular piece of the product we were building out would take to To build out and then they would compile that data and run different algorithms against the nice bozer or something like that But they would try to to merge all those estimates together without having at least initially a significant influence of one person against another and and I think the brilliance of that sort of storybook game is that it Captures all that stuff that people were maybe paying way too much money for in the 90s and to the single back of cards that people are sitting around a table Working collectively with in a short period of time in a room without necessarily needing to to bring in an external party, so I've been fond of it and yet fond of processes that sort of emerge from it as well So you know a very basic way of starting to look at your work and this is maybe more of an organizational tool than a game is Is just trying to divide things up into to four categories, you know looking for those things that fall into the High business value and and low effort because we know that That's where we get our maximum benefit typically, you know, there's always exceptions to things But if you can do something in five hours that Give you eight points of business value. How are you measure business value? Definitely seems like the sort of thing to consider tackle And Talking about scrum poker brings us to to business value. Is that something that anybody else anyone here's worked with before? So a few people very similar to scrum poker except you're getting the People on the business side of the conversation not to estimate the complexity of the the user stories that you're undertaking or the Time involved with them you're getting them to estimate the value that they're going to bring to the Organization or if they're your customers the value that they're going to bring to them as as customers It's a very similar process everybody's got a deck of cards and You're going around the room and saying you know describing this thing as much detail as you can this is this is the new widget that's that's gonna Change the world for us and and here's why and then everybody puts down their They're part of the same time and use a simple calculus to figure out What the business value is and that might look something like this if we've got Jennifer Muhammad and Pierre and Darcy and My other things we're looking at is rebranding everything Jennifer thinks it's five Two Darcy thinks it's three they're all within Within one or two points of each other on this modified Fibonacci scale and so We take the one with the most votes essentially we take take the three event system Super valuable the Mohammed Jennifer thinks that it might not add any value to the organization In fact, it might harm the organization somehow to have an event system And they're so far apart That they're much like in scum poker They have to come back and have a conversation usually starting with the people that are on the the far right with so Mohammed can explain where that extraordinary amount of business value is coming from and Jennifer can explain Why it is that she thinks that there's there's no business value there at all and then after that conversation has had you estimate again ideally come to a No place of balance where people have sort of met in the middle so to speak Makes sense Innovation Games is a book a fella named Luke Homan Wrote almost ten years ago And a lot of things I'm gonna talk about today come out of it not all of them by any means but he was focused more on Sort of sales driven conversations, but but I think that most of what he put together is still relevant and The sort of release planning conversations that that we're talking about So remember the future is I Think an album title by German Prague band, but also a Game that this gentleman came up with and the notion is that You get let's say you've got five people in the room together and you're all sort of brainstorming about what to do with the product to improve it to move it forward to sort of come to a place of Perfection or near perfection so you all take a look in your Imaginations and where the product is in the future. Let's say five years down the road ten months down the road three months down the road whatever makes sense and then you take post-it notes or some other mechanism and you You List the different features you're gonna have to add to the product in order to get it to that that new vision that you've come up with yourself and And let's say you had up with five or six post-it notes out of the deal You keep them to yourself at first when everybody's done that process. It's sort of like scrum poker or business value poker Everybody reveals you know, they're their chief of post-its and and then you start looking at where there's overlap so if Bob and Jane both came up with with a few tickets around an event system You look for the ones that match you look for the ones that are you know, Jane had one idea That was a little bit different and Bob had one that was a little bit different and you bring together these these these post-its to form a sort of centralized vision from from everybody's input shape the product tree So this is an interesting way to take the results of something like that to remember the future game and to to group them together sort of overlapping the product as it currently exists to get a sense of Of where the balance is in your ideas So typically what you do is is create a tree like structure on a board or a wall or something like that And you cluster your post-it notes on it with the ones that represent either functionality that you currently have or functionality that's sort of like step two to get to step three Closer to the trunk of the tree and things that are farther out The the tips of the branches so to speak and it becomes an interesting exercise because you can sometimes see that that this particular branch with this particular cluster of ideas is sort of over elated with With different post-it notes and this one over here has has less so maybe it allows you to spot gaps in your thinking or what you've been Considering in terms of building out these things it allows you to spot, you know Maybe we're giving too much focus to this particular part of the product at the expense of something else just gives you an opportunity to To view things through that that lens of balance sailboat allows you to Take that product after you sort of decided what is your building and spot things that will allow you to to sort of speed up Your development of it and things that might slow you down so maybe it takes the form of a picture like this forgive my rudimentary Skeption But you got some post-its to represent things that I saw that sailboat for so Is is whatever it is you're building Things on post-its blowing into the sails or things that are going to propel you forward that are going to be beneficial to you like I don't know. Maybe this Library has been released that's going to make it extraordinarily easy to develop this particular part of the product or Anything along those lines anchors or things that are going to sort of slow you down. Maybe there's a bottleneck Review process somewhere that's it's going to be necessary here That's going to prevent you from working as sort of swiftly as you'd like to or it's going to force you to think about working in a In a slightly different manner or forming your iterations in a slightly different way and Sort of weird jagged thing and the right is meant to be an iceberg and then those are things that could you know totally tip over the boat and prevent you from Achieving success Does anybody familiar with the bleak strategies? sort of So Brian, you know the musician and Peter Schmidt the artist in the 70s I was doing these artistic projects and they were sort of looking for ways to to unblock themselves creatively or to find Scenarios where they weren't thinking of like everything they should be or you know just Opportunities to sort of change context swiftly and Get yourself unblocked or or find ideas. You might otherwise be ignoring and they created this deck of I think an end of being hundreds of cards where It was late night in the recording studio and they were sort of stuck on something they'd pull one out and it would Change their perspective and they'd move on There's sort of a similar thing It's been created by a gent called the Monchilio That could have pretty sure that's presentation Called 35 parts and I'll pass this around if you guys want to take a look at it his notion is that you would take this deck Shuffle it out to everybody that's at the table and sort of give everybody an opportunity to to weigh on on each of these 35 points and it's sort of a linear fashion I'm not actually convinced that that's actually the best application of the stuff But I really like this notion of having this sort of deck of cards that allows you to to pull out and identify Things that you might be missing or allows people in conversation to sort of flag things and and bring them to people's attention and they can be Anything you know the street samples up here making sure that in your estimation or whatnot that you're adding in Some slack time to account for the unaccountable How does this feature and make the user awesome pretty good question to ask stuff along those lines Stuff about just process making sure that you're You're taking appropriate notes making sure that you're estimating things properly making sure that your time boxing conversations probably things along those lines And then I think what is Very swiftly become my favorite and all of this stuff is the the hundred dollar game So anyone here played the hundred dollar game One person all right, so two people So notion with the hundred dollar game is Is you know, we've got that Situation that I described before you've got to say ten people sitting around the table and and they have Split interests and we have you know through our our agile work Purically identified that we've got a capacity of 35 store points a sprinter a hundred store points a quarter or whatever happens to be the store points are different for everybody but You know That there's no way you can be able to accommodate everything that's that's being asked for and and do you want to? to encourage a collaborative conversation in your organization about how to to Figure out what it is you're going to achieve over the The next quarter say the hundred dollar game allows everybody to be given a hundred dollars virtually And to bid on different features or different aspects of what it's what's being proposed to be built out over Over through the next quarter And that typically looks something like this where you've got your your five six seven eight nine ten twelve twenty however many things that you can undertake and And everybody goes through an exercise Simultaneously where they take their hundred dollars and they spend it on the thing is the most important thing and You know this case Jennifer super keen on the rebrand spends fifty on that and forty of bulk email Darcy spent zero on the rebrand When all the money is spent the registration system sort of wins out is the thing that That has the most votes and when I've undertaken this myself We've usually done it in a way where we do this first round where we don't actually speak a whole lot about Personal justifications and whatnot until the voting's finished Excuse me and then Then yeah, when this first boat is in like this We we go around the table and everybody explains Why they spent the money that they spent in the way that they've spent it And then we go back and typically do another round where people adjust their bids Based on the input that they've had from other people and I found this to be At least within our organization a really effective way to To define some guidelines for what we're gonna build out it Doesn't always a hundred percent You know end up being the sort of soul guiding factor in what we undertake, but more often than not What we bring into a given quarter is is pretty close to What's coming out of that conversation? Typically we're doing it in a room full of 8 to 10 people with people on the business side as well as people on the dev side taking part in the conversation and Yeah, it has been super effective for me Anyone want to try a quick round of a hundred dollar game? All right, so badly slash hundred dollar game Should give you access to a spreadsheet All right, feel free to I think everybody should have added access free to add yourself into a column and We'll do a quick round of bidding on On the things about it. All right, so I'm pretty keen on branding. Do I end it noon precisely? Must yeah, they're 45 minutes. I'm sorry a little bit more time I'm gonna bet 30 on the ol re-brand 20 on the event system I'm not particularly fond of bulking now Registration system to me is 25 The tier pricing also 25 down at the bottom of my spreadsheet. Yeah, my hundred total we can start Doing an average of these things over on the right-hand side and ultimately getting a sense of sort of where we're Where we're ending up Yeah, this is pretty much been our our process it's been effective so far So it gives people a sense of thing you don't find it's important to hide other people's votes during the process It's interesting because it's just But it has some level of influence and you're right. There was some sort of like for that first round blind process. It might be improved I haven't found that it's Felt like it's colored our Process too much so far, but I think I work with people that are Are fairly strong and their ideas going into these conversations, so They're there's probably less bendable at first. I definitely you can see it, you know You can see it in that second round change things and you can see people reacting as well too. There's you know Clearly not many people are putting dollars into this particular initiative I guess I will abandon my points there and put them somewhere more strategic that sort of thing, but I don't necessarily think that's a Problem Yeah, I'm sure it's a small degree of this There's a bunch of other Games along these lines is the worst consideration the product box where you sort of very quickly build up faux advertising for your product to sort of Come up with ideas about describing it and what it contains a start of your day We sort of work through the actual real-time use of a product over over different periods of time 2020 vision Where you sort of wanted a time sort Item a against item b in terms of importance and do that sort of binary choice thing all the way through the line to to sort out a list Kanban pizza game Which I'm not sure that I still fully understand but looks really interesting and Sort of an assembly line Look at how you might build something out and what needs to come first and whatnot And yeah a bunch of books here that have influenced my thinking on this and then a bunch of blog posts as well that I think have been useful and Go on to the slides Billy slash club version game That's it questions thoughts concerns go for so Typically convince people to sit down to play the game Participate What's that free coffee free coffee is a good idea So I'll say this and that in my most recent context of the Linux Foundation it's a New mode Not so much. Those are organizations of new modes We're less than 20 people the next foundation When I worked for them is maybe a hundred and sixty something like that in total and people are pretty Invested in seeing things associated with their organization forward. I Worked for Miguel University a long time and I don't need to say anything that anyway sounds disparaging, but I will Suggest that when you have you know 3500 People using the system that you used to build websites and the vast majority of them have very Well some some needs an alignment an awful lot of them that are are are spread out and Yeah, I think there were we're definitely situations there were where we struggled to To get everybody to participate Or even you know having contend with those numbers and how do you choose the right the right to the right registers edited So what now but that's more of like a fairly Unique massive organization problem. I think I mean more like I mean more like level of hostility rather than size, right? So in a context where you're meeting resistance, right? How do you have you contended with that specific issue? I don't necessarily mean Could be as could be an NGO. That's just where everyone thinks, you know, too busy What do you do? Is that ever it's not something the county possibly I don't think I've encountered it to the the point where I would describe it as a still it I definitely encountered it to the point of People feeling like it's it's something that that doesn't have merit or it's it's a distraction from the action conversation Typically I try and push swiftly into to an example like the hundred dollar game where you can in five minutes just sort of And then I think for the most part people tend to Get things like this when they see it demonstrated what they can see Okay, this is instead of me sitting here for five hours having a conversation Where it's pretty hard to To pull anything out of it. That's that's that's empirical. That's measurable We can start things off or at least early on in our process get into something where we can quantify very exactly What it is that we're trying to describe in terms of our own interests and way that might get the interest about others And I think when when people are brought to understand That that that it sort of has that simplicity to it that it has that Quantifiable descriptive quality to it. I think it sells itself Matt Give examples of times you've seen this process devolved and just could totally off the rails Man I think the closest examples I have to that are sort of two occasions either You know you end up with Let's say you've got the capacity to do six things or something like that And you end up with the top five on the list or the top four in the list are really Clearly obviously things that are sort of winning out, but then you've got four things under underneath that are Equal in their votes more or less and the way we saw that is just by doing additional rounds of voting till people sort of Get hip to the the fact that they have to have things one way or the other and then usually that's worked out I think we've also seen it get tricky and I'm fortunate to work for a company that's fairly balanced and it's approached bringing in opinions and Keen on collaboration between different teams and whatnot But you know you've got your completed list and then the CEO comes in or the CEO sort of the CEO comes in and And this is very keen on this one particular thing over over something else in those circumstances I've I've Generally resisted having that one opinion sort of outweigh everything else But I mean your company cultures perhaps different than mine and I I'll leave that to your personal Jesuit and Look for other solutions. So, you know, we may Bring in a small allotment of work to do research on something that we're not we're going to take in a future quarter to get The ball rolling on something that's been identified as particularly important to one particular person just to Which is always, you know doing a little bit of exploratory work without nature If you know that that it's very important for the company to say the next quarter round to Move that sort of thing forward is pretty beneficial as well Okay Because I'm wondering in a context where Things added back and some things may evolve and you know, that's a much of a priority And I think about like for example, the only thing where you need to some kind of factor code or Google managing or the little boring stuff That doesn't have much value for most people like marketing But at the same time is required and so I'm wondering how many times do you kind of run those games? Not every spring obviously because it would be way too much time But I guess once a month for me, well if you're a one-month sprint not much value So how many times do you run those games? So I would say we run that stuff just Once a quarter and and it's part of our our larger one or two-day quarterly planning process But we do So in addition to to building out a plan that accommodates You know, we're gonna take this and this and this and this on based on this conversation We've had in this plan that we've had we also tend to work with with Quarters and and buffers as we do I draw development. I think I did that Talk on this maybe four years ago that I might still have the slides for it I'll see if I can take those out for you, but And it's probably evolved the fair amount since then, but you know we We make sure that in every sprint we set aside seven points of our capacity for for unknown stuff and that might be security updates that might be stuff that's that's just from a Maintenance perspective should be prioritized, but is in no way glamorous like nobody really Cares that much about about whether your website is HTTPS Only until you know three months later and they suddenly notice there's SEOs tanking or stuff like that We will not put these in the in the point game. We will not game those like that We will just consider that we need these so let's play with the rest of the amount of development and See what would be the best business value of every other point. Yeah, and in the release planning. It's usually high-level Things and not necessarily small things that sort of fall into security maintenance So it would be you know, we want to treat this events. We want to create this This SMS system or something alumnus lines a sizable chunk of the product and then At the sprint level when we're doing that sprint planning and a regular basis home We're going through our backlog and you're rooming it. We are taking the time then to to estimate the amount of effort involved in doing like a security update or doing this AWS maintenance thing that's that's probably not particularly easy to communicate to all parties involved but yeah making sure that that stuff has a home and that it's not ignored and that it's Sort of safer that I think it's fairly important. We do that too with other things We make sure that we've got an allotment of work around UX that sort of goes beyond just hitting all the high bars we aim for with with all our regular development But something that allows us to go back and iterate and improve and make sure that we've got sort of a polish from from end to end across the system a little bit of time that's spent as well on a apps making sure that the site is properly accessible if There's ever something that somehow Yeah, you know just just Clean and how should I speak the stuff that should always be undertaken and every take of the better take anyway, but making sure that it's double-checked and kept solid You had another question, I think I did so Closing the loop or following up, you know After the fact right everybody sits down and plays the game and puts whatever That's they're going to put on whatever you know big feature Do you do any retrospective work with the results of this game? So You know now we went and we built it It was actually, you know the ratios actually work out like this Here's what we thought of us here's who turns out to be good at estimating Here's who where or here's the kind of thing we turn out to be good at estimating here are the kinds of things We don't this is there any a follow-up. I think I can I don't think I'm Sharing any great trade secrets or anything like that by displaying them. I'll go back to last year something just to To give a sense of things but usually what I do when we finish the release plan is I put together Something that reflects the report proposed plan after we've done some some high-law situation around it We put those totals together and then we try and fit those points into to whatever our shirt of empirically determined capacity And then I created another table that I've updated from sprint to sprint where we take a look at what we've actually done And how it sort of fit into what we said we were going to do and How it hasn't you know very often will you know, we're a young startup and And an opportunity comes along that we hadn't even thought of when we were in release planning and we'll change things And shift course and these numbers will shift. This is actually a pretty Pretty there's more extreme shifts here than we probably normally see but yeah, you can see here We decided to do this action page Thing and then you know halfway through building up. We decided we were going to make it way more fancy than initially anticipated and and the numbers on that sort of blew So yeah, we refer to this or a table like this every sprint And and when we're doing our sprint planning And even our demo usually refer back to it and you know, this is what we achieve This is what we talked about achieving and yeah, this is what we're sort of Keeping track of that way I think if you're ever looking to do another talk People would be interested to see in that part of it. Well, you know, yeah So we're not just playing these games and then throwing away right this point being like Not only does this help us figure out our business value today when we're doing the release plan But it also looks into this bigger process and here's what that bigger process looks like for us Well, it could be your particular case or whatever. I think people would be interested in Because I see a lot of frameworks out there in the agile market Talk about this front-end gaming But not talk about how do you change things as you go, but there isn't a lot of this like let's look back I Thank you everybody