 Right the the session hasn't started, but I thought it would be an interesting exercise to do a bit of scoreboarding So I have a question for you, and I would like you to raise your hands if you believe That you must have estimates to correctly manage a project to meet a certain deadline Who believes that having estimates is a must To meet a certain deadline. Could you put your hands up? I'll just quickly try to count you don't put your hands down, please alright, that's 25 and Related question how many of you believe that estimates can be made accurate up front obviously not after the fact So if you could raise your hands Can estimates be made accurate? Is it even possible to do that? Can you please raise your hands? Okay, the number is zero Did I get the number right? somebody object All right. All right final question one one. All right good Okay then the Final question how many of you wish Not that you know how but that you wish you could work and Deliver software and be successful business-wise without estimates I'll just quickly count. Please keep your hands up. It's about 50 So the challenge for me is to see how these numbers change and How they change so that this goes up this goes down and This goes down. That's what I want. That's my definition of success for this talk Will I make it? All right, I think we are on time right we can start Good so welcome everybody There's still some seats and there's more space in the back if you want to get out of the door So that the others that come in also have some space so Feel free to move around first of all, I want to thank Evan for a Wonderful warm-up for this talk because the talk he just gave about agile contracts fits right into what I'm going to talk about So, thank you Evan. I know you're outside, but it's on video So you can look at it later and thank you for the clicker as well because I didn't estimate correctly and my batteries ran out So the title of the talk is how can you predict the release date of your software without estimating? and and the keyword here is predict not estimate you can Read a lot more about this in my blog The blog is full of articles and I've started collecting articles from the community So if you have written articles about no estimates yourself, let me know. I will publish it on my blog I want to give as much Credit to others that are working on this space as I can and just I think it was yesterday or the day before There was another no estimate stock and the day before there was another no estimate stock and last week There was another no estimate stock and this topic is growing And there's a good reason why this is growing and the good reason is this a Lot of us wish that we could work without no estimates. Well, guess what I've been there. It can be done But I have to warn you This talk is not for the faint of heart. We are not going to talk about easy subjects We are going to challenge some of the most Cherished beliefs and I do use the word beliefs on purpose of the software industry Let's be clear. There are hundreds perhaps even thousands of the many years of study research and PhDs into estimation There's a lot of vested interest in keeping the estimation industry alive my vested interest is in keeping the software industry alive and sometimes You have to make sacrifices a tweet by Paul clip most backlogs are ways how many of you work with backlogs every day Okay, almost everybody. How many of you know what is in the 10th position of your backlog right now? 10th You don't so why the hell is it there? Because obviously the fact that you don't know proves to yourself that it's not relevant right now. I Mean it might be relevant in the future or it might not we don't know yet But one thing we know for sure. It's not relevant right now. So the question is why did you estimate it already then? right, that's what Paul clip says when he says estimating backlog items is therefore super waste not only are you spending men Years managing backlog items. You're spending tens of many years estimating backlog items. That may never be touched May never show up in running software and Then you go to the next level Which is revising backlog estimates of course because you know a story we thought was the one is actually not a one It's now a three. Oh my god. All of our estimates are wrong We have to revise all of the estimates That's what many teams do. I don't know if you do that, but I've been working with teams that actually did that I was doing that myself Back in the days when I was just a cargo culture. I Just followed the rules and at some point I figured out this doesn't really work So for me just like for Paul clip revising backlog estimates is Inmentally deranged territory It's not even waste anymore. It's a totally new dimension of waste and Sometimes when you do realize these things you have to make a choice Once you know that estimates don't work and I do know from practice from data and from my own Experience from being in rooms where people were falling asleep or just shouting about the story point argument. I Had to make a choice. I Was clearly seeing that estimates were not adding any value to the work we were providing So of course, I could have decided to ignore the problem and wake up the next day as if nothing had ever happened and continue my life in oblivion or I could take the red pill and See how deep this rabbit hole goes How deep does the estimation rabbit hole goes? Where does it take us? What's on the other side of the estimation mirror? So just a word of warning what you're about to see is dangerous And I'm not saying dangerous as in a funny way. I have been insulted almost aggressed Certainly verbally almost physically and of course many people have tried to discredit me because I believe No estimates is a possibility Not only do I think it's a possibility? I have data to prove that estimation is not a valid possibility anymore Not with the knowledge we have today Not with the experience we've gathered over the last 50 years of software development and about 30 years of massive Software industry it was really only in the 80s that we started developing this mass software development industry and I know about you guys, but I'm old enough to remember that that's how I started I started When the industry started this industry is my life This is where I've been all my life even as a kid as I was programming my first few games Because obviously games are the only interesting thing to program at least for kids, you know, I've been here This is I have an allegiance to this industry that goes beyond my profession This is my industry, and I don't mean mine in terms of ownership I mean mine as in this is my tribe you guys are my tribesmen and Women and what I'm here to tell you is that some of us have seen beyond the matrix We know what's behind and We know that we don't need to stick to old ideas to be successful In fact, I would say that the old ideas. Oops wrong slide the old ideas are poisoning our future We cannot progress as an industry if we continue to stick to old ideas. I'll give you just a few examples Kent Beck Of course, he wasn't alone. There was a large group of people working with him But he came up with this crazy idea, which is one small part of XP, which is called test driven development How many of you let's use a scale of one to five where one it's totally crazy completely absurd idea and Five it's the most natural. We've always done it this way idea as of today Where is TDD in your mind? Is it the one? Can you raise your hands if you think it's closer to the one than to the five? Okay, no one raised your hand raised their hands. How about how about is it the five? Is it really something so common that people would say this is how we've always worked. This is so natural It isn't the five yet But it isn't the one anymore This was a crazy idea and I'm taking TDD as one example because that plays into one of the examples I have later on but TDD is just one example of how our world has changed if you had told your boss, let's imagine that you were as old as I am and You were had told your boss in the let's say 2001 2002 time frame that Mr. What would be a good name mr. Williams? I would like to try TDD this week What would you think about that and of course mr. Williams or even mr. Smith? I think that will play better. Mr. Smith will say Are you crazy? You know what TDD is right? It's like you write the test Before you write the code. Why would you write the test before writing the code? There's no code to test Are you an idiot? Didn't you write your fluxogram? With your class diagram That's how most people would look at TDD in the early 2000s or pair programming what two people doing the work one Are you slackers? No way. We're gonna divide the work more efficient Yet pair programming is probably not a five in the sanity scale today But it's definitely not a one anymore There's a lot of companies actually working with pair programming as their approach to software development today Thanks to this guy and the community around XP your world has changed your world your Software industry world has changed This guy scrum how many of you are using scrum today How many of you were if you are old enough using scrum in 2000 Because it wasn't there No one knew about it This guy has changed your world So when people talk to you and say oh change takes time It's a slow process and you know it will never change here. We've always done it this way Let me be very clear about this your world has changed drastically from what it was 15 years ago Scrum is just another example, but this guy's build on the work of other people Taishio no crazy guy out of Japan. I mean crazy in a good sense because he tried all kinds of new things Lean manufacturing the Toyota production system thanks to the work of Taishi and of course others at Toyota at that time If you read a little bit of history about Toyota, you know that they were the rebels the Japanese government were trying to shut them down They were not the establishment guys of course There's something to it because they were changing the world and establishment doesn't like to change the world Thanks to Diana I now know the phrase that I can use in this situation, which is the monarchy has never funded the revolution So thank you Diana for that Another person that changed your world yet another person that changed the world Who knows theming who has ever heard of theming? Small percentage of you the reason why agile exists is because of the work that guys like him did We don't know the origins of the ideas. We use they are useful without knowing the origins But it's important to know that this person Actually changed the world and they he changed the world in many ways. He was working in the Job training in industry. I think it was called our training within industry program in the United States during the Second World War He changed how manufacturing work for mass production he went to Japan and Because he was kicked out of the US by the way, nobody wanted to hear his nagging He went to the to Japan and in the process created the most productive highest quality industry in the world Of course, he didn't do it alone. There's always a community around these people, but he changed your world actually He changed the whole world of automotive manufacturing because now you have all kinds of companies copying what Toyota learned Back in the days after the Second World War Changing the world is possible and this is what we need to do to our industry. We need to change it once again We need to change it by using the work of other people that have been there that have learned using their knowledge using their ideas and Asking this simple question. Can I apply their knowledge to my situation today? And if so, how can I do that? And that's what no estimate is about It's about applying the knowledge of all of these people to what we are doing today How many of you know what this is? No, please raise your hand like honestly. This is a Real question. Okay, many of you know what this is So that's good because the next question is how many of you use something like this every day today Can you raise your hands? How many of you use this in the last week? Okay So this is a UML diagram probably from a tool like rational rose or something like that And when I started programming you didn't write the line of code before you had probably 20 or 30 a4 pages of these things You weren't even considered as a credible developer if you didn't do your work first there This is how far our industry has changed Today with things like test automation. We had a lot talking about BDD or behavior-driven development this morning I think we have had talks about TDD about setting up infrastructure and so on. We don't need that in fact The examples that Allah gave this morning Show you an insight into how we can totally eliminate this as part of the process Of course, you can still use the diagram to discuss with your colleagues to represent ideas in a graphical manner The actual UML language is not the problem. It was the process that was the problem It was how we used it that was the problem because obviously how we used it is that some intelligent people probably in some headquarters in Europe Or us wrote these things and then they send it over to other countries I'm from Portugal. It's an outsourcing country as well. Of course, it's much smaller than India So you don't hear about it as much, but it's also an outsourcing country and this is how people look at us a Little known fact is that actually most of the developer today most of the developers today are actually human beings And when you look at people that are you know, let's be honest guys like Barry beam They are you know standards of the software engineering industry Their work is important. What they have produced has helped us go forward But when you see people like this writing books with this kind of title you have to ask Do they even know what they're talking about? Have you ever tried to do TDD? No, I mean like really tried to do TDD not say you did Write the test before you write the code Well, that sounds stupid enough, but how about the next one run the test before you write the code what and there's another rule No one talks about say out loud the result of the test before you run the test before you run the code How about that for crazy? You're talking to yourself writing tests for code that doesn't exist But what that is what TDD is is the essence of discipline so when people like Barry beam who is a credible person with a lot of Influence in our industry comes and talks about balancing agility with discipline. You know the world has changed so much That the people who used to know the software engineering world Software industry they don't know it anymore It's gone. It's beyond their understanding TDD is the epitome of discipline if you do it right you are more disciplined than any Other software engineer that came before TDD was alive or was a practice that was used widely So really agility chaos. I don't think so and we start to see these changes having a huge impact PMI Agile certification So no estimates If you don't know what it is or you haven't seen it just Google for it But basically no estimates is only allowed for idealist extremist and chaotic people. I Particularly like the third label by the way because I happen to believe that we are just stumbling on to the meanings of what chaos Really means and I would advise anyone interested to read the world of Peter Glick on the birth of a new science called chaos theory because that has a lot of input into what we are going to talk about For me no estimates is only about these two things if I could summarize no estimates. That's it Customer collaboration over contract negotiation Responding to change over following a plan because let's be honest you wouldn't estimate if you wouldn't have a plan So if you estimate and you have a plan What do you do next? Yeah, you re estimate right or at least you try to follow the plan until it fails and then you re estimate That's not responding to change That's not what the essence of agile is. Let's be honest. You can't estimate Unless you have a plan and if you have a plan and an estimate your intention is to follow it You might tell yourself otherwise But that's a blue pill and I'm here to invite you to take the red pill So no easy answers from me just challenges if you estimate and you plan and you put those two together, you're not ready to change your Openness to change has decreased significantly and Then the next step is the customer collaboration because at the end what we are trying to do is to deliver value Value is what justifies our salary is what keeps our industry alive. So why do we keep on managing cost? That's what estimates are used for instead of understanding value and Jeff Patton has talked about this many other people During this conference have talked about this. It's value. That is the challenge of our industry. It's not cost Just to be clear if you improve estimates, you're in the percentage game You can improve estimates by 1% 2% 5% Actually, if you look at the PhDs out there, they would be very happy if you can muster a 2% improvement in your estimates If you remove work from your backlog if you throw the backlog away if you focus on value Use things like story mapping BDD lean startup You can remove huge amounts of work from your backlog Now we're talking about orders of magnitude improvement for our industry and for the efficiency of our companies for the Effectiveness most importantly because that's what that's what keeps customers coming back It's about delivering the right value Just to dispel the myth. No estimates is not complicated. It's actually very easy. This is all Nothing more That's all it is You start by selecting the most important piece of work you need to do. No, not the 10 20 or 30 most important Story is the one most important story. You don't know that then you ask what the hell are we doing? What is our business model? How does our company make money? How can we find where the value is? Those questions are more important than estimating So that's why that question is there select the most single most important piece of work that you need to do Second step break that down into risk neutral chunks and by risk neutral. I mean very simply that You can't bet your company on being right, right? You are not going to bet your salary your house your employment on being right about the answer to the first question So what do you do? You take a very small part of the answer to the first question You break it down to the point of if you're wrong. You're not going to lose a lot So in very simple terms if the answer to the first question is something like a couple of weeks work The answer to the second question should be a couple of hours work What's the next most important thing we can deliver in the next two hours? that delivers value and By the way, if you think this is impossible come to my workshop on Monday because I'm going to prove to the people there Despite their own beliefs that they can deliver a whole system a complete system from scratch in less than 24 hours And they will do it right in front of me. Actually, they will do it in less than 45 minutes, but they don't know it yet So don't scare them Then you develop each piece of work You do it you check how it works Meaning does it deliver the value and then the final step step number four which is actually borrowed from test-driven development By the way, you iterate and refactor and that's the most important step that no one talks about because that's the step that incorporates Learning right because once you've done the work that doesn't mean anything now You have to show it to someone get the feedback change the way you've implemented that and then Go up the cycle again, and maybe when you go again to now step number one You will find that a piece of work you selected before is no longer the most important. That's fine That's knowledge that is important for the survival of your business model accept it live with it better yet profit from it Because if you can do that faster than any competitor I guarantee you will be in business forever or as much as humanly possible, obviously So there's one question though that you must answer Before you can use no estimates and the question is very simple is your stable your system of development stable? And by system of development. I mean something very concrete I mean the point where you pick up some work from somewhere could be a backlog or could be a customer interview To the point where you deliver that work to production Or a production like environment So system of development is not what the developer does. It's not what the tester does. It's not what the business analyst does It's not what the product manager or product owner does is what they all do together. That's what we are looking at How many of you have been in this situation a boss comes to you and say hey I'm going to go ahead and ask you to deliver 10 stories the next print I mean how hard can it be our customer needs it and it's important for us. We need that input, you know cash is Scars these days and we would like to make the customer happy How about that? How many of you have been in that situation? Okay, about what one-fifth of the audience so that's a lot less than I expected because every single team I've worked with has gone through this situation, but now let me give you this piece of information What if this was the team? You can see at the bottom the velocity of the team the blue line that goes up and down is the number of stories They've delivered for the last 21 sprints or not the last sprint the last 21 sprints the lila color is The upper control limit and the green color is the lower control limit meaning that anything between those would be a Normal variation within the system that the team is in and now you look at the target How do we reach that target? Another question is to you any ideas on how this team could reach that target of delivering 10 stories in the next Sprint Sorry Yeah, split each story into three that might only give you six, but it might also give you 12 So it's a fair bet five would be an easier bet right split each story into five So eliminating the waste is what they should do over the long term Absolutely, but can they do it in one sprint and the answer is obviously not because otherwise they would have done it already Right, but over the long term. That's exactly what they should do There's no easy answer to this the easiest answer. However is the team would cheat I don't know about you guys, but many do and they could cheat in many ways So one way you multiply the number of stories another way is that they could reach into the software development secret bag of tricks and Take their very special superpower tools like avoid testing Don't do code reviews commit only when it's done and So on and so forth and what would that give them crappy results, but they would probably reach number 10 So if your goal is to reach a number This is your natural reaction What I Thought that we were about delivering value. Oh, no, no, no the customer doesn't know value He wants stories, you know, he has to prove to his boss, you know stories are They're worth money stories, you know, I mean we have them in the contract like Evan so eloquently put it. This is where the danger starts right But this is the the situation in many companies and this needs to change and by the way, it has changed It's up to us to go and see what others have done it and use it ourselves At the end the basic the basic question I have when I go into a team I look at their data and I ask can I use this data to observe? Can I use the data? I've observed to predict the system throughput and we're talking about the whole system right from idea to production or production like environment So that we can predict the future release and also detect Changes that affect the system. So obviously it would be important to know if the velocity of the team has changed in some way Wouldn't it right? I mean, otherwise you're still doing the same error as you did with estimates You're basing it on Estimation rather than data so when the velocity changes you need to know So here's how you know There's two rules two very simple rules borrowed Probably murdered as well from the work by Shuert and Deming If you look at the velocity of a team and I mean number of stories not story points because those are just like gummy bears They don't exist Velocity outside limits three times in a row you have You know goes up goes down, but it's always outside the limits then you have to ask okay Something is wrong here. I can't predict the outcome of this team I need to know more about their velocity and that's all you can say you can't say it's going up It's going down. You can't say it's good. It's bad. It's green. It's yellow. It doesn't matter. You don't know period That's what that rule means You have to wait for more data Second rule is if there are five or more points in the sequence They're all going up or all going down this can happen if you add more people to the project If you have more people to a team or if you change the technology or whatever happens that affects the whole project Can cause this kind of effect where there are five points in a sequence Either they're improving their velocity or people are living and the velocity is going down and the same Output the same conclusion needs to come which is I don't know Maybe it's going to stabilize at the lower level. Maybe not. We don't know we need more data before we can make a decision I've applied this to many project teams I have actually collected a database of more than 20 projects which you can access online The link will be up in a second and I looked at this and I saw hey wait a minute This is actually pretty reliably applicable to many different teams Hardware teams software teams large projects small projects companies. I've never heard about they all follow this hmm That should start sparking new ideas and questions in your mind if these apply Applies to all of these teams that I have never heard about could it apply to my team? And that's the challenge for you Can it apply to your team and it's very easy. It's very easy to apply Just put the data in do the calculations and check if it applies. These are only eight examples of that But there are more examples in that database But just to be clear I'm not talking out of gut feeling out of opinion the fact is no estimates delivers Here's an example. I took a project and I asked if I take story point count And if I take number of story count, which one is more accurate at predicting the delivery of a team So this is a long project 24 pro 24 sprints. Excuse me 24 sprints, right? So that's a whole year more or less Which metric most accurately predicted the output of the whole project? So I asked the question in two ways I asked if I take the data from the first three sprints That's the first question which one works better and then the second question what if I take the data from the first five sprints because obviously the Intuition tells me that if you have more data, you can be more accurate So maybe story points are actually good enough if you have more data Well, this is only one project. You can find 21 more at this link But the results were this Story point predictive power ie if you use story point velocity, how well would you have been able to predict the release of that project? The output was 349 and a half story points don't ask why the half they just used it it was their choice and Then the predicted output was 418 story points So obviously if you look at this you have to say story point prediction was quite okay It was only 20 percent wrong which for most companies today is acceptable that's the sad part of the story by the way and 20 percent on the wrong side I E if you had used story point information to predict the output you would have been wrong 20 percent on the wrong side I e you would have been late 20 percent How about counting just counting the number of stories? The true output was 228 stories the output predicted was 220 that was a 4% error 4% That's better than any estimator will ever claim about their methods And I'll explain why in a second, but get this 4% on the right side You would have been 4% earlier, but okay, let's take five sprints You know story points can get better you get more used to it you calibrate better and so on So what really happened? So same of course true output and the predicted output was 396 which was considerable improvement in terms of prediction now. It's only 13 percent wrong But again 13 percent on the wrong side. How about the story count? 4% 4% on the right side now if many of you will have that question in your mind Yes, I double triple check the numbers. I guess that makes it what six times checking the numbers And this is just a coincidence. I don't expect that you will always get the same predictive Number or the same prediction or forecast if you use three sprints or five sprints. I don't know it may happen May not happen. That's not the point. The point was that story points was consistently better and this story repeats It's not just this one project. So obviously the answer to the question which metric was more accurate that predicting output The answer to the question is obvious It was counting the number of stories and by counting the number stories. I mean just doing the work not estimating But there's more Here's a graph that was shared by Cory Foy This is a two-year project and he looked at all of the different story point Bucket so you have zero on the left and you have I think it's Hundred on the right and you have those black lines that go all the way up to the top Those are the minimum and maximum duration in days. The left axis is days for each story for stories of that Particular class so you see that stories of zero points lasted between zero days and 200 days Stories of one point lasted between zero days and a hundred days and So on but look at the small green triangles if you can see those the small green triangles are all between zero and fifty and Most of them are around 2030 there's some that are a bit below and there's one that is a bit higher But most of them are about that time and the green triangles are the average cycle time So regular estimates is the black lines no estimates is the blue line Which one do you think will give you a better prediction of the throughput of your project? i.e. how many stories you can deliver per period of time Just by looking at these numbers one has to say I would take no estimates any day of the week. I Mean come on look at those stories number eight. It can go between five days and what is the highest number? I think it was 280 days Well, that's a lot of variation, isn't it? If I bet the project on one story that is eight points that happens to take 280 days I just kill the project or any hopes of profit at least right Doesn't make sense So which one is more predictable? Somebody whom I dearly respect and thank for having shared his work with me bill handlin at Microsoft Did this experiment? You will see next the forecast of Release date for a project using story points where the story point numbers used were one three and five So this team actually estimated the whole backlog, you know, super waste kind of thing They did it with one three and five and then they projected the release day release date was October 20th 2014 so it was pretty early. So there's a lot of time to go until they reach the final date But that was the release date projected Well, let's do some Excel magic Let's take all the fives replace those with threes and let's take all the threes and replace those with twos What will happen? I'm assuming people who like estimation will say chaos will happen in fact The release date changed to 14th of October So 20th of October 14th of October six days difference in an almost year project What you mean to tell me that story points don't actually have a meaning That we can just use, you know random numbers and it will work Gets better What if we go and we replace all numbers with one that's no estimates by the way We take all the story points out that are not one and those that were two are now one those that were three are now one Those that were five are now one. They're all one What do you think happened to the release date projected from that data? 29th of September I Don't know if this is right or wrong, but what's important is There was a three-week difference in a 38 to 42 week project. That's eight percent That's less than the error of margin for acceptable estimates The margin of error, right? So what the hell are we doing estimating? So obviously bill who was a who's a very kind chap and send this data to me and share his thoughts in an email He said at that point I stopped thinking that estimating was important. I Would have done the same except I had already done it before he did it On his own in his team and you can do it, too You can go back to your laptops right after this session skip lunch do the calculation. This is important stuff guys If you do this and you prove that it works you might save hundreds of man-hours per week from your project When was the last time you had that chance when was the last time you were given a chance to improve improve your project's predictable predictive prediction abilities by this amount and Reduce the amount of work you did to do it I have a lot of stories to share about this and I'll share some of those in the In the workshop on Monday, and of course you can grab me in the corridor and we can chat about this but Let me let me be very honest with you. This is a life-changer Because at the end what I'm trying to give you is something that may if you can pull it off Remove the stress out of developing software. I Used to work with Tommy Tommy was a great developer very dedicated very engaged Completely committed to the company. He was working for but he was in a death march project You know what the death mark project death March project concept is right Who does not know what a death March project is? You know what it is or you don't know what it is You don't know so a death March project is you're in the desert You have this mile posts Right the mile posts are every two miles. You're crossing the desert You're counting the mile posts. You know how many mile posts that are to the end You have water for five days, but you don't know how long the project is going to be you're in the middle of it Five days are gone. You are running out of water. What do you do? Do you turn back? Or do you go straight ahead? That's death March If you go back you die if you go forward you may not die But you don't know right, okay There's actually death March is a term that is used a lot in software engineering and it means that it's it's a project where you can't win There's a lot of overtime a lot of rework a lot of bugs a lot of people shouting a lot of people unhappy So Tommy was in that kind of project He was in the middle of the desert and one day I came to the office and I asked my colleagues. So where's Tommy? Uh, Tommy's called in sick. He's not coming Okay, I'll wait for him. I'll talk to him tomorrow the next day. I asked so where's Tommy? I he's staying a few more days and After a week, I was worried I went to his line manager. I asked so where's Tommy and He turned to me and said Tommy's not coming back He's gotten a burnout and he's in the hospital Tommy was never able to work again That's what burnouts can do to you by the way They can kill your brain It's a self-defense mechanism. Your brain is attacked by your body so that your body can survive and This happens in software projects still today And what I'm trying to do with this with these stories is give you the know-how and The data to back up the change in how you work so that you don't have to go through that kind of projects You don't have to be Tommy. You don't have to have stress at work. You don't have to burn out It's possible to develop software in a humane way in a sustainable way And by the way scrum is not the way to do it There's story points in it. It's the same dynamics the same pressures the same bargaining The same crazy adherence to plan that scrum was there to fight So stop estimating is my plea to you today But it gets better and this is Funny and sad. What's a good estimate? Maybe you have asked that question. What what's a good estimate? How do you classify an estimate as good? Well, some PhD people have written about this in a paper in I think it was a 96 if I believe correctly Steve McConnell quotes the paper in his book and he says a good estimate is Something that is within 25% of the actuals 75% of the time They don't tell you what happens the other 25% of the time Presumably because the estimates can be a lot more wrong than just 25% You have seen in this presentation today Examples that are far better than the definition of a good estimate far better Why are we still using estimates? This is an email I got from a person who read my book and He went home. He did the homework just like I'm asking you to do now over lunch And he said the deviation between estimating the natural velocity would have been approximately 15% lower If we would have used no estimates We've analyzed data from 50 sprints. So not a couple of sprints and at no time at No time not even once Was story point based estimation better than no estimates He did his homework he found this out if you find this out for your project, do you believe you will use story points again? Only for show obviously if you have to This is what this person did and this is the kind of change that can happen to you If you do that work that I just asked you to do and now comes theory. I had a conversation with Dave after his keynote In this conference and he said estimates assume that we are in an ordered Domain if you were at his presentation, you will know exactly what he means by that where reductionism works So reductionism basically is breaking things down into small granular items and then putting them all in order Ie break one project into tasks estimate the tasks put them in a sequence and boom you have your estimate Software however is in the complex and complicated domain where reductionism cannot and We know this from theory Cannot work So if you didn't believe the data those were all case studies those were all examples At least start to question if you know the right theory Because when estimates were created the idea of estimation does not come from software engineering It comes from other industries that much is known It's just been of course adapted and perhaps even murdered by the software engineering profession, but We didn't know about this until the 70s The complex domain did not exist until the 70s and the guy who wrote about this He wrote it in a Swedish Meteorological Scientific paper because if he had written it in his native u.s. He would have been probably shipped to Japan like theming was Because no one would have believed him chaos theory today. However is one of the most fascinating Scientific endeavors that explains a lot more about the world than we even hoped to understand Even Einstein was alluding to the chaos theory even Heisenberg was alluding to chaos theory Even though they didn't have the language because it was only created in the 70s now We know and knowing gives us a responsibility Do you want your kids to work in the same kind of projects you work in today if you don't You have a responsibility to change your industry and just in case you needed more evidence 80% late or failed projects chaos report happens on a regular basis the larger the project the worst the problem is with estimation We all know that but you know that gives you yet more reason to have smaller projects smaller stories smaller deliverables This is from one company where the Diagonal line is perfect accuracy and the crosses are each project and they are comparing the estimated time versus the actual Time and I ask you to look at the crosses that are at the top left of this diagram These crosses are projects that were supposed to last between you know five and ten maybe 15 days and lasted more than 200 days Would you bet your house on that kind of investment? If you win you win nothing you're on time if you lose you might lose everything How about that for a bet? Do you want to bet your life on that kind of an investment because that's what we're doing when we estimate in this company? How many of you by the way the bars are percentage delay to original estimate so the higher the bar the worst the estimate How many of you would Consider project number one a success. Please raise your hand the others don't have an opinion Not enough that okay, that's a good answer, but that's not what I'm asking if you had to choose Okay, so which one was let's do this which one was the most successful project number one or number 17 Number one that would be the consensus by the way That's what the chaos report uses for the definition of success So number one the project number one. This was actually the company where I worked at project number one I have no clue what it was. I don't remember. It's irrelevant project number 17. However save the company Project 17 created a new business model Created new markets created new jobs created new revenue, and it was 200 and almost 50% late from the original estimate So just to be clear being right on the estimates even if it were possible is irrelevant for your business So this is what I'm asking you When you are asked to estimate think about this It's like trying to bend a spoon with your mind Except there's no spoon. So take no estimates and experiment This is my plea to you today Experiment and learn. There's a lot more. We need to know about our industry, and we haven't even started Thank you very much If you through today, you can get the whole book for free just go to that link and Sign up and you will get the book for free the book explains a lot of what I'm talking about But it also tells you a story of a real project and how you can take a Obviously failing project and turn it around with the help of the ideas that I've shared with you today The whole story is there. You can see what I think about this how I've used it in the past And of course, hopefully you will share it with your friends and start to improve our industry So just a second wait for the microphone. We have a question over here The microphone is Coming. All right, you can also tweet me by the way I'll be around and it would be a pleasure for me to talk to you more about these ideas and Share some of the stories of how no estimates came to be Hello. Yeah, thanks. Was cool. It was a nice session. I enjoyed My question is like if we don't have Estimates and plan whether it whether in a gel or a waterfall, right? You have stakeholders like marketing sales and customer who wants us to give kind of a roadmap, right? So how do I present to them and how this is one question the second question is How should we create the agile contracts in that case? Okay, so the question is if I understand correctly How do you coordinate the work across multiple departments if you don't estimate up front? Yes, okay, so this question is it's a very important one that everybody asks So thank you for asking that but it's also a question that does not tackle the real problem The real problem is that we want to use no estimates But we're pretty fine with marketing using estimates. That's their problem Or finance using estimates. So yeah, if they want to use it, why not? But that's not how you change your industry How you change your industry is if you focus on creating value end to end So my answer to that question is you can use prediction to know what will be ready at a certain point So there's forecasting in the book. You can read about it. I explain how to do it You can even given ranges you can you can even fake. It's a scientific answer It's not but it's a forecast and the idea is that you can do that But you don't need to if you bring the whole value chain into one team See the thing that agile is about that no one talks about for some reason is about changing the industry It's not about changing how you code We've heard many talks here today and the last few days that Challenge us to understand what value is to interact with our customers in a more cooperative way to build partnerships As Evan talked just before me And that's what we need to start doing. So the answer to your question is use forecast if you must that's better than estimating But try to get rid of it by including those departments in the life cycle of your product or project development And your second question was he's about contracts say for example Like if I if if customer has to pay like every quarterly or half yearly, right? He needs a kind of medium through which he can able to assess So when we say okay customer a this is the feature a we we are trying to you know groom into 10 stories And these 10 stories we forecast that it could be available by quarter three Can we have a contract based on this means it it looks Not very tangible or convincing to customer also, okay? So that the contract story is very complicated, and there's a lot of ramifications I can't tackle contracts what I can say is that if you deliver software early and often and Regularly at some point your customer will not ask you for any estimates will not even ask you for any Forecasts he will just ask what can I say sale next quarter? And that's all he wants to know because at the end the customer needs your client needs to do the same as you need to do Which is to understand what is needed from their customers perspective and create a package. They can wrap up and sell Right. Yeah, that's what they need to do So they don't the contract is a part of the trust building with your client and you can build trust in many ways So if you have a contract with time and materials, which was one of the things that Evan suggested You could start with that and then turn it into a win-win Partnership by doing for example outcome based contract, but that's I'm out of my depth here I don't know about contracts in yeah, but I know it's possible because it has been done And there's a fellow in Japan who was giving a talk a few months ago about what he called the subscription based model For software development where people subscribe to your capacity to deliver software Rather than tell you these are the requirements delivered by you know next month or whatever Yeah, we can continue to talk after but it would be good to give others also an opportunity Thanks one over there so my question is so we had a Person days estimation in the conventional system now in a giant spoke about story points Now we are saying no estimates, but we are still looking at number of user stories that we deliver in the form of So end of the day. It's like just changing the Unit of measure, but some are other way. We are still talking about estimates, right? So that's why the book is so important. There's a very crucial difference between estimating and forecasting It's in the dictionary and if you study statistics, there's a huge difference, but the idea is that Forecasting, which is what I promote and think is a good idea is based on actual data And there's a model the one that I use that tells you when you cannot know what the throughput is and by the way What I'm talking about in no estimates is measuring throughput not estimating the size and Sequence of the work because what story points does is to try to remove the sequence of the work It does it very badly because it is together with the idea of commitment to a sprint Those two things shouldn't go together That was a very basic mistake done at the beginning of scrum that many teams still use it But in what what I'm proposing is you use forecasting of throughput Throughput is not defined by what you think it's defined by your system So how many times the team is interrupted? What's the time difference to the other teams if it's a distributed environment? How good are your tools your throughput is affected by everything that you work within and by measuring throughput you're measuring the capability the real Observed capability of the system of development. You're not estimated You're observing data and then you're extrapolating i.e. Forecasting based on the data you have and the model sets out very clear rules to when you cannot say anything about the future Right you remember That's one of the questions that that I ask if I look at the data of your team The first question I ask is can I use this data to forecast the future throughput? And there's very clear criteria for that and if I can't then I can't I'm not going to try to guess That's it. I don't know Right Just how do we fail using this technique? Sorry? How do we fail using this technique? Okay, so This is a very good question. How do we fail using this technique? So the first failure mode is if you focus on the throughput and not on the value That's one failure mode The other failure mode is if you try to deliver a monolithical Piece of work like a story that takes a month Instead of chopping it down into maybe two-hour chunks Somebody was talking about in the keynote this morning that they chopped their work down to 30 minutes That's smart because you have more data you reduce your risks and you get feedback faster and agile is all about one thing Getting feedback faster So if you're not interested in getting feedback faster That's another failure mode and there are relationships between vendors and buyers and between teams and Management that are in that state So those are just some failure modes But the failure modes that affect this technique are the same failure modes that affect any other technique including estimates In the software development industry So in if you're if you're saying is no estimates full proof No, it ain't But is estimation full proof. It's even less That's the irony of this Thank you Sorry, could you repeat again? So my question is around the fact that estimation Per se may not be important, but the essence of estimation or the process that you go through Involves the team coming together and understanding what is the definition of done for a user story having a common understanding of what needs to be done That is still important The end result which people quantify it as this is eight points is 20 points is 25 points Doesn't really matter. So I I have written a blog post you can go to my blog and actually see that I've written a blog post the title is something like Estimate but throw the numbers away, which is basically what you just said, but I wrote that blog post more than five years ago I've since I've since discovered That actually there are much better ways to come to a common agreement that do not involve the antagonism bargaining and blame games That go with estimation because I mean let's be honest if you're in a team where you have one senior that is very well Respected and you have a bunch of other not so senior people working with that one senior when the one senior says this is a five No one that wants to be respected will say it's a 25 or a hundred Are you crazy? How the hell do you code boxing gloves? right and These is part of having humans in the system This is not how it should be or how it was meant to be this is just how it is We can accept that it's a problem and move on and not lose anything in the process. I hope I could convince you of that Or we can or we can wait a second because I can't take any more questions or we can just say We'll do it as as the same way as we've always done it I'm sorry. I will have to take a break because I have my challenge here So I would like to quickly ask these three questions again Who believes now not before the presentation that estimates are actually needed to be successful at software development? Could you please raise your hands? at least one part is okay, then Can estimates be accurate? You still have that You're not just saying that to mess with me, right? okay, and Who wish? Who of you wish that you could work with no estimate starting on Monday, okay? That's a lot more than 50 Thank you very much and please come up and talk with me either now or during the day. Thank you very much