 I've been working with software since 1993. I've worked with a lot of recognizable brands, including Amazon.com, eBay. I've worked with ThoughtWorks, HomeAway, a number of small startups, and I have been an individual contributor and a manager of development teams during that time. The first thing I'd like to do is take a little bit of a straw poll. So everyone in this room who has to provide estimates as part of their daily work, please raise your hand. Okay, quite a vast majority of you are required to provide estimates as part of your daily work. Okay, how many people in this room require estimates of other people? Raise your hand. Okay, quite a lot of you, good. Okay, so I hope that the content of this talk will be relevant to both groups of people, and there was a little bit of crossover. So some of you are passing on the pain, it looks like. So I wanna first give a little background on why I decided to talk about estimation. As over the years, it seems to me like estimation is a very big pain point in software projects. It seems to be a source of great tension between developers and the business, and developers and managers, and between managers and managers in the business. And so what I hope to do today is at least start a conversation about an approach that I think is a little more human around estimates. As a project manager, I get asked one question more often than anything else, and I suspect that most of you will identify with this question. When can I have it? How many of you have been asked this question? Okay, how many of you feel completely comfortable answering this question? One person. Okay, do you wanna tell us why it's so easy for you to answer that question? Okay, he said it's black and white, you can have it or you can't, okay. So the question wasn't whether you could have it, the question was when can you have it? Sorry, go ahead. Right, okay, the gentleman in the back is a big proponent of deadline-driven development. Okay. Okay, so let me ask the rest of you why you don't feel comfortable answering this question, what makes it so difficult? Is anyone willing to brave an answer? Go ahead. Yes, that is a very brilliant answer. So there are a lot of dependencies, it depends on many, many factors, and it becomes very difficult as complexity increases to provide any kind of valid estimate. Okay, so let's do a little bit of role-playing here. I've just become the CEO of a brand new company, it's a fake company, I'm not really a CEO. Just in case somebody twitters that. Okay, so I have done the market research and I've come up with, in my fake company, the next cutting-edge product, okay? So I'm really excited about this product, I think it will change the face of the marketplace as we know it. I've got my backers and I've hired every single one of you to help me develop it, okay? Are you ready? All right, so the product itself is drum roll please, sliced bread. Okay, so I've decided I want to develop a loaf of freshly baked cutting-edge sliced bread, okay? And this is going to be the signature product for my company, half-baked, incorporated, okay? So of course, as the CEO of the company, I'm really excited about my product, I only have one question, one burning question of you guys. In my mind is the CEO, you guys as my developers are here to answer one question, when can I have it? Okay, so I would like you guys to, you've got sort of some groups at your tables, so I want you to spend a minute, you in the back kind of split up and itself organize into groups, and I want you to discuss this question, how long will it take you to make me a loaf of sliced bread? Okay, and I want you to try to come up with an answer, how long will it take? Okay, so I'll give you about a minute to discuss with your groups. There's a question back here. Are you asking that question to the right audience? I don't know people here who would be having the domain knowledge here. Yeah, okay, so there are a lot of people that don't know how to bake sliced bread, so just do the best you can. If there's somebody at your table who knows something about it, try to make use of that skill, but just take your best guess. Only one loaf to start. I'm looking for my first loaf of sliced bread. I'm looking for my first loaf of sliced bread. To discuss, so that should be time enough, right? Yeah, do you need more time? Go ahead, the infrastructure. So let's assume you have all the tools that you need to make. Okay, I'll give you another 30 seconds. No, this is not a product that exists in the marketplace, so it has to be made from scratch. Okay. So the second one is we need to get the resources because none of us have created that, so we need to get the resources or get the enough trade. Okay. So we have to take one of them to one day. Okay. And then provide the value, the actual cost of getting the best, which will take 20 minutes to be served, but this way, there's a deal. Okay, that's good. Okay, fine. Right, right, okay. Okay, does anybody else have another estimate? Okay, go ahead. One day. One day, okay. Why did you come up with one day? So we'll gather all the requirements and then outsource it. Yeah, there's a... So I'm guessing you're saying that you think that outsourcing makes things faster? No, somebody has the expertise to give it to them. I mean, there are a lot of bakeries. Uh huh. Okay, okay. Okay. All right. Okay. Okay. It looks like all of us... It looks like all of us... It looks like all of us... It looks like all of us... It looks like all of us... But they really want to know what type of work that you want to do. Okay, okay. It's not that... Fair enough, you don't know what type of work I want to know. Okay, so I'm going to help you guys out a little bit, because it seems to me like you're struggling with the domain knowledge. Um... Um... Um... So... Um... I'm just going to break it down a little bit. Um... And just go through the tasks that we expect for making all of Brent. Okay. So you have to make the dough. We're assuming we have all the ingredients. We're assuming that you guys know how to do this. Right? Okay. So you have to make the dough. You have to let it rise. You have to preheat the oven and you have to bake it. All right. So, given that, if any of you want to revise your estimate, this table you set in two days, can you stand by your two-day estimate? Yeah. Yeah, yep, yep. Okay. One day roundup. Okay, fair enough. Okay. I think if it's... Pass me the fresh. Half a day. Okay. Okay. Anybody else? Yep? Since it's a new company, the first day will be learning how to make it. Yeah. The second day will be actually making it a package. Okay. So you still think it's two days? Okay. Fair enough. Okay. Six hours. Six hours. Okay. Do you guys feel really confident in your estimates? No. Why not? Okay. So, I tried to put myself in the mind of a developer, and I thought, okay, two hours for this. Right? Okay. Maybe two hours. Some of you, you know, said more. It probably would take more time. Maybe half a day. Right? So, let's say we go ahead and stick with the half-a-day estimate, for example. And we say to me, the CEO is going to take us half a day, for two days or whatever the estimate is. And we start working on our loaf of bread. So, as we're building our loaf of bread, something happens, and I realize that I forgot some stuff. Right? Okay. So, here are a few other things that need to happen before my bread can be complete. So, I realize I don't have any of the ingredients, sugar, eggs, flour, yeast. I don't have anything. I have to activate the yeast. I have to knead the dough, and I have to slice the bread. So, I started work. I've given my estimate. And all of a sudden, these new things that I just didn't think about came along. And I realized my half a day is just not going to be enough. Right? Or even my two days might not be enough because, you know, I live in the wilderness in Alaska. In order to get supplies where I am, it might take me a week. Right? Et cetera. So, there are a lot of different reasons why that estimate might be wrong. So, I have to go back to the CEO and say, CEO and say, you know, okay, maybe a whole day. Maybe, for those of you who said two days, maybe it's now four days. Right? Okay. So, so what happened there? What happened? So, yeah. So, the most important thing here is that as soon as you gave an estimate, an unspoken deadline was set. Right? It doesn't matter what that estimate was, you created the deadline by giving that estimate. Okay? So, the next thing that happened is, you didn't meet it, and the CEO is disappointed and you feel bad because you didn't make your deadline and there are some things you didn't think about. Some of you tried really hard to think of all the possible contingencies and maybe you met it, maybe you didn't. But, I'm going to give you a little hint that your day is about to get worse. I don't know if you can hear my music in the back there. Okay. So, there's been a zombie attack and the world, we're now in a post-apocalyptic zombie world. Right? There are no more stores. You can't get the ingredients that we're looking for for our slice of bread because the stores are obliterated. The streets are filled with roving bands of zombies and there's only one ray of light in all of this. And that is that Half Baked Incorporated is still open for business. Okay? Somehow we all managed to survive. Say again? So, we sabotage... Yeah, we might have, you know, caused the zombie attack. That's very possible. No, I don't think we're that evil. We're misguided but not evil. Okay. So, we're still open for business. You guys are all still working for Half Baked Incorporated and now, guess what? That loaf of freshly-baked bread is much more valuable. So, me as the CEO, I'm thinking I really want this loaf of freshly-baked bread. However, all the ingredients have disappeared. So, what you guys estimated in terms of making this loaf of bread has sort of gone up in smoke along with the zombies earlier. So, what we've decided to do is look at the basic ingredients that make up that loaf of bread. So, here they are. Water, butter, sugar, flour, yeast, eggs. Okay, so, what I need you guys to do, the question is exactly the same as it was before. When do I get my freshly-baked loaf of bread? Because now I really, really want it. So, you need to answer the question again. But, you need to come up with all these ingredients from scratch. Okay? So, there's no butter in the stores. You have to make it from scratch. You might find a cow roaming around or a chicken. So, you've got to grow the flour. You've got to forge for seeds. You have to figure out how to make yeast. I have no idea how to make yeast. I think there's some fermentation process that happens, that's all I know. And you've got to grind flour. You've got to mill it. You've got to make the sugar. Okay, so, I'll give you about a minute to talk amongst yourselves and tell me how long it will take me to get my loaf of freshly-baked bread. Because that's all I care about, is my loaf of bread. No, you can't get rid of the eggs. It's egg bread. I need the egg bread. I need the eggs. Oh, you have a question? We have all the time in the world to monopolize. Yes, we have all the time in the world. Time is no object. All you have to do is come up with an accurate assessment of the amount of time it will take. Do you have people around to consume your bread? Well, there's us, at least. So, there can be a multigrain bread or any flavor of a bread that we can provide. As soon as we draw it, Devlai would be difficult for us to come up with an estimate. Okay, so the question was can the requirements grow? Can it be a multigrain bread? Can we add things to it? So, I'm sensing a bit of scope creep here. So, as the CEO of Half Baked Incorporated, I'm going to be very clear about this. I just want a loaf of freshly-baked bread that includes these ingredients. And as quickly as you can get it to me because I've got a market out there waiting for this bread. Ten days. So, gentlemen in front says we can't estimate. How many of you guys agree with that assessment? Yep, yep, yep, yep. Okay, about half of you. So, the rest of you, you feel like you can estimate this? Yes. Okay, and you would feel completely confident in those estimates? No. You would stand by those estimates no matter what. So, as soon as you give me a number, that is a deadline. Yeah, go ahead. Yeah. So, these assumptions have to be validated here. If the assumptions are true, the estimate will be accurate. So, it does not mean that the estimate is one time accurate. We need to be validating the assumptions. So, that is the key thing. So, our assumption is, should we have the one which takes maximum lead? For cultivating sugarcane and making it sugarcane. So, we took six months for that. Sixty to three, one eighty. And that half here for making the break. So, one eighty days plus half day. Okay. So, we... Sorry. My assumption is you got one. Yeah, yeah. Yeah, yeah, yeah. Okay, so the gentleman in the front here said that one hundred eighty days plus half day. Yeah. Right? It seems kind of random, but do you feel confident in that assessment? No. Okay. So, he said one hundred eighty days plus half day because there are a lot of assumptions that go into it. They chose sugar as the long pole in the tent, so to speak, because you have to cultivate the sugarcane, you have to process the sugar, et cetera, et cetera. Okay. So, what he said to me, I don't know if you heard the whole conversation, but he explained all this to me. And he said, you know, there are a lot of assumptions that we made in order to get to that figure. Okay. So, me as the CEO, you know what I heard? Blah, blah, blah, six months. That's what I heard. Okay. And it's very true, right? So, as the CEO, I have a one track mind. I just care when I get my product, right? I want something out there cool to show people. I want it to seem like I'm, you know, producing stuff in the world. I want my revolutionary sliced bread to be out there being eaten. So, when I ask the question, how long is it going to take you to get me my sliced bread? All I care about is how long, right? So, that's what I'm listening for, right? Okay. So, I'm probably being unfair to some CEOs out there. I'm sure that some of them are a little better than this, but I can tell you I've met many CEOs like me, and they're not, right? Okay. So, moving on. As we said before, the question remains the same one. Can I have my sliced bread? So, the point of all this is that this is extraordinarily difficult to do. We're going to revisit the sliced bread in a little while. But, and you're asking yourself what does bread have to do with software development? I thought it was a really good metaphor because in a way, when you think about software development, what we do manages the entire supply from scratch to market, right? In some cases, we might put together collections of things that already exist out there to make a finished product, but actually most of the time we're building from scratch. And teams of developers often specialize around the individual ingredients, right? So, developers might know all about the flour, and the flour is the foundational system for building the bread. But there might be, as somebody else here pointed, very wisely pointed out, there might be 10 different kinds of flour. Rice flour and buckwheat flour and whole wheat flour. And each kind of flour that you use has a different effect on the end product. So, as a developer, if I'm asked to do an estimate of the whole loaf of bread, I'm going to be thinking about my type of flour, right? And, you know, whatever experience I have in actually building that whole loaf of bread. So, the next thing I want to do, a slightly more serious exercise, not involving zombies, I just want you guys to talk amongst yourselves and think about what are the things that influence project delivery? What has an effect on whether or not you can actually deliver a project? Just list things out. So, go ahead and you can discuss and then we'll start listing things out on this flip chart here. Just start calling some things out and I'll write them up here. Resources, yep. Dependencies, okay. Technical expertise. Okay, unplanned leaves. Distributed teams. Yeah. Software, hardware environment. Company processes, yep. Go create. The CEO. What else? Assumptions. The main expertise. Okay, what else? Budget. Yep. What else? Time. I would say project planning. Motivation, yep. Project planning, okay. Okay, project planning. Estimates. Risks. Okay, what else? Risks. It's kind of what we're listing, right? Okay, so risks, what else? Environmental conditions. Environmental conditions, okay. Anything else? Technological limitations. Technological limitations, okay. Acquisitions and mortgages. Acquisitions and mortgages, that's a good one, yep. What else? Okay, we have a good list. Do we have any project managers in the room? One, two, three, okay. So this is the list you guys came up with. So I'll show you my list in a minute. So do you project managers, do you feel like you can accurately plan for all of these things? Most of you said no. Who said yes? No, okay. They clearly said, I said no, okay. Did anyone say yes? Okay. Is there anyone in the room that feels like you can accurately plan for all of these things? Good, all right. This is the list I came up with. All right, so let's just go through some of these. That wasn't me. Okay, so team collaboration. So your team's ability to collaborate has an effect. Complexity, skill levels and experience. Somebody said that. The health of the team, the philosophy of the team. The weather that day. I'm not joking, the weather really does have an effect on whether you can deliver it. Semantics, how you interpret things. Because we use words in order to communicate. So semantics is basically very important. Phone connections if you're on the distributed team. Project tools, politics at the company, local, state, national and international level. Okay, so I'll give you an example of the international level if Obama decides to change immigration policies. And, you know, an engineer can't cross the border. Then we're, you know, down on engineer. Crime rates, finances, religion, education, economy. We're the developer, we're the CEO. We're the developers' kids. I'm sure many of you can identify with that. And of course, zombie attacks. Right? Okay, so I'm going to ask you to think about one more thing. What can a typical team member control? Okay, you don't have to tell me. Just think about it. Spend about 10 seconds writing down things if you want to just, you know, think about it. What can a typical team member control? Here's my list. And this is arguable. So, as a typical team member, you can most of the time control whether I actually come to work. And I can pretty well control the task I'm doing right now. Pretty well. Not always. Sometimes those tasks get out of hand. Okay, so do you see where I'm going with this? Okay, we are asking engineers who have control over these things to estimate based on this. Right? Okay. It begs the question, can we estimate it all? Right? Is estimation something we can do? So, human beings are not hardwired to be able to estimate in exact units of measurement. Does the gentleman in the back have a question? Yeah. Right. So, the gentleman over there by the pool, he said, when we ask for estimates, we're really only asking a developer to estimate for these. We're asking them to account for all of this. Right? So, which in a way, beautifully makes my point. Right? So, even if you are aware and you understand that developers don't have, in any way, shape or form any control over this stuff, the estimate that you get might account for that. Might. Right? And then, it's not usually inaccurate. And the reason why is because, as I was saying, human beings are not hardwired to estimate in exact units of measurement. Okay, and we'll do an experiment later to show you exactly how not hardwired we are to do that. But, I want to kind of talk about what we can do in our estimates. We can learn to account for these things, collaborative effort, complexity, and relative size. But we can't do it individually. We can't do it as individual team members. We can't do it in silo teams, and we can't do it separately from our whole team. Okay? The only way we can do this is collaboratively. Okay? That sounds like an abstract concept, and it kind of is. Okay? How do you learn to estimate for your whole group? How does a whole group learn to collaborate so well that as a group they can estimate? Well, you take an empirical approach. So, did anyone take the empirical workshop that happened earlier? Or did anybody in this room? There was one that happened earlier today. Yep. Do you want to hazard a definition of empirical? Do you want to give a definition of empirical for the group? Okay, I'm putting them on. It's okay. I have one. It's okay. You don't have to do it. All right. So, this comes from the Princeton University Electrical Database. Empirical means based on experimentation and observation rather than theory or logic. Okay? So, why not logic? Why is logic not part of empiricism? Go ahead. Because it is based on testable data. I need the data and whatever the data shows me, that is what we have to consider. Sometimes the logic fails over there. So, we are going by what the data is showing me. That is how the data is. It's a very good answer from the... Another thing is when we talk about experimentation and observation, all those things factors in what we all discussed over time. Over time. So, these things average up. Yes. We cancel those. Yeah. So, the only way you can account for so much complexity even within a team is by observing the patterns over time and acting on them. Right? So, where do we start? We start with our best guess. So, we start with an initial guess, but that guess is only a foundation from which we operate and then we observe the result and then we try something else. Right? So, that best guess is often based on intuition. And we assume it's not going to be accurate. Okay. So, in the spirit of empiricism, I'd like to do a little experiment here. Okay. Just a tiny little experiment. Okay. Any of you recognize this bottle? It's from Ikea. All right. I call this the guess what we all suck at estimates experiment. All right. So, how much liquid does this bottle hold? 100 ml. 100 ml. Okay. I'm going to start writing this down. Anybody else? We have not 25 ml. Okay. 100 ml. We need 25 ml. What else? How much? How much is the same thing, right? How much does it hold? 100 ml. 100 ml. Okay. What else? 200 ml. 200 ml. Okay. What up? 50 ml. 50 ml. 25. 200 ml. 20 ml. Okay. 220 ml. All right. All right. Okay. So, all right. Do you all feel confident in your guesses? You said yes. Okay. What was your guess? 150 ml. What was your guess? 100 ml. 100 ml. Okay. So, all right. So, the exact amount is 150 ml. Okay. But you all made a fatal assumption. And that is that I, as an American, understand remotely what milliliters means. Right? As an American, we don't measure in units of milliliter, right? So, 0.1. If you gave me these estimates, I wouldn't understand. 0.2 is, we had, how many? One, two, three, four, five, six, seven, eight. Eight estimates. There was one person who got it right. So, probably based on experience, I'm guessing. But most of you didn't. Right? So, the question. So, if I then asked you, if I then told you that this bottle is 500 ml, would you be able to roughly guess what this bottle is? Yeah. Okay. Relatively. Yeah. So, you can compare this bottle with this bottle. Right? Okay. So, as human beings, the way we are, the way we intuitively measure is by comparing, either with based on experience or based on the facts at hand. Right? But, if we try to guess exact units of measurement, we're almost always wrong. Right? Unless we get lucky. So, can we actually estimate this loaf of bread in a post-apocalyptic zombie-infested world? Can we do it? I'm seeing a lot of people shaking their heads. Okay. So, I've taken a morale hit in my company. Okay. So, all right. I believe we can't. And I think the way we do that is we change the question from how long does it take to how hard is it? Okay. There's an important distinction here. Okay. If I say how long does it take, then you guys are going to try to give me an exact answer. It's human nature. Right? It's a very directed question. You're going to try to give me a directed answer. How hard is it that it's a little more ambiguous and it can give us a foundation that we can use for our empirical approach? Okay. So, I need a brave volunteer. Okay. Go ahead. Can you come up here, sir? So, we're going to talk about estimating effort. It's okay. I promise. Yeah. No. So, I have a very important question for you. Okay. Have you ever created a chicken egg from scratch? Okay. No. Okay. That's okay. That's okay. So, okay. Good one. So, I have a question for you. When can I have this egg? Yeah. What? Okay. Let's say I'm giving you the chicken now. Chicken's not required to be pregnant to lay an egg. Point of fact. So, it's an actual mistake though. I mean, don't make knowledge. So, I'm going to show you the data. Yeah. Another story that I'm talking about. Okay. Okay. Sunil, that's your name, Sunil, right? Sunil. I'm a CEO. I'm losing confidence that you have the skills to get me my egg. That's business CEO. You know, she's living a life. Okay. So, I don't need historical data to decide. Or, I mean, another story to decide it. Compare it and decide it. Okay. Okay. Fair enough. Okay. So, I'm going to, Sunil's having trouble answering the question. Right? And I don't blame him. I absolutely do not blame him as me. As a CEO, of course I'm going to blame him. But as me, I don't blame him. Okay. So, I'm going to go to my other CEO self and give that person a piece of advice to change the question. Okay. You have a chicken with everything it needs to lay an egg. How much effort is required on your part to deliver me an egg? Yeah. I didn't come back to say anything. You want to have something to compare me. Well, I don't know. I mean, you've never actually been involved with a chicken laying an egg before. Okay.