 Hello! Wake up! Hi everybody! Are y'all out there awake, ready to have some estimation fun? Woo! Let's hear it! For estimation! Woo! Okay, we're ready, I think. Do you know any more like energizing exercises we could do? Energizing exercises. Everyone raise their hands now. Yes. Okay, good. One more time. Alright, I can really feel the enthusiasm here. Okay, perfect. Okay, go ahead. Alright, I put in this in lock right. So, some short interactions here. First of all, Shannon, the instigator of this thing. Do you want to switch? Oh, here we are. Hi, I'm Shannon Vitas. I have been working with Drupal for a few years now and doing project management for about six years and I'm currently taking a short break between jobs to do some work on Drupal 8 and work with the initiative owners and come talk to me afterwards if you have questions or you want to talk more about project management. Hi everybody, I'm Mathias Axelsson from Happiness Web Agency in Stockholm, Sweden and I'm the CEO of Happiness and also a certified scrum master. Alright, and I'm Jacob, the estimation guy who was invited by Shannon to take part in this and I happily accepted. I've been doing Drupal since 2005 and I'm one of the co-founders in Node 1 now, Wunderkraut. Alright, so this is a panel originally but we figured it would be better to do some talk in the beginning to maybe raise some topics and get some ideas going. So when we're doing our part here, if you come up with any ideas, questions or anything feel free to tweet. Here's the hashtag. At the end we're going to go through the question list and we're going to answer your questions as a panel. That's the second part of this presentation. In case you can't see it, it says Dress Demate, like Drupal Estimate. Yeah, Dress Demate. Alright, so why are you here? Okay, so when someone asks you what, you know, to do an estimation for me how many people have this reaction right here? Kitty's scared, not good. How many of you have this one? Sad kitty, doesn't like estimation. What about this one? I'm going to straight up murder you if you want me to do an estimation. No, well, I hope it's not any of those. But if it is, guess what? You're in the right room. We're here to talk about this. And at the end, if I don't skip over the slide, there it is. At the end, what we really want is for you to feel like happy kitty. Doing your little estimation dance. I know what I'm doing. So we're going to cover what is an estimate, what not to use them for, what affects them, and how you can do it. And then at the end, you get to ask your questions. So remember to use that hashtag. We're going to be checking the feed so we can get those questions. All right. So what is an estimate? So the technical explanation of an estimate is it's a calculated approximation of a result which is useful even if input data can be incomplete research. And like we said, essentially with the little information you have try to come up with a likely scenario. And we believe that a lot of you are project managers. Is that true? How many of you are project managers? All right. So majority. So you're probably, this is you. That's the idea anyway. You're a lady. Deal with it. Yeah. And these are the questions you have. What is an estimate and what advice can we give you? All right. So estimates concern work, some kind of like the effort required to finish a project. And estimates should encompass a number of tasks, not just the work itself. So if you look at all the bubbles here and arrows, first of all, we have the bubbles to the left, the loop, the iterative work you do including requirements, clarification, writing use cases to your refinement, theming development, and testing. And then when the project is over, you also need to figure out how to put a site live, deploy it, and you also, in many cases, you also need to provide training for the customer. And during the whole course of the project, you will try and work required as a project manager to provide guidance and to make steer the project. All right. So onto requirements, requirements clarification. So one example how you can clarify requirements is by working with use cases. Now a lot of requirements come in terms of user stories. This is a format you guys are familiar with, user stories. All right. Can we see some hands? All right. So I don't think they need any further explanation. User stories are pretty brief. They talk about who does something, what they do, why. Use cases on the other hand, they'll be more extensive. That's what a developer would need to actually to clear out all the ambiguities and all different ways you can interpret the requirement. So that's something you probably need to do a lot of the functionality in order for developers to really understand what the customer needs to do. And this is going to take time. This is something you need to estimate for. Likewise, the UI often starts with a simple mock-up, sometimes with a wireframe. And over the course of the project, as you understand more about the product requirements, they go from wireframe to maybe a comp to maybe a finished mock-up. The real work. This is what we usually think about when we estimate. This is stuff we estimate, but it's just a fraction of everything. The coding and the theming and everything that's included there. Testing. Testing can be as simple as having someone click on the website and see there aren't any bugs or any defects. But in bigger projects, it becomes more and more important to do automated testing. Big products become so big that you can't possibly, as a human being, figure out every combination of content or pages or anything. And you also have what we call regression errors, bugs that reappear or appear later. Something that used to work suddenly doesn't work, because it did something that broke the functionality. The best way to avoid that is to have an automated test. An automated test is a simple piece of software to make sure that the other piece of software does what it should. And again, this is something else you need to take into account. Finally, training. Training your customers. Training editors, training staff, anyone who's going to use the website. And what they need to think about. And this slide where you can see it's also including deployment. And even if you're not deploying this site, even if the customer's IT department is doing it, they're all going to have tons of questions. You're probably going to have to be like a Drupal sysops consultant. And that's time that's going to take time, and you need to take that into account and charge for it. All right, so the PM part here. So I'm talking about this, because I've seen this happen a lot in the buffs that I've been holding. You can't account for PM time or even a percentage of it. And I'm here to tell you that's a really big mistake. That's really, really common in estimation. Always include your PM time. It's valuable. It's going to keep everything on track. Account for it and plan for it in your estimations. And we're going to tell you the percentage that we usually do later on in Jacob's part. All right. My clicker is... Not works. All right. So why do we need estimates? Well... So these are the questions that prompt the distinction. How does an estimate help you plan the project? What does it communicate and how does it help your budget? Now, estimates are for good and for bad. And we're going to get to the bad later. A use for planning, budgeting and communicating regarding the project. Planning is the most obvious and common use for estimates. Say, for example, that... Say, for example, that your estimate is probably going to take 20 hours. You know that each of your team members can work 32 hours per week. That would mean that they could work for essentially five days. And including that is also part of management time. Like this. Oh. You got a calendar plan. Perfect. That's one way to use your estimate. It will give you an indication for how many days you need to book your resources. Your developers, your testers, your theme or your designers. And that's also often assuming that the project would be like a straight road. You know, the pedal to the metal and just raise ahead. But in reality, projects you should turn out more like this. Like an old packet trail up in the Andes. With mudslides, you know, and holes and even worse. And more often than we'd like, it looks like a traffic jam in New York. These are things that happen. And a lot of many estimates and many project plans don't take this into account. And how can it happen? Well, a lot of projects are scheduled like this. This is a small team at a company. And I'm managing three projects in parallel. Project B, for example, is planned to take seven days. But a week later, a week and a half later, customer comes back and tells them, I need a few major adjustments and this is super important. So your team has to work weekend. Meanwhile, project A is running. The green one here. And everything looks fine, you know. Everything's running on time. You're really happy. Until like two and a half weeks later, it turns out you have tons of bugs. And they are business critical. Maybe it's an e-commerce site and they're not making any money. They're losing money every second. And meanwhile, you're running project C. And with this kind of planning, what's going to happen is you're going to have a very tired development team eventually. Yeah. And they're not going to be that cute, I think. So budgeting. So here's the second part. The customer. What can you get and for what money? Essentially, what they want and what they're prepared to pay. In a lot of cases, customers know what they want. But you don't know how much effort it's going to take until the end of the project. You don't know how much it box weighs until by the end. It makes it easy for the customer to put a price tag on it. You can actually have an idea of its worth to the customer in terms of business value. But that is not always the case. If the customer can provide an estimate of how much value each of these features provide for them, either halfway. I mean, that's good progress because I'm going to show you later what that's used for information. And suddenly, we can weigh the business value against the budget. All right. This doesn't tell us the whole picture because you actually need to estimate how much time it's going to take. And for that, you need to use your team. Your team can rate the feature in terms of how complex it is, how familiar you are with it, and whether it's depending on other functions they've built or external APIs or such factors that will influence the time it takes to complete it. And you can post it as a yellow post it like I said there on the side of the box. So they think that this box here, which has a business value of 100 euros, will take 80 to 20 hours to deliver. So with this information, we can suddenly compare the estimated effort compared to the money they have. And that is provided. Provided this information is pretty accurate. You can make some interesting... You can make some choices here now. If you put the value in the effort slash risk on two axes here, now the effort to risk, I'm going to explain that briefly. The more effort something takes, usually that also reflects at higher risk. So something that takes two hours to make usually has a higher risk of running over that estimate proportionally. You usually want to go for the things in the sweet spot. The low-hanging fruit, the things that are easy to do and deliver a lot of value to the customer. And reversely, they're the stuff you want to avoid, the red sector. The stuff that's expensive and doesn't deliver a lot of value. That's stuff you would rather actually avoid at all if you could. Then you have this stuff that actually brings a lot of value but a lot of work. But on the hand, if there's a lot of work, there's a lot of risk, which means that it could potentially run over a budget. So using this grid here, this quadrant, you can rearrange your features and get a visual idea of what stuff you should go after first and what stuff that's most, you know, could potentially run over a budget. Generally, you want to go for the low-hanging fruit and for the hard stuff first. So the top left and top right are probably the ones you would want to plan early in the project and get done with. So you can focus on the riskier and less valuable stuff later on. But this assumes that the team is actually pretty sure. But they won't be sure until the end of the project. They don't have a clue about the difficulty. And you want to turn the think into what they know, not what they think, but what they know. So finally, estimates aren't just numbers. They aren't just lists of numbers and features, and they actually tell a lot more. Your team wants to make sure the customer understands what they're asking for, and the customer wants to make sure that your team understands what they're expecting. In situations when you get an RFP, answering with the lowest estimates, you should not, what you should do. You should actually come up with a number where the customer feels it's reliable. The communicates that you understand, the complexity of the task, and that you have the required experience to completing it. And if you're in good terms, you have the same estimates. But now, everything goes well. You're friends with your customers. So yeah. I'm going to talk about risk and risk management. Can I ask how many of you are using risk management in your projects? Yeah, some of you probably already know everything I'm going to say. But I think it's much easier to estimate for existing clients. It's much harder for a potential or new client because you don't have any insights about their business needs and you don't have any experience from working with them. So you're kind of on unknown territory. Clicker. Don't hug the clicker, Jacob. Yeah. It's on the right, in the arrow. There you go. Yeah, there we go. I actually served in the Swedish Army and one thing I learned from there is that risk management can help you see when the map and the territory don't agree. Let's look at some of the burning questions here. Why should I care so much about risk early on? How do I manage and communicate risk? And are there common risks with Drupal? And what's the point of all these questions? Of course, we want to help you to avoid risks and surprises and the corrects like this fine transom all crashed. Don't hassle the half. He doesn't like the half. Yeah, and when you perform a risk analysis, what are the steps? There are three simple steps. The identification phase, the assessment phase and the mitigation phase. And let's start looking at the risk identification phase. Here you're going to try to identify all the factors that may jeopardize the success of the project. And of course, experience is a key component here. If you've been in the business for several years, some of the pitfalls are obvious. But if you... Also, it's really important with the knowledge and experience with the client. So it can be a good idea to survey users and stakeholders and try to ask and extract knowledge from the client. And what is risk? Two factors, impact and probability. Sometimes it's quite obvious if there is a critical risk, but if you don't, if it isn't obvious, you can use a tool called Composite Risk Index or Composite Risk Grading to evaluate the risk. And it's kind of easy. You value the impact and the probability on a scale 1 to 5. So let's look here on the index. Here is the index with identified risks. We got a video component rated as a high risk. Both impact and probability are rated high. So the risk grading is maximum. The integration is medium risk. It's quite high on both numbers. And migration is quite a low risk. And the asteroid is extreme. It can be called a black swan. And like asteroid, it's really extreme. But an example from the reality is the pizza incident I had with my team some year ago. Me and all the dev team went to a pizzeria, had pizza. Everybody had the same pizza. Everybody got stomach disease. We're knocked out for a couple of days. The project was a bit late and it wasn't so fun. And to manage the risk, I could have sent half the team to McDonald's. But still, that's maybe a bit too much. Big consultancies, they sort of split their team on two aircraft. So in case one crashes, it's not the death of the company. But a pizzeria is a perfect example. So don't forget about the black swans. Watch out for them. You got to dinner tonight. Yeah, so let's look at some monitoring. Okay, so this is an example of risk management and classification. So if you've got high level risks that are in that red zone between 17 to 25, this is your opportunity to say, okay, how do I want to manage this and how often really do I want to look at these risks? Because you're not going to look at your risks every single day. But if you have some that are in that high region, maybe it's a good opportunity to look at them on a daily basis if they're going to completely blow up your project. Or look at them for triggers. What's an example of this risk is really active and happening right now on a weekly basis for some of those high ones. And then all the way down to maybe you ignore them if they're really, really not important. They have not much impact and you can totally accept that risk. So, thank you. Here's an example of what you would do then. Once you've got your nice little risk register and you've listed out your risks and you're aware of them, you're doing your estimation process and you know how these things might affect you. One thing that you can do to communicate this risk clearly to your stakeholders is to do a risk chart. So this basically has your impact up here on the right side and your probability down below. And if you've got something that's got an impact of two and a probability of four like the migration here, you stick it in that slot up to over four. That's migration. That's where that lives. So this is a sort of low risk that you don't want to, you know, monitor a lot. And I've even gone ahead and scheduled some special classifications at the top. So that one falls under dude seriously. So we need to watch out for it. And this is one way that you can look at your risk chart and kind of see as your project evolves. What should I be looking at? How often should I be looking at it? And it's also really easy to communicate out to stakeholders. So this is a good tool. It's a really good visual tool. I call it the risk matrix and really, really good for showing for decision makers to take decisions. Yeah, so you identify the risk, you assess them and now it's time to manage the risk or to mitigate the risk. And first of all, you can accept risk and take no action. Sometimes risk can be good. And if the business impact is minimal, maybe this is a good idea. You can also try to eliminate the risk and, for example, remove it from the scope or lift it out of the scope. If you are having a fixed price on the larger part of the project but if you identified a critical risk, you lift it out and try to work with that on a running account basis, for example. And you can limit or share the risk, for example, just by working agile, I think. That's a really good thing in kind of sharing the risk between you and the clients. Yeah, so, yeah, let me tell you a true story from the reality about risk management. This was some years ago. We were approached by a television network and they wanted to build an online community where users could upload and share video. And we identified the encoding of videos as a critical risk. Different video cams have different formats and to encode these formats to recite was at that time quite tricky. We found a third-party component on the market but we had no experience from using it so to reduce the risk we suggested further research. But the client wasn't interested and they found someone else who could take care of the video component part. But what happened? Of course, it went very bad. It took a really long time and it didn't really work and the project was late. So, I was quite happy. I didn't hold the bomb there when it went off. But still, it's no fun to be in a project where things don't run smoothly and I think it can affect your business long-term because the project went bad and the client, you're part of the project. And a lesson learned from me with this story is that communicate the risk to all partners. Don't just use it for yourself, your estimate or for your internal planning. Take the risk matrix and send it out to all parties in the project. Finally, let's look at some of the usual suspects in Drupal's specific project here. The first one, the user interface is easy to change or I fix it later. It's always a huge risk postponing and it's a good thing if you work agile or with Scrum you always have a review and a demo in every sprint. So, watch out for that one. And the second, the API is stable and the integration should be easy. This is a mistake I made several times to underestimate integration. It's really hard and sometimes you have to reach out to the main expert and seek help before even starting. Isn't it out of the box? Clients, it happens to me a lot that the clients expect more functionality and I think a way of handling this risk is to educate the client and be really clear about what's not out of the box. And there's a module for that. This one is really interesting because there are so many modules and I think there is a module for almost anything but as you probably know just because there is a module it isn't quick-quicks, it can be a dev version and so on. Okay, so let's talk about techniques. So we've covered what is it, why do we need it, what are we going to use it for, how to properly plan and avoid pitfalls of the risks, what are the common ones. Now let's actually talk about some techniques that you can use to estimate your projects and who's still awake out there? Can I see, okay, good, everyone's still awake, good. Okay, so I'm going to stay on the clicker, thank you. So first of all, questions, what are they? How do they work? What do I use them for? When? Help. Okay, we're going to help you. So next, estimation is like a walking stick in my opinion. It's like a walking stick for a blind man. You can use it and it will help you to not walk into a wall, but it's not going to give you sight, said old Chinese proverb. It's Shannon, but... No, maybe not. We can't remember. She wasn't sure it was just her, so we wrote that. So it's an old Chinese proverb as far as we know. Okay, so we've got three example techniques for you today and at the BOF that we're holding tomorrow at 11.45 in Athens, we will do some examples. We'll do some run-throughs. So if you're really interested in trying out some of these techniques, we'll come tomorrow to the BOF. So the first one, ballpark, top-down estimation. Who has used this before or knows of it? Can I see some hands? Okay, so I'm surprised not to see more hands because I think this one is really, really common. People are like, oh, my client is an estimation. He's got a few things. He wants to know what the budget is going to be. I'm just going to give him a ballpark answer. Maybe I'm going to ask my developer if I'm a really good, cool boss or cool project manager. I'm going to go and ask them. But this does not always happen. Sometimes people just guess. In many cases, you maybe get a request by email someone and say, we want to build this site with this much money and you can easily in your head do the math and realize, no, that's never going to happen. Does anyone, okay, kind of talk them into sort of lowering the scope somehow? So I think this one is actually applied way more commonly than you would think. So the way that you would use this correctly would be to assemble some of the features and discuss it with your team. I would say more than just the developer because as Jacob rightly pointed out, there's a lot more that goes into an estimation than just the real work in quotes, all of the coding and the PHP and the testing. That's not everything. There's peripheral things that need to happen. So I would estimate it with your team and put together some guesses. And the reason why you would do this is to really just get a ballpark vision that you can use strategically. This is not something that's going to be super long-term useful. So it's a short-term solution. Next, you have the weighted estimate. So this one is a little bit more detailed. How many of you know of the estimate lecture I did in Copenhagen in Chicago? Lots of you have seen this. This is very similar. Yeah, you take the features, you figure out how to do them, what actions you need to undertake, and then you figure out how long it would take to complete each action. And the point here is just to try and break the project into all its parts and see what things can be potentially hard, difficult, so forth. You put them in a matrix like this, you estimate them, you type in the likelihood that it's going to take longer than that, what we call the confidence here. In my version, it's called degree of experience. It's the same thing more or less. In the end, you get a range of hours. This is really useful to level-set expectations with your client as well. This is going to take exactly 46 hours, and this is exactly when we're going to do it. This gives you a little leeway where you can actually be realistic when you're setting expectations and say, look, we think it's going to take from this to this. This is reality. We can't predict everything that's going to happen. This is our guess. The problem, though, is that the customer is going to think, oh, yes, they can do it in 300 hours. You know, like 450. Everything over 300 would be a failure. That doesn't really understand how to read this. True, true. So the final method is the Delphi method. How many have heard of this? Oh, more than I thought. How many have heard of planning poker? Okay, so a lot more than you. They're the same thing. This was actually invented by the Rand Corporation in the 70s, but the idea is the same. More heads, more brains, more brain power, better results. Right. Oops, I accidentally skipped this guy. So what you're going to do is put a team together of experts who have experience with a specific feature that you want to estimate. And in planning poker, they're all going to explain their estimates and you're going to try and come to a consensus. Generally, you have a stack of cards. You have a scale. And everyone gets a vote, but you don't see what the other person is voting until they put the cards in the table. If someone puts a 2 and someone puts a 20, then you know that there's either someone who doesn't know or the entire user story is not clear enough or the requirement is not clear enough. And then the point is for them to discuss until they can agree on the estimate. You don't move on until everyone is in agreement. This estimate is good enough and accurate. Exactly. So finally, when should you use what? And these are just some indications that you can take away with you. So ballpark top down. That's when you want strategic decisions. You're looking for decision-making, not accuracy. So the pros, it's easy and a fast solution, but it's a short-term solution. So do not do a ballpark estimation and use that throughout your entire project. Please, you'll make your developers very sad. Can I build an e-commerce site in 50 hours? Sort of that. Then you have the Delphi method for specific features and you need several experts to do it. It's going to be smarter and more accurate, but it's going to be feature-specific. You can't use this and develop your entire project estimate. You need to reuse this several times in order to get a full project solution. We're going to discuss the problem with estimates, too. Get to that in more detail. And finally, you have weighted estimations, which is what Jacob explained pretty well. So it's used for getting a variance, like a range of things that you can work with, but the only drawback is that it takes some time, but you can use it to help level-set expectations. Okay. So, can you overestimate? Yes, it's obvious here, because I basically told you that you can. So what happens when you estimate the issues that you start off, you really have no idea what you're working with, you learn more, you try to think about potential solutions, you try to make educated guesses about the customer's expectations. The problem with this is that the more you work, the more confident you get. Your confidence goes up as the purple line shows. But the problem is that the more confident you get, the more likely is the fact that random things will affect your judgments. Your estimate will actually get worse, you more work with it. It's been actually observed in many, many cases. So it's actually better to just put a reasonable amount of effort into your estimates, and not too much, because that can actually give you worse results. Don't get too comfortable, I think, is the takeaway. Yeah. All right, so I mentioned that there are a lot of don'ts of estimates, and I know that many people think that estimates are a really bad idea, and often they are. The idea that we could early on look at a project and based on a lot of variables, we have no idea what they are, come up with a number for a customer. Yeah, that's often a pretty ridiculous idea. Estimates, though, they are useful. They can be used as a means of communicating something, setting expectations. If someone comes to you and wants an e-commerce site, a simple estimate will tell you that it's probably going to cost them at least a thousand hours, because you know that e-commerce sites are business critical. You need a high degree of reliability, quality, testing, and so forth. So estimates serve the role, but what's often the case with estimates that are used for people higher up in management to get people that do build exercise to commit to estimates, and then use that to push the blame when things aren't delivered in time and so forth. Well, you said you were going to do it in 15 hours and now you're not done and it's 15 hours. Oh my God, what's happening? This is not to be done, people, don't do that. It's not going to help your team and all you're going to do is demoralize everyone in that project. Secondly, it's important to realize that a lot of features are like icebergs. You'll only see the top of them. Imagine you're on a ship at the North Atlantic and there are tons of icebergs and a lot of fog. First of all, the icebergs you see right ahead, you know that they're bigger than you look, but then once the way ahead, you can't even see them. In many Azure projects, you start with a backlog. And one of the ideas of Agile is that you learn more and more about the project as you go on. You make decisions as reality becomes apparent to you. What I've done at one point was that I sat down with a customer and they said, yeah, we have 1600 hours. Can we feed our backlog? And we estimated the entire backlog in detail. But I realized what I was doing. I was doing waterfall by Agile. I was doing waterfall project management in an Agile project. Water Gile. So what's important is to actually accepting it to a customer to understand that you can probably plan the next sprint to the sprint after that. It's like flying a tracer boat and see what's ahead of you. And as far as you can get, everything ahead of that is just epics. And trying, like we said here, work with risk management to sort of deal with this. But you can't make anything nearly accurate. And there are nearly accurate statements about these things. You don't know what problems you're going to hit until you're actually in the water zooming towards that giant iceberg. And then you can figure out how to mitigate the risk. You can peer down into depths of the Atlantic Ocean and say, we're going to die. I hope you're not Titanic. Titanic. So those are some estimate issues. And the last one, don't do it in a vacuum. Don't do it by yourself. Neither should a project manager be all by themselves. And a team of developers should not be by themselves. If you're a project manager, don't make estimates. Yeah, just don't. Ask your team. It's not your job. You're not the one doing the work. You're not the one who can say how long this is going to take. Get your team together more than just development and discuss it as a group. This is the only way that you're really going to cover all of your bases in estimation. So if you're just going to your development team, you're making a mistake. If you're just a project manager, you're making a mistake. It should be a group effort. It's a lot of us who are specialists. We're going to burn for what we do. But that also makes us vulnerable because we also look at, if all you have is a hammer, everything looks like a nail. It's especially true in this case. Developers think they can develop themselves out of problems. Designers think they can decide themselves out of problems. Product managers think they can plan themselves out of problems. So just relying on one of your one of the people in the team with expertise will make you blind to all the alternative solutions there are. So finally, our takeaways. We've got some things that we want you to remember. Okay. Do you want a discabit or something? Yes, I wanted to do a hypnosis thing. Everyone watch my hand. Now remember this. Okay. So first of all, management by estimate. It's a very bad idea. This is what we were talking about. We're using management of your estimates as a blame factor. You said 15 hours. You didn't do 15 hours. It's your fault this project failed. That's not the right way to motivate your team. It's not the right way to get better accuracy on your projects. All you're going to do is create a culture of fear and estimate they're afraid that you're going to point the finger at them and say you're wrong. You're putting responsibility on them in a way that makes them at fault where the better solution is to focus on lessons learned and to focus on trying to find techniques and identify the reasons why the estimation was off. So that's number one. Yeah. An estimate should be more than just development, like I said. Like a big arrow. That stuff. And also make sure you involve people that do all of those part. And they're just more than numbers. Like in the case before. Estimates communicate to the customer how you perceive their problem and it tells them how you would go about solving it. So remember that even estimates have a subtext. They actually carry a message more than the number and more than the sum itself. And risk management. You can avoid the problems you can identify. It's sometimes I ask the developer how long is it? How long? How long is it going to take? And I get like, oh it's 20 hours but I forget about the risk it may include. So it's really good to discuss it with the team. Let's add one quick note to that. When you're doing this risk analysis and risk assessment about your pretty little chart you don't just want to identify them and try and say, oh well I think it's going to be huge so I'm going to put the impact as five. Really try and understand what is the impact? If this thing breaks does the project lose all of its value? What really is the monetary impact to something you can measure as well if you know the value that the client has provided? So there's lots of ways that you can use this really to increase accuracy and to prepare for the inevitable problems that might arise. Yeah and sometimes it's worth just avoiding something altogether it's just going to lead to trouble ahead and I'm honest. Just say no. Just say no. All right. Roll the film. You're in charge. Sorry? There's two more. No, we've done all the work. All right. So we've been kind of asked to ask you to review this. Tell us what you think about the session. So if you have a camera you can take a picture of the QR code and just type in that address in the browser. But you need to log into the Drupalcom website in order to be able to provide session reviews. And when you go to that link there should be a button or a tab somewhere to click and sort of review this session or something. I'll give you a second to take a photo. Please do review. We're really interested to get feedback. Good, bad. Only good. Next one. All right. Thank you for questions. It's like the Twitter feed. Yeah. So while Shannon is checking the Twitter feed for all your questions that's time for the panel part. Can we move to the next slide? All right. So I'm going to do a shameless plug-out for our buff tomorrow. If you want to discuss this more, if you want to discuss the complications of doing estimation and agile projects and the parts that we'd like to talk a lot more about the problems with estimates, the benefits of estimates, what to use estimates, what estimates to use come to our buff. We're going to do some estimation. We're going to show some estimation techniques so we can practice that and we can have a good discussion and it's a lunch. It's a lunch break so feel free to bring food. And we'll have an estimation lunch. All right. Time for questions. This is still loading. So we got to put the mic back there for questions. Yeah. Do that. Did you guys tweet some questions? Hand if you tweeted a question? I see about two questions. Good audience. Really everything we said was right. Okay. Awesome. So we'll start with this one from this lovely young man who just popped up here. So what ways can you suggest getting quote buy-in from new clients when we're explaining the agile process? It's a great question. So basically what can we do to get our clients to accept agile? There's lots of things. How many minutes? How much time do we have? I think we've got about 15 minutes for questions. My general answer to this question is that waterfall is usually about an incorrect assumption or perception about control. Like you as a customer you as a customer are in a better position if you can control more factors. That control however is a complete illusion. A lot of products are driven about requirements. And I did a session in Vienna about this called Upgrade Your Offer and I believe we as a group of companies need to focus less on requirements and more on the kind of business value that our solutions bring, our customers. And it's when you should focus on business value, you're going to realize that the requirements are just secondary. They're just one idea how to actually bring that business value about in the website. My suggestion is that you throw out the requirements and you sit down with the customer and try to figure out what is the business value. Eventually you'll see it's a pretty concrete thing. It's a number. You can see the write-off of that investment. You can actually translate it to actual money. And when you do that then you can see what's a reasonable budget. A lot of customers, they haven't even done this work. They haven't even thought this far. Because there's a few reasons for building a website and building internet are often taken at a management level. And the IT department are tasked with drafting the requirements because they've seen so many requirements documents. And they are the ones tasked with turning to you for you to produce a number on a document which may not be relevant at all. So the big case is saying that projects which are focusing on business value and the agile in their nature will deliver better results. They will have to deliver what you... I mean, essentially, if you can't deliver everything what should you focus on? And there's really no way of knowing unless you know the actual business case for a project. Yeah, so try and understand what does your client want? Why are they there? Where's the value? And I would say, if you can, try and start small. Start with something that's going to provide value for a small amount of effort so that you can start off with a win and you can gain that trust because I think one of the reasons people shy away from agile is because they're very afraid of the risk. And as Jacob was saying, they don't know what to expect. So they want to plan everything out. But if you can get a win, then you're going to gain some client trust and you'll have some capital to work with. Especially in experienced buyers. There's a book called The Lean Startup by Eric Rees and he talks about something called you can build Drupal site almost the same way. You can take a feature, a blog for example, and you can make a simple blog using fields and views and you can just have headlines and pictures and now we see we get it and you show the customer, this is a blog, it took us two hours to build. Is this what you need or do you need anything else? And it turns out this fancy blog that maybe they're advertising agency come up with isn't really necessary at all. This simple blog with a bit of theming will deliver the action that you need to do. It will cover the needs of that certain target group and it will fulfill the needs in terms of user experience. It's very hard to have an opinion on something you cannot try and if you can build a prototype which we can, because Drupal is an amazing prototype new tool, the customer can make these decisions early on and they can decide right away, okay we're going to scrap these requirements. That is something they could never do with a waterfall. I think that azure methodologies provide uniquely. Okay we got another question here. So how do you deal with what seems to be a habitual tendency for engineers to underestimate the amount of time needed to deliver? So basically why are you engineers not giving us numbers that are high enough? Blame game again. But you have some pretty good techniques for that. Well I think first of all having a risk management discussion with your engineering team is one way to make them understand that everything is not peaches and roses. Things are going to happen. Things are going to go wrong. And you need to discuss that with them so they think about the impact. And you're not just saying, okay well I'm not very comfortable with this estimate so I'm going to add like 45 hours on to it just in case I really screw up bad and need to cover my butt. What you want to do instead is say what is the actual impact? What do I think this might set me off? Well then maybe I should look into it a little bit. Instead of going into it blind and just saying I think I'm just going to slap on all these hours. Do a little bit of testing. Do some prototyping. Get some experience and find out even if it's just for your own knowledge what is the potential impact of this? Try and learn something before you take that leap. I think it's the safest thing to do. So definitely discuss it with your team and do some critical thinking instead of just oh I don't know, I'm scared. That's not going to help you in the end. I think the best advice I can give is sort of bring the team and customers close together as possible. Don't be some proxy between the customer and the team. Estimation by proxy it's not a good idea. Also planning poker is a really good thing because when we started to work with planning poker the estimates were much more accurate than before when we didn't work with planning poker. We're going to do planning poker tomorrow in the buff if you want to see how it works. If you haven't done it, this is the opportunity to try it with a bunch of people who also are excited. Yay! Are you saying the teams aren't excited? Well, now maybe they are. Raise your hand if you're excited to do estimation. Okay, well half of you are. You need to come to the buff and then you'll all be raising your hands emphatically, both of them. That would be chocolate. We'll go buy some chocolate now apparently. There will be. Okay, so how about another question? Pre-project workshops are very helpful with more accurate estimates, but how can we help the client understand that? So basically doing discovery before the project is helpful. How do we convince the client that this needs to happen? Shannon doesn't want to talk so much. I can, but I'm hogging the mic. She's hogging the mic. So can you repeat the question? How can convince clients to do discovery? Well, what we've been doing for a long time at Node 1 was that we've been doing pilot studies. We basically what we call we did a we suggested a solution, almost like a recipe. And the reason that this, the work we do here is going to be valuable to you, regardless of turn to us or another agency. Essentially you're going to have a roadmap for building your site in Drupal. And we're going to use our expertise to figure out all the traps and pitfalls and everything. And that worked pretty well. In most cases, almost every case actually turned to us about the site, but they got a document from us which wasn't just, which was actually genuinely actually had actual value for them. So if you can sell your discovery as a sort of a pilot study which will help them get better prices and save money in long run, then they would rather pay for it. But it takes some sales work. I think it's a good idea to show some of the work you've been doing for other clients if it's possible, because then it's much easier to understand for new clients or potential clients that, oh, this this pre-project workshop can really be you can get out so much information from it and you can use it either you choose to work with someone else later or if you continue the cooperation. How many of you use workshops as a way to get to requirements in place? Okay, cool. I think workshops are super, because the customer gets involved too and if you can involve the team even better. That's a piece of advice I can give you. I think it's a good idea to use more workshops. They can also use it for marketing sometimes. If they're doing that workshop and talking to users, that's also a great way for them to gather information directly from their base. So there's lots of avenues for marketing that solution to your client. Okay, so another question. Any preferred tools for risk management during estimation? I'm going to put together different resources and give that information at the bof. So if you want them, you have to come tomorrow. Basically. You get super cool spreadsheets. Yes, we've got some spreadsheets that we'll share. So let's see here. What's another one that we can ask? So how do I manage the sales team when they want me to reduce my estimate? Sales is like, no, that's too high. That's out of their budget. Well, it's like, what do I say, there's to me, it's like a bus. Like you can't argue with the weight of a bus. A bus weighs a certain number of times. An estimate is a certain size. You can't negotiate with it. It's like a negotiation. If they want the lower price, they're going to give something else. That would be reduced to scope or something. Like you told me a great example. Actually, you had it. Do you remember what you said before? Oh, I remember. I'm going to quote Matias. So he said basically that you should only negotiate how much you're going to do. Negotiate in the effort. Not on the actual value of that effort. So if your team is coming to you and saying 25 hours is too much for X feature, then say to your team, okay, how can we reduce the effort necessary to produce X feature instead of, okay, well I guess 18% because we need to come in under budget. That's the surest way to be wrong. Absolutely wrong in your estimation. Either that's a risk that you're willing to take as a shop for your client. We're going to accept the risk that we're going to be probably wrong on this estimate and we're going to eat that. But I don't suggest that solution. I think it's better to go back negotiate with your client and say this is a problem. We need to reduce scope because we think for X, Y reason this is going to be too much effort for what the value that you're talking about. So let's find a way to either share the risk. We'll do 10 hours free. You do 10 hours paid. Maybe you change the scope, change the architecture of the solution to make it simpler so that it does fit into your scope. Or maybe you just go straight out to your client and say look, this isn't going to work. We do like to handle it because we don't want to take on the risk of this thing becoming giant and having to fix it ourselves because that's not fair to us in our business relationship. An estimation is also not just something you do. If the customer can actually estimate the business value, you can wonder if a project will bring them maybe 2,000 money something in terms of value. Maybe that's not so smart to have a budget of 1,000 to build that project. That's more like 1 to 10. The project will build 10,000 money something, 10,000 euros of value and cost you 1,000 to build. Then it's a much easier equation. So do that math. And I wonder how close do you want to cut it? Okay, so let's do one last question. So this one is, how do you combine a requirement for a fixed price with an agile development planning approach? So basically how do you agile with a fixed budget? I think this is a really, really common situation in the bof that I had yesterday. We had this question as well. So thank you for treating that. We tried this. We've been playing with different versions here to try and get customers to sign up to this and I would say don't. I mean, something's got to... Okay, if you have a fixed budget then you've got to have a fluid scope. You can't have both fixed... Or quality. You have to realize that the importance is to deliver the things like I showed in the quadrants before, the diagram. The ones at the top left. Focus on the ones that at least effort the most value at first. And then cut the not so important stuff. And that means you can't commit to the scope. Yeah, your client needs to understand that it's give and take. They cannot have all the points in the triangle all the time. If they want a fixed budget then they can't be flexible on other things. And they need to agree to this from day one. If they don't then I would say don't do the project because you are going to really be hurting at the end of that because you're going to have to take on all of the risk of, you know, building this thing that you have no control over because your client is going to dictate your scope to you. So I've seen this happen and it's a really tough time for the team. I don't think you should do it. If you can change the fact that the scope is, you know, unfixed or say, okay, but guess what? For this fixed budget you're only getting our intern. You may take a hit on quality but they'll work as much as you want. I don't want to be an intern at your company. No, it's an example of another way. If you have, you know, people that you're bringing up and teaching maybe if you've got this high risk fixed budget project you can bring in someone with less experience who, you know, presents more risks than the clients say, okay, if you are not going to pony up then we are not going to, you know, take the time from our best developers and put all of our other projects at risk because when one project goes in the red and you have to, you know, try and save it, sometimes other projects can be affected by that like we saw in the ABC, you know, diagram. So that's probably an answer. No, I agree. It is a problem because a lot of the tenders you get you have to come with a fixed price and at the same time a lot of companies and organizations want to work agile and it's kind of it's kind of in a twilight zone now and and you have to be clear about some of the lowest prioritized stories are not going to happen in the project. So I don't think we have time for any more questions. There are more questions to answer. So come to Buff tomorrow and ask all you want. Yeah, we're going to take some of these other questions offline. So I think next slide we'll show what we need to show. Who's got the clicker? We lost the clicker. The clicker's here, Shannon, but what information do you want? We have the Buff information here. So tomorrow 11.45 to 1 in Athens is our time. So come to Buff, come to it and talk to us, tweet to us if you've got more questions or things that you want to discuss or come find us. We're going to be hanging out for a while. And I think that that's it. So thank you so much for your time.