 everybody. And thank you very much for coming to our presentation called Drupal Project Estimation for Fun and Profit here at Drupal North. We've got myself, Alex Berkachev, my colleagues, friends, and Ken. Friends is a Drupal architect, Solutions Architect, and Ken is our very senior project manager, who's, believe it or not, has done bigger projects than Drupal. And so I already mentioned a little bit that we're evolving what is based in Montreal. We love events and we love hosting cool people. And we work with fancy looking logos. So here's the three of us. Yeah, and I've been doing Drupal since pretty much the start of my career after after McGill for over over 10 years. France, I think, almost as long. Yeah, and he comes from FFW, a small Drupal shop all over the world. And Ken has been doing agency agency work for a better part of the decade as well, maybe more and most probably a twist image, which is more legion and legion and legion market. So a little bit about software estimation. We cribbed about half this talk from from this book that I picked up. It's kind of a classic in software engineering circles. If you have a like software engineering degree, they'll probably talk about either this book or Steve Mcconnell's book code complete. And it's got a picture of Microsoft keyboard on it. And so these are really like the Bible. I didn't read all of it, but at least two thirds and it has a lot of great examples. And the first the first concept that you have to take away from from reading this book is that when people talk about software estimation, there's actually several distinct senses when somebody says, Hey, I need an estimate for how long this project is going to take. So an estimate could be a business target. You know, the sales guy says, Well, the client is going to go for it. If you can do it in, you know, 50,000 bucks or 500,000 bucks, because that's what the client wants to hear. Another thing, an estimate is an unbiased definition of, you know, sorry, it's a biased idea. Well, how long do we think it's going to take? Actually, if you ask a dev, who's going to do it? And then a third one, and this is more than project manager is it's a commitment. It's a commitment to do this thing. It's a promise to the team and everyone, the client, that it's going to take that long. Now, it's very clear that the same thing, an estimate, often a single number, when it has these three conflicting goals and stakeholders, it's quite tricky to figure out what are we actually talking about and what are we producing. And this duality or whatever multi-faceted nature of what an estimate is, is often responsible for why projects go over budget. You know, we said it would take a thousand hours, but is that what we wanted it to take, rather than what we thought it was actually going to take? So this is an idea that you need to be mindful. So to make things a little bit smoother, it's important to realize, are we talking about a preliminary estimate or are you talking about a commitment, saying, I'm going to do this. And it also really helps if both the client and the person who's giving the estimate are mature about the process and about what they're talking about and what they expect from each other. It also is very important to have good trust each other. Like, you know, I presume that I want to make as much money as possible, right? But I don't want to do it on this project. I want to do it over the course of my career as a vendor. So I want to build a successful project. I want a happy client. I want the client to come away feeling that they got what they came in for. So it shouldn't be seen as a negotiation that's winner and loser. It should be seen as both parties are trying to get a sophisticated arrangement that respects the goals and constraints of both parties. So I think that's a really big part of it. So another idea that we try to do here is come up with a few ways of defining an estimate. This book talks about top-down versus bottom-up. And this is something that we practice at the Bobby Wombo all the time. So for the top-down, it's kind of saying, well, what's the project budget? What do we think it is? What kind of projects do we do? What does it look like? Does it look similar to other projects that we've done? How many people are going to work at it? It seems like a three-dev project and it seems like it's going to be at least a year long. So this is the top-down approach. And it's very useful because when you're doing projects that are similar to what you've done before, you can just kind of compare. And then there's the bottom-up approach, which is that you start listing all of the deliverables, all of the phases of the project, which includes discovery, content strategy, UI design, UX design, however many meetings you're going to have for that, which reports you're going to produce or slideshows you're going to do. You're going to include a kickoff meeting and how many people will be present. And then, of course, you're going to start breaking things down the development, which is, in my intuition for Drupal projects, about 60 to 70% of the budget goes to that. But in some projects, especially a design oriented ones, it may be lower. But you're going to break that up into sprints. You're going to break it up into phases. And you're going to say, well, in this phase, we're going to tackle these things. And we think this phase should have three sprints of two weeks each. And the deliverables in the first sprint will be this, the second sprint will be this, the third sprint will be this. And then you account for the QA documentation, demos, project management that go into every sprint. So that's more the bottom-up approach. Do you guys know which is better? What's the better approach for estimation? It depends. That's one answer. A better one is both are necessary. So it's kind of like in math, when I say math, when you're doing logical and mathematical proofs, they talk about forward backwards technique. You start with the assumptions, and then you try to derive some knowledge from it. But very often, you already have an intuition that you want to prove, like the thing you try to prove. So you actually start with the end. And then you work backwards and say, well, if it was not the case, for example, is it a logical tautology? So that's proof, reduction out of certain. And that's a very important class of proofs that for things that cannot be proved otherwise. So in mathematical proofs, you have to work forwards and backwards and see which one leads you to the proof. That's how mathematicians do it. And it's the same thing with estimation. You have to do both a list of everything that you know about. But then you also have to say, well, based on what this project feels like and what I've done before and what this client is used to doing before, historically, what does it feel like? And then you do both. And then you reconcile them. And then you reconcile it to the to the sales person's goals. And then you can reconcile it to the client's budget. And all of those things will end up being a single magical number. And that's it. It looks very easy. Yeah. Another another important technique that goes hand to hand with top down or bottom up is is is these three estimation approaches that Steve McConnell talks about count, compute and judge. So this is this is judge obviously is the gut. It's the intuition. It's like, ah, I often do this, like people, my co workers hated it when I say it, but I've been doing this for 11 years. I've done like 100 Drupal projects now, some bigger, some smaller, and this feels like this size. And and people like, why are you doing that? But actually, I like to think I'm pretty good. But you know, so so Steve McConnell says that you should be always used judging all the time. But you shouldn't. In fact, you should use it as a last resort or as a way to check things. And if possible, you should resort to more scientific and systematic approaches. So one is count. One count that I do for every new project that comes in is I go on Google and I go to site for if they have an existing website, that is, I go to site my client domain.com. And I just see how many results Google returns. And that's not very good recently. They've changed it so that now it's more of an estimate instead of account. But but nonetheless, it's still a very useful index. It'll tell you if it's a 50 page site, a 500 page site, a 50,000 page site, although beyond a couple of thousand, it stops being accurate so that you could be factor more. We also have a crawler that we sometimes use for these things. And there's other tools, but but this Google thing, it takes one second and there's no excuse not to do it. Another another thing that you can do often, even without access to the back end of a site is you can you can say how many content types do you guys have? How many languages? How many modules contributed and custom? We asked this to our clients when we do estimation, even if they won't give us the code. We'll say, well, OK, how many lines of custom code do you have? Or I can I can ask them, OK, you want us to rebuild a Drupal 7 to site onto Drupal 8? This is very common these days. How much of a budget and time did it take you guys to do it? Often they don't trust you, especially if they don't know you, so they won't tell you their budget or time. But you can ask them proxies for these things. You can say which agency worked on it. How many months did it take? How many members of your team worked on this thing? How many admins do you have who are going to enter the content? How many new pieces of content that you that you're going to get? Did you notice that drank a cup of coffee? So this is the kind of counts that we're talking about, that when you do all of these things for a given project and you do it for the previous project and you know by this metric or by that metric is it bigger? Is it smaller? Is it roughly the same? And this allows you to index and have a common language with your co-workers as well, with your client and explain why it's like this. Computing refers to using these counts and with historical data. So an evolving web, as of about a year and a half ago, we've gotten fairly religious for logging time in our time tracker, which is toggle and then redline for our project management. It's automatically synced thanks to the work of Jigara co-worker. And so we get to look at the previous projects that seem the same according to those indexes and then say well that one took fifteen hundred hours even though we only estimated 1200 or that one took 400, it was fine. So this this is what it means to check your judgment using metrics. And another thing that I want to tell you we do, I do all the time, is value-based processing. I didn't do it explicitly before because when we're in the smaller budget range and the clients would come to us asking for many, many things. Basically I was just saying well if I have to do all those things it will cost way more than I know what their budget is. So I was really sticking with the bottom-up approach and then working backwards to say this is what's included, this is what's included because I know that's the thing. Now that our brand is a little bit better and we kind of have a target market and target price range that we already know that we make exceptions for but you know we know what kind of projects we're good at and are successful financially for us. We sort of stick to that when it's a good fit for the client. We look for clients that are good fit for our deliverables and then we talk to our potential clients and say here's what we did for this other client that that was great and here's roughly what that looks like in terms of price. And actually a lot of my competitors are doing that. I'm a developer first and have become like a estimator and sales person but many organizations have just sales people who know nothing about Drupal development, who don't know what a content type is. So how do they price it? Well, they pick a number that's the most that they think the client will pay and what they know is the client's budget, a lot of nonprofits actually published their last five years of annual reports which if you look carefully you can glean between the lines what their budget was on IT spending, marketing spending and also they'll know who the competitors are and how much they charge for something like this. So value-based pricing is a really big part of that and when you come back to estimation as those three different things this makes a lot of sense. I have a slide here. Yeah, so we want to point out that in addition to top-down and bottom-up there's different ways of doing bottom-up right? There could be a list of deliverables but equally valuable to me is my staffing profile for the schedule and how many people will be working per week per task, per category of people. So you know include your major like deliverables and milestones but then for each one category of resource I'm sort of using the word and then don't forget to include project management of course and just say for all of those things here is what this given person will be doing that week and we typically throw in 20% for project management so sometimes it's less but usually it's more. It's something we usually run over on but that's a good compromise and I also want to point out something that you probably know. So has anyone here worked on a $500 website? Talk to the bottom. Come on, don't be shy. Has anyone here worked on a $50,000 project? Okay, getting warmer, look at this. Anyone worked on a $500,000 project? Okay, about a quarter of a room. Anyone worked here for a $5 million project? Okay, just one guy. So I mean we've done all of these for the $5 million it wasn't evolving up who's the prime but we're definitely an important part of that project and I can tell you that sometimes if you look at the end deliverables it's hard to tell if it costs $5 million or $5,000. In fact the $5 million projects look like they you wouldn't want to pay $500 for it because it looks like garbage and because it's political right? So the same nominal deliverable if you budgeted differently for different organizations and their goals and expectations it'll be structured differently. It's not the same deliverable because people might spend like a thousand hours per month in meetings. I mean this happens like a room project all the time and it's normal because that's how government needs to do business. Or I have a friend who just took a job at the Veterans Affairs and I think he spent the last four weeks writing a utility to run unit tests on every commit you know something that we get for free with CIRCLCI and they have an open-source solution they didn't have that so they were super happy with him billing I don't know like a couple hundred hours just on this little utility that's going to make their whole multi-year project more efficient. That will never fly for most of our clients. So this is important to be on the same page. And another concept from estimation I'd like to touch on is this idea of a corner of uncertainty. So when you when you just get an RFP or you get a lead of a voicemail that says hey trans call me back I might need a new website you know you might google the name of the guy or the phone or the woman and the phone number to see what organization he's from you look at their existing site and and you'll make assumptions as to what they want and you'll talk to them you know a little bit more then RFP sometimes yeah or the RFP it's the same thing and you have a list of questions in the RFP then you're going to go sit down and you're going to make assumptions around those list of questions and you can put that in your in your proposal and those assumptions may half line up with what the client had in mind and half not then the project starts and you all those assumptions after a price has often been settled and the schedule is often been settled then if you start a discovery phase where you start questioning those assumptions and it turns out at least one third will be discarded and replaced by something else but hopefully it was an unbiased estimate so you're not too far off and then so you're getting warmer and then once you start actually writing the specs and you wrote the specs and you did the design then you kind of maybe know you think you know 75 percent of what this project is going to consist on but then your developer starts working and starts billing out your content types and reusing all the existing code that already exists and it turns out that no although that code that you're going to assume you're going to reuse is not reusable at all because that was Drupal 7 and this is Drupal 8 and this is multilingual or so so then when you start development you kind of get visibility in that process and then obviously as development goes on and you're into QA then it starts tightening down but at the same time when you hit QA you the client will come back and say oh great my boss now that the site looks like it's done my boss finally took a look at it which he never really saw the design despite the fact that he signed off on them and now we want design changes and we want all these extra features that we don't have on our existing site I mean we're putting the RP but we cannot launch without them so so this is this is the idea of this column uncertainty and so and so it's okay it's actually okay to do estimation when you don't know a lot of developers who are like logically minded robotic creatures you know feel very uncomfortable with this uncertain world but this is the reality that we're in we all don't know at any given time we find out more and more and we just do our best to estimate based on any information we have and how do we in real world deal with this column uncertainty we actually put the most defined easy reusable non-negotiable tasks in in that phase one phase two is the stuff that we the client would like to have but they haven't quite given us enough information and phase three or whatever it is you slice it up like how you want phase three of the things that speculated next next project maintenance we'll do as part of ongoing maintenance after launch and and so then we jiggle things so so we actually the way we hit our estimates it's it's not because we're so amazing at estimating but because the estimation process doesn't stop when the estimate is done we redefine the project scope at every single step to make sure that it still fits all great project managers do it I mean then clients are happy when you do it and they're very unhappy when you don't know so that's that's a life lesson and then there's a very important rule in software estimation which is the first 90% of the code accounts for the first 90% of development time and the last 90% sorry 10 10% of the code accounts for the last 90% of development time so it adds up to 180 that's that's the rule so i have uh i have some some software estimation checklist but i think i'm running a long time we'll come back to it okay and there's another book i don't have it here because i lent it to a friend like years ago and i guess it's not a friend because they didn't give it back uh it's called the mythic of man month anyone anyone heard of it no it's a it's a very classic book in software engineering it's from the 70s or something by there by uh Fred Brooks who was leading IBM system to developments their operating system and uh and so he was he had like five hundred a thousand developers under under his team and there was a huge model monstrous project for for one of the major enterprise operating systems and he he formulated what he calls the Brooks law which is uh adding manpower to a late software project will only make it later and he has charts about people communicating and meetings a number of like email ccs and bcc's uh he also talks about the fact that when you have a small team that's a clear responsibility there's going to be a visionary who is like the quality control the architect like the designer like who owns the the functionality whereas when a project grows that gets diluted there's conflicting visions and so he saw it in his real life work now he defines the term of man month which was very popular back in the 70s planning days and he says it's not very useful but hence the mythical man month and uh i also remember him talking about a set second system effect which i already mentioned you know how he said post launch all those lovely things you want we're going to do the post launch so it turns out that in his experience the first phase that was a beta prototype always every browser and that went fine went over budget but it went fine we launched it but actually the big failures of his career were the second system it's the time when you get to that okay and now we're going to do everything we wanted and so that's the one that like you kind of use this trick of let's stay focused and just do the bare minimum and see what we get so that's uh that really killed so with that i will uh hand it over to you okay thank you very much i haven't had coffee so i'm going to slow down a bit uh so we talked about estimation and and uh alex has given us a bunch of different techniques and i think what friends and i want to focus on is this idea of the range and so the range is where you're going to provide a high and a low estimate for a particular project or a particular task and hopefully you feel 90 confident that it's going to fall within that within that range so we're going to we're not going to spend too much time but just basically uh just to just to throw this out there um these are like just an idea of 10 different things where you have absolutely no idea how to estimate them and so usually if we had a little more time we'd say as an example anika you could try what do you think is a surface temperature of the sun you're going to try to come up with some different uh maybe some logic or something some come up with a range how about we do something like if everyone pick one then take a note of what would be the range that you think this is in great take one or two if you're feeling like it okay wait guys table one remember table two table three that is four it's table five table six, table seven, table eight, table nine, table ten you guys in the back you get a pass uh so it's table one table one uh can you all take a second you don't have to talk to anyone can you promote yourselves and answer your question no google please remember to use the range right so so now with the information that we have the head person of the table which is defined to be the person sitting closest to the aisle two numbers guys i need two numbers that doesn't matter table one surface area temperature of the sun in celsius ok a hundred thousand degrees celsius to what to how much it's the same five hundred thousand okay five hundred okay it's a great second question a lot of people shine high zero two a thousand what zero two a thousand oh good table three uh you didn't actually get it right but i think so three i think we would say one tenth of the uh it's bearing it's bearing i guess one tenth i look at two to look at the ninth area but yeah so i don't know it's i think it's five hundred million celsius so i don't know a hundred million fifty million fifty million okay two fifty million two fifty one uh maybe uh between here forty good okay good uh next one table four alexander the greats to zero to zero bc bc five hundred bc to zero okay okay next between one hundred trillion and five hundred trillion okay volume of the greats one no no that's currency us currency okay that's clear okay okay that's a trick you can say if you want to add that okay table oh table six is only kb or sorry one hundred three i do i'm talking to leaders here but doesn't matter we have answers for both yeah okay table six table six is the total volume of the great lakes uh i don't know so like fifty thousand leaders like five hundred thousand leaders okay okay okay fifteen thousand leaders okay this is what happens when you have one person doing that yeah that's a good lesson guys that's a good lesson table seven is the titanic box office receipts no go king is that what you're looking at no go go no go go i don't really reach consensus internally but i'll say what i think i i thought it was like forty to fifty thousand commerce what no titanic titanic what's the titanic we did it wrong that's another valuable lesson guys okay read the rfb titanic please how much how much money did titanic make lost off of receipts for receipts for the money how much how much ticket sales one hundred fifty million two okay two that's the low right yeah where's the tickets or dollars okay next one table was at table seven table seven length of coastline is a table eight okay table eight table eight pacific ocean pacific ocean okay okay table nine is the number of book titles published in the u.s seventeen seventy six say a hundred million hundred million okay minimum up to okay okay and then the heaviest blue whale ever recorded uh table ten so uh we said five to ten times okay how much five to ten times what's it time i'd say one or one or two kilo whales and it's fine okay gonna show that interest now okay here we go so alex let's see the first one we said a hundred a hundred two or five hundred wrong that's his they were really wrong we said zero to a hundred very savvy that you got 31 so that was good that they had a white paint you got a number two it's uh it's north it's out there so you mess you mess up also there's not two hundred degrees next time guys guys please no no talking is going to be very difficult okay so area of the asian continent uh the answer is 17 million square miles or 44 million square kilometers and we said 40 to 50 million square kilometers wow that next one great no 500 bc to zero ad you got it in there great job it's a picture uh 100 uh million to 500 million was uh for the current scene sorry billion no no that was it was really not way off okay there's triple projects that are bigger than that don't judge it okay the next is uh 50k to uh no we saw the order 150 million to 250 million we're a little bit off a little bit off but i think we're off yeah and for the great lakes we're off on that too we're i said in leaders i don't know cute but you're away we don't even have to bother converting no judging next 30k to 50k kilometers uh 30 to 50k and it was quite off uh and then uh we said 100 million to a billion quite off and then heavy as blue whale we said five to 10 tons it's quite awesome okay so you got two rights got two right two rights out of 10 okay right so i hope that teaches you all a lesson about how we're all expert estimators exactly so so can't please continue actually just just to say that because this comes from this this book and so they had given this survey to 600 people and only two percent were able to score eight uh eight or more correct answers and most of you are right here we're right here you're average so we're just average guys and so most of the people were between one and three correct answers and the so the sort of the takeaway from this is that even though you feel 90 percent confident you're actually 30 confident right exactly but the point is we're getting your range so you could have made your range bigger to encompass that that's the idea of course if you can have more research you can decrease your range but it's the same thing but that's the same thing if you have the research and information you can try to make a smaller range and still be off i think the point of this exercise is also to be mindful of without doing sufficient research uh and without being cognizant of how little you know you're gonna have pleasant surprises at the end of your project so good thanks very quickly we'll just touch on this very quickly it's another estimation technique called planning poker and so basically let's say you have a project that you want to estimate you have a series of of user stories in your backlog and then you'll have a team of estimators and hopefully one representing each department so let's say one representing UX design development project management QA each one of them will get a stack of cards with these different types of numbers and then what you're going to do is the product owner is going to read a user story to you and describe to you all the requirements and the business requirements and then each person on the team will choose one of these numbers let's say these are just for now is holding story points it's a point system and then you each estimator will choose one card based on what they feel is the level of complexity for that particular user story and they don't show anybody else so each estimator will take the card and then when they choose their card they put it face down and then when everybody's ready the product owner will say go and then everyone reveals their card at the same time and what you're trying to do is you're trying to get a consensus among your whole estimation team for a particular let's say a particular range or an exact figure and then you have a discussion with your group as to let's say for instance let's say four of you picked a three but why do you have picked a 13 then it's up to you to have a discussion with everybody in the group to say well why did you feel that this user story is much more complex whereas everyone else felt it was kind of like low complexity and so the idea here is that the estimation group reaches a consensus using these cards and then you take another vote again until like until everybody's test degree is going to be more towards a three or more towards a five or the 13th and so the advantage of not showing your cards is that you don't bias anybody because if the first person to show their card first you might be influenced by their opinion and then maybe change your estimate based on that so that's just very quickly to talk about planning poker and I think a lesson out of there is that it's good not to estimate an hours and because it's not realistic that it's ours it won't be so at least this is more honest that it's relative complexity of this task versus another task and then you could say well we take three big tasks and ten small tasks that's a more realistic approach than they're trying to say well according to this thing it says it's going to add up to 20 my dad is doing this to me all the time this is going to all add up to 24 hours so why don't we tell the client it's going to take 24 hours and I'm like ah to me it seems like a 200 hour project guys so let's not do that so when you have points rather than hours it's a little bit more honest that way okay we don't have that we're going to do this very quickly Alex but basically we just wanted to do a little exercise again with you guys so now we're not talking about oceans or titanic or whatever and this is like an actual request an actual request from the client so basically you can imagine a product catalog page and a standard one and of course with Drupal 8 and it's going to have all of these requirements given to us by the client so imagine that you're going to have this page with a bunch of products card style each card must list name description tags product type there's going to be multiple product types that to choose from with different fields and then in the search you're going to be able to either match it exactly with the skew number or one of these search filters and then the filters are going to be by main business unit by product type or tag words and then when you first arrived at the product catalog page we want to show you all the results by default but then as you start clicking on the different filters that we're going to use Ajax to refresh the results on the spot and the other thing the other technology piece is to use solar so we're just going to give you maybe like one minute since most of you have experience with this to maybe just give us an estimate a range for our front end development and back end development and you can use the tables to to discuss among yourselves okay so there's uh yeah so basically we spent uh it was a 28 hours front end and 75 back end for that so doesn't include PM doesn't include QA designer UX but that's just a straight up development work for that so the total that we we logged in our system was a hundred and five not in five but the thing that we should be estimating therefore to if the client asks us for a single commitment number that means we should have probably given a range done no if you ask for a single number which is often the case probably want to go to 125 to 150 for the dev and then you start adding uh all of the PM design QA deployment revisions and so on and then you probably will tell them 200 to 300 uh and uh and this is with the benefit of hindsight and then you might even give them some extra work if you come in under which which happens some of the time okay so one last thing before i move to what was the biggest problem that you encounter in this project like like the search what i think this the solar the solar integration the solar configuration was i was probably the most problematic i would take right for this part so the biggest there was some back and forth uh not understanding exactly what we needed to do for the solar and then we had to go some back and forth with the client to to get that right it was not super complex but there was some misunderstandings the range of hours the range of hours originally estimated 40 to 80 for backend and then 40 to 60 to front end right so we were that solar part um well the the whole back end was 75 and how about you can wait or how long the solar part the solar part i don't remember exactly this way at all like once you make your search do you go you have a link to go to see a detailed page of the product something like that like did this when you get the result no that was not part of it just a page no it was to go for the solar they won't like uh become briefed or uh cinematic or just there's an auto suggest there was another suggest what's that in uh slides man don't we miss it it's not in the slide no that that should have been listed though because that adds extra complexity to to test and extra modules to itself yeah yeah no we did an exact match search for uh sq okay but doesn't mention the auto suggest on the on the search field okay okay any any question guys yeah Alex in case you're interested like we do estimation almost completely different okay next year yeah i'm pretty sure there are so many ways of yeah so we basically estimate based on uh valley delivery so this is more on like feature delivery and like you know solar implementation number of number of this and that right which aligns nicely to fixed price uh fixed scope right we're much more agile in terms of how we estimate so we extend agile not just a software development but also to the estimation process where we'll look at initially like you know like the color of uncertainty right at the initial stage look at like the theme level and look at the epic level and then we'll estimate the value associated with the theme and the epic level based on heuristics of doing similar things in the past and also like uncertainty on those levels right um so that all the estimation are based on that value delivery right because an epic is always a statement of value delivery rather than the specific how long it takes yeah so we work off of a backlog that is low fidelity and then we would find that over a period of time with the client to get the higher fidelity but then the scope always has to be variable um but at the end of the day you know practicing agile the idea is that um although the scope is variable the product is going to be better even if you don't complete everything so it's just a different way of looking at the agile or sir at the estimation process from order of an agile perspective yeah and i would say that we do this implicitly we don't have a process for it this is the start because it serves us like a base of what things cost us almost right and then we implicitly will validate well does this make sense to even propose to the client this many hours for this kind of work yeah those guys so we always do that analysis that you're describing anyway and it's absolutely necessary and we should create a process for it but we haven't yet so maybe we'll go to your time next year yeah any any other questions or comments the name of the one comment kind of sorry um was it i noticed that you had a little like kind of tell but it thinks the one thing that i find that usually is a lot in a lot of projects is production pushing yeah getting an actual production and then any type of care or you have a warranty period that comes after this and i think that's always important thing because we always put it as an afterthought but it costs us money yeah we often put like a hundred hours or something for post-lunch uh warranty and goodwill budget question over there yeah what is the name of that of the QA and DMP? percentages yeah i think we're using in that in that so particular there was believe 15 for QA and then 25 for PM okay last question folks okay okay thank you very much thank you