 So thank you being here. First of all, my name is Maxim. I'm the co-founder of Adyax, we are here also too, and we do only fixed budget projects, a lot of them, and the funny thing that almost all the estimates are known by me, so I'm the guy who is always wrong. So that's why I was like presenting this session. The usual size of the projects runs from 3,000 to 15,000 man-hours, so it's quite big projects. I don't know which was size of you. And so we'll talk about estimates and probably the estimating is quite the most dangerous work because you are playing with money and if you're losing money on the very big projects it might be really complicated for some companies. How much of you did project was losing money on projects like I say the truth. Okay, that's more. Okay, so that's quite normal. I mean everybody makes errors and estimates and from what I know that 70% of IT projects are late and if you are late you're probably losing money on some way or margin at least. So this, everybody knows these scary deadlines approaching and you add people to do more and the deadlines are never met and after a while nobody cares about deadlines and yes I will do it as that and of course in the company if you do not met deadlines you are probably wrong with estimates, you lose money, your team is being like more lexist with estimates, with control of the time frames. And that's the funny thing is that we are quite smart here, all the guys we are attending DrupalCon. We know views, panels, some of you are over in style Drupal8 and know how you're same for me now and again, we're still doing very basic errors and estimates even if you are quite smart guys. But in the everyday life like your daily life you do a lot of estimates every time and you're doing pretty well. I think imagine you are in Prague like right now and you have a good friend calling you saying what that, saying let's having a beer somewhere or wine in a good bar in Prague and well what you'll do you'll probably open your Google Maps, you'll check out the address, you say okay I need 28 minutes to go there, I will add five minutes and it's only four task jobs, you go down, you walk to the metro, you take a tram, you take a metro and you arrive and even there you'll probably know some friends that are always late. Even these simple tasks. So on the project it's like in this project it's very simple projects, you always have some errors in estimate and unpredictable errors. Some unpredictable events what you should have think about imagine you will go there and there is a strike so you'll be late and you have to go by walking to this bar so you have to add these things of this risk to your project plan and to your time frame. Unfortunately some events have so big impact or so unpredictable that it's almost impossible to prevent, imagine there is an atomic bomb throwing down the flag, there will be no bar, no friend, probably no more you, so it's like the infinite journey for you and there is no way to predict this risk. So but in Drupal projects and more in IT projects there are not so many unpredictable events that can occur, basically we could split them into three main groups. The first one is who is doing the estimates usually, it would be the project manager, the salesperson, the senior developer, the guys who is in relationship with the client and actually this is not the guy who will actually code the thing and you cannot have a company with 100% all these only senior positions so the guy who will actually work on it will do it. With junior position so you will have for example somebody will, how much time you need to theme a web form or half a day and then if you give it to a guy and he will like look what's the web form, how it works with widgets, I have the problem in Internet Explorer 8 and then he'll spend two days. So this is a usual problem when you do estimates there's probably a senior guy doing it and that's my idea that this junior will give. The second problem is the perimeter problem or is this understanding or changes in the perimeter so if usual problem like we face every day like I need a workflow okay we'll do a workflow you think about like a couple of rules, a couple of rules and at the end the client arises and no no no when I wanted a workflow I wanted a notification, five rules, permission, user groups and dashboards and everything and then you say oh this is not what was expected and it was written in the specs. Do it. Okay you lose money. And the final is less important I think it's the human factor well that's like maybe two or three times in ad-ex the project manager leaves or breaks a hand and well left hand is not that important but right hand is more. So this is almost all that can occur during a project. But what do we really estimate in a Drupal project? So when you take the code the RFP you will probably have to to split down the project into parts. So before going into details let define some variables. We'll use those variables all around the presentation so the first thing is the complexity of the website. Well it's it's statistics okay based on the 300 projects we've done so far but you will probably recognize some patterns there. There is a very simple website like not so much content types no workflow very simple data flow well some templates classical small company websites. Then you have the medium sites with more content type with some external data sources from XML some workflow custom business features maybe some data migration and finally we have these complex websites the big one with transactional e-commerce website community websites with a lot of content types templates and and data migrations and external web system like ASAP or whatever. On the other hand we have this front-end thing because a lot of Drupal websites you're actually doing theming so we'll also split them into three. We have like standard desktop output blocks and Internet Explorer 10 support only with some Chrome Safari nothing complicated. Then we have this mobile theme but not for all the features and finally we have this full responsive design like fancy thing like squeezing your website web browser and everything moves around with like nice advanced G query animation so this so we'll use this F123 and S123 later so remember it. So the first thing the guys in doing Drupal will do is like setup models there are a lot of models in Drupal 20,000 of them so you spend a lot of time selecting models creating content types configuring all of them user roles permission rules views everything so it takes a lot of time but you don't go into deep details at the beginning so you'll spend probably some couple of days for small sites and more just about the figures those are statistics related to our projects doesn't doesn't mean that it works for everybody but I think that the relationship between the three are quite okay so well it's complicated to say estimates it's an estimate of estimates. The second thing is the development platform setup many guys forget about to put it in their code but you take time to set up red mine user accounts, mailing lists, MySQL if you are running your websites on Aquia or you have to also set up their things, Git account, GitLab whatever so you will have to spend time and often you forgot to put them in the code. The second thing we have to talk is a context this is a quite complicated thing because it's like a cancer inside your Drupal side because when you talk about context you take talk about path auto and URL setup you have advertisement, tagging, URLs, page title, context, micro data everything and all those small tasks because it's not complicated to sub page title path out with like taking hour but if you multiply this by the number of content types and if you add a lot of context to this this can be really painful and take much more time that you think it would take for example we if I will give you an example we got a website e-commerce website with only 30 pages and we asked it to the client guys please provide me the tagging for analytics so we was adding like no problem I added three days to do that it will be okay and finally we received 70 pages tagging specification from the client with every single click tracked down and the cart and to check out with different business logic everywhere so we spent two weeks doing that and again the guy said it wasn't the specs well actually not the specs but we'll talk about the change request later then the most important for me I think even if if you are late you can leave just after this slide because because the templates are actually I think there is a direct correlation between the project size and the number of templates of course you have some particular websites with complex business logic or complex external systems but again templates for me is if you know how to count your templates you almost covered half of the working estimate because there is a lot of work on templates if you take a website classical website you will start with sketching with the client you will work with this client to sketch the all the websites templates like contact form sitemap homepage article product page whatever then you will probably do some wireframes to validate it with exact proportions and and fonts and things and then you have some back and forth things with the client and then you have to design these templates and probably you will need some static HTML for these templates to validate it works on all responsive and multiply this by if you have a responsive with three back points you multiply by three and then you have your developer at the end who is doing the templates actually creating the TPL files in Drupal so at the end adding one template adding a lot of work of course these numbers again depends of the complexity of your templates of your site but well yeah if you have 30 templates if you have a site which has 60 templates I think the budget will double it's quite good number so next point is data migration it's a quite complicated subject because it's very complicated to estimate if you are running with a Drupal to Drupal data migration it should be okay you have a lot of modules migrate feeds whatever if you are running a project with some structured data MySQL MSSkill it should be okay and then if you have this website with no only title and HTML inside and the guy said please if you have the bold put it in as a teaser if you have ULLI put it in this multi line multi entities well it you will have a lot of pain so usually when you got the RFP you don't have any big so much information about the data sources so you cannot really estimate exactly then but what we've noticed is that it's relatively dependent of the number of content types so usually to migrate one content type for example article from Drupal to Drupal it will take you one day just to figure out from structured debit two three days per content type and if you are going from HTML don't well I cannot estimate you just say to the client we'll see or add some numbers from bank okay we'll see but now it's quite complicated because you will face a lot of problems with HTML migration another thing guys who do estimates always forget that it's really complicated is the number of deploys because actually when the deploy is a crazy thing because it's quite normal it's quite easy well we do agile we'll deploy at each sprint it's okay and actually it might be a real painful thing because depending on the number of deploys you will have in the project and then in the hosting company you'll you'll using the deployment be right real nightmare I give you an example we have a website where the hosting company wasn't aware of Drupal actually wasn't aware of hosting I think but but the guys was deployed was manually using FTP and uploading the entire website no differential deploy so imagine the other guys developers have to create features who back on them together created a tar file and send them and then they deploy manually and they forgot all the time they deploy because they have free front-end web service and they forgot one and random errors and everybody calls us saying hey guys you did crap and we did not crap okay proof okay so depending of your of your partner is hosting company the number of deploys it can be really costly so usually you will spend like half a day with Drupal Clouds because there are a lot of them now and for Capistrano so it's classical hosting company but you automated the deploy process you have like a day and also free days but again these numbers remember that during a deploy you not only have to deploy actually the thing but the developers has to create features for features doesn't work all the time you will have to export views you will have to create module updating you will have to put it all together deploy test the deploy test the rollback if you want and then deploy actually to production or preproduction so it takes time and if you have a deploy each screen you'll have to think about this time the test in the QA well how much of you have a dedicated test teams well yeah of course I wonder why developers do bugs actually have some of my developers here looking at me but well everybody do bugs so that's why we need testing testing might be a dedicated team or or actually your developers doing only tests it's it doesn't matter but what we noticed is that testing is directly related to the number of hours of development this is kind of empiric number it works all the time so this is quite I'm quite sure of this estimate I include in these tests of course not only manual testing but also HTML cross browser testing and reaching automated tests with Selenium or simple test whatever so if of course it depends on the complexity of the website because for example if you do a e-commerce website the developers will spend like two or three or five days doing the checkout process but there is a lot of lot of features inside the checkout process you will have to test it in different scenarios cancel rollback go back and try again so depending on the complexity of the website and the business logic some business cases might be very easy to code and very long to test import export things like that take time because you have many use cases well then then you have the management this is again this is very different from one client to another basically depends of your organization how you manage the project how smart your developers are how smart your project managers are how is your relationship what we've noticed actually is that it's not really related to to the project itself it's more related to the client I've got the client with them with a social network close social network with kind of medium-sized project and everything was quite okay and and it was a nightmare because five person knowing nothing about web calling all everybody every day and asking things like I cannot connect at my site at adjax.com I suggest you tape it what adjax.com but no ad and Arrabaz think and say yes not normally it's not writing any mail and all the time during one year it was like this so for that kind of clients you will need like much more project management but usually these figures work quite well again it depends of the size of the project because usually when the projects goes bigger you have more you have to face more people in front of you at the client side you have the functional guy the marketing guy the design probably the top management thing so you have like stealing committees thing so you'll have to spend much more time last thing you have to quote also specifications actually who writes specification for each project well that's not enough guys yeah that's yeah I think that's that's that's a big error if you don't write specs because well with all this agile thing yeah we don't do specs we do sprints okay but actually specification is the only way to not lose money on a fixed price project because even if you do agile we do agile sometimes with our clients but we write specifications sprint by sprint so we have a quite Bible of the project so when you write specification in the most detailed way possible the client will never say hey it was it wasn't my original RFP whatever you say guys you validate the specs once you validate the specs everything else out of the specs will be an evolution or change request and this is quite easy to to discuss because you brought the spec he validated it easier to discuss rather than going back again and again to the initial RFP and saying what was this workflow thing so well yes again this these numbers are quite well it depends basically on the size of the project but really rating specs is quite really important but it's the only way have you wait you can manage this change request later wait we forgot the most important the features I mean the all this business logic like makes your website unique and different from the all others of course we talk about theming you Drupal setup things but the features are are quite unique well there is no magic rules to estimate features it's like everything you have to split it in the small parts the more precise you will be in your code the better will be your estimates for example voting don't put like three days for voting try to think about like voting I have to install five star voting API set up all the content types that are touched by the voting I have to theme the five star then I have to create a block with the most voted content I have to theme it and I have to integrate this in my website and I have to deploy it so the only feature that seems very easy five star and voting actually is not that easy if you split it in small parts and you think about how much really I need time to do each small part that's this estimates then when you do estimates basically you do it against the RFP or tender or specs and then you will see like you'll probably see like everything going from one page one email RFP going to 300 pages RFP well we got all of them and it doesn't really matter how much money the client have it's well that's the culture so again we've split it this RFP into three big parts the first one we have the user centric RFP so this is the new kind of agile thing we see in the RFP right now guy saying as an admin I can create a content type as an admin I can as an anonymous user I can go to an article page and this for 300 pages well I don't like this kind of RFP but there is some advantages because you have a detailed presentation for the feature because it described on different personas you can really understand how the feature they want to implement work because for example for our workflow problem they will describe as in for each role they should describe how this workflow works but you to create to estimate the feature to understand it's the definition of this feature is spread across several user stories so we have to summarize it in one piece which might be quite long and then you will be have to be very careful with the templating counting because usually guys doesn't present some wireframes so as a user I can see articles okay you know you have article templates you probably had some home page but that's kind of complicated to count the number of templates the second type of RFP you have is the page centric RFP so you have a bunch of wireframes or designs sometimes annotated sometimes not like this is my website please do it this is good I think I prefer this because actually you it's easy to count in place and you have you reduce your risks of misunderstanding but you will need to be very careful with all the business rules and the back office and because we got the client for example for real estate websites and the little blocks like these promoted apartments and actually we would like okay five days to do that and what the client actually wanted was much more complicated because they wanted like very complex business logic depending of search result which search result is a Joe search result using Apache solar with if you are promoted and if you are a partner near than 50 kilometers then push this block there well and all this all around so you have to be careful with these templates only RFPs and finally we had this feature list this is usually all school okay I want watching I want comments I want articles I want homepage do it well it's easy to get teachers okay but you have to imagine all the context templates asio and all the stuff behind the features and the difference between different videos might be really important that the thing finally had this hidden costs because when you are reading the RFP many things are not described and again if you're wanted to provide a good quality job you have to bill and prepare that so for example you have the back office clean up okay Drupal 8 looks good so the keynote but Drupal 7 right now and you have to clean up things because you will not present to the site administrators and contributors all the crappy things with path to ring or whatever so you will probably need to spend some time of cleaning up workflow notification user permission the client usually don't think about that but they wanted actually so you have to probably add them as an option do I why do you accept up it's a classical thing I will add CK editor that will be fine and then you will see like crappy red things climbing inside the your side because the guy's adding copy paste from word or whatever so you will have also to prepare like make the YZ big render exactly like the front end does and things and of course you have optimization Drupal is a slow CMS by default so you'll have to add caching memcache varnish edge side includes probably some CDN and all these together make style lazy load images it's small things but you will spend time on it and the client will not think about it before one thing we've noticed we lost a lot of projects because we were adding all these things inside our response so we was like guys we're a small company how maybe how the hell we make we are more expensive than like kept Gemini it's just why and after that we understood that there is some tricks when you're answering especially public questions about how to avoid being the most expensive the first thing is being the most precise possible in your quotes at the beginning we was like putting 20 or 10 lines in our quotes and then we ended up with hundreds of lines in our quotes why because the first thing is it would be easier if you win the project to say a guy's look there was one line for that there is no line for what you're asking me for sorry and then it's much more complicated to discuss for example if you say okay theming 25 days and the client say no 22 okay and by I signed today okay 22 but 20 okay 20 and if you have one template saying three hours here home page five hours article three hours product page four hours this point four hours well you'll say four hours for product page yes four hours three no four okay so it's much easier it's much more complicated to negotiate with you if you're very detailed and precise and the other thing is that it shows you are like understand his project and not just like putting 20 days so then once you set up all this bunch of lines in your quote take care because you probably will be the most expensive because other guys doesn't think about everything and will not put everything so put options put everything as options back office options everything that is not asking directly in there if we put as options and then you will have the chance to explain to the client one why you put this option and why he needs that so we'll enable the discussion and then you enable the contact with the client so then you can maybe win the project because you are talking to each other and you show to the client that you understand his needs and you think more than just what he asked it for. Of course in years there will be a lot of unclear feature and you will not have the possibility to discuss with the client before the release of your answer so take the low estimates and explain exactly what you will do don't ask for workflow don't put workflow three days check workflow using rules module with three roles and no notification with one dashboard three days so when he asked for five dashboards and five roles this is sorry guys we wrote three so it's more precise less problems or you can do like this some big ones we we know that they do that you underestimate the bill and hope to get money from their own this is a very dangerous thing we don't do that at edX but I know some companies will do that because to win the client they sign the bill they lose money on the bill and they know that they'll have support and MCU and things like that so the build phase well now we now what to estimate we know how we know well we try to know how to answer the RFP now we we want the project that's funny and that the problem starts because then you can really lose money because if you don't win you probably don't lose it so the first thing is to measure the time because well we're doing Drupal shops we're running Drupal shops we pay salaries salaries is money based on time so the more time you spend the more money you lose so you have to measure it if you don't measure it you will not be able to control anything so first thing is it's set up time tracking software RedMind does it for us but you can use whatever you want and and sit and ask everybody time logs it's like just mandatory everybody who's using time logs actually here wow yeah good well you need to to time log everywhere project managers everybody and usually what you should do is one line in your initial quotation should should be related to one task in RedMind and then all the issues like bugs and evolution should be attached to this super task so everybody looks time and at the end you can say the difference between your initial quotation and actually what the guys that did so not only you will see if you are losing money in the project but you will see where and then you can check out with guys who was responsible this task why we spend much more time that we shouldn't what we also create is a project backlog with on the left my initial code and then the real tasks on the right so we can say configure certain types it seems to be like we lose time on that but actually this is a very good way of control because as the developers will feel these things actually they can warn you saying guys we are going out of estimates because the project manager cannot think about everything on the project he's like doing client developers testers have problems and and he will not be having an eye on everything these kind of things will help you and then we have per spring backlog with estimates and the real hours that was done so to sum up with this everybody must lock time that's a mandatory you should keep the link between initial code and the next and one thing we noticed is that before the estimates wasn't shared with developers we was doing like we do estimates with gift ask and then we like they see us red like our guys you should speed up they understand that we are out but the we didn't share that now we share the estimates with everybody in the project every single guy working in the project knows about the estimates of the world project it helps like to have the spirit of control and you have to stick to the plan if you set up time logs and specification do it even when you have panic that's if you have panic stick to the plan don't don't cancel time log don't stop the specification it's it's matter to have been late and explain to the client you are been late than to completely destroy your process inside your company and then the credit this is the way we found out how to manage evolution change request so imagine the fact that you have answered the very detailed way your quotation you created your specification for each sprint everything is covered and then the clients ask something more so the good thing is to set up some kind of shared ex less file with the client where you put the feature he asked the estimates and and and the total at the end this to avoid the idea is to avoid sending bills like please sign me this please sign me this is can be like harassment during the project 300 here 200 small amounts small numbers but here's receiving this please sign please sign please sign it's not okay so set up a policy with them with your client saying we are okay by mail and then you have this control shared doc and he is he sees the total number of amounts and at the end of the project you just build a total so he's not frustrated he's not well I think it's better and what we also introduced since this year is a stop day in the middle of each project we stop everything like for a half a day or for a day and looking okay let's let's check what we have to do and what we still have how much burn we how much money we burn so far it's very important that everybody stops and speak by skype in a room whatever about this like let's go with the middle what what we have still to do and how much time we spend because it when you're doing the project you own spritzer is print release doing code bugs and you're not like you don't take a step apart to check out the problems finally what's really makes your project going wild is this never-ending acceptance period like yeah yeah we feel finished like eighty five percent on ninety five percent of the future it's okay and then the project I have acceptance content acceptance problem bugs and still and that last four months and this is what makes you lose money you should define at the very beginning of the project the acceptance period time and and you should be very very careful with the client a very strong with the client saying guys you have to X you have to test the website you have to report bugs be careful you have only two weeks three weeks one month three months whatever but you have a fixed timeline it doesn't mean that the you will have to solve all the bugs during this period but it shows to the client and to your team that you have a fixed time for the acceptance and then the site will probably go live and then you have this guarantee period so you you must define this time yes and and also you have to define with the client what blocks the release that's something really important when you start your your acceptance period you have to talk with the client saying okay guy please when you open a bug you put blocker only bugs bugs that actually blocks live so in this way when you define this rule as a client he's okay to go live with some bugs which which he would be reluctant with an acceptance period when you didn't define this blocker issue definition with them so once you set up that he understand that blocker means we stop everything and we solve this issue and is blocking for the live so it helps to time box the acceptance also you have to the client are very scared about what happens after the day we released so you have to explain him this guarantee period I don't know your contracts how they put three months six months two months whatever but you have to explain him don't worry will be still here to make him go live because your guarantee period you're probably three months or whatever is by contract so you know that you have to go live as soon as possible it's better for you so you have to calm down the client explaining detail what happens during the warranty period well the last thing is the support and maintenance some private job for French guys support and maintenance is we noticed that you lose the project stops like it's going live use you was you you you was paid for the for the project this was okay but you don't stop to losing money because the client will still call you I don't know how to create an article oh please I think everything is broken on our own homepage and usually it's problems of contribution maybe some bugs but usually some kind of support tasks and it's very complicated for especially for a small company working with big clients since one year on a project like saying also you cannot call me you don't have a support contract sorry bye it's it's not possible so to avoid this kind of discussion at the end of the project again talk about support in the middle of the beginning of the project saying guys we have a support policy by ticket fixed price whatever but we had this support thing this is for you to help you do you want to take it take talk again during the project and intensify this talks before the release date because if you if you start talking about support at the end of the project he said like it will be like it'll be complicated to take to talk with the client about support of course you can also go with some aqua or commerce guy support whatever but if you probably do support on your own projects you'll need that ticketing system or whatever so well that's it so well if you have some questions you are welcome I'll try to answer them you have actually you have the mic Microsoft it would be easier if everybody's can hear you but okay yeah thank you I have two questions first on the functional specification into what detail do you go you talked about being able to point to the functional spec and saying hey this wasn't in there so that's going to cost extra that's going to be a change order the problem that we run into my company is that often the functional specification is too detailed for the client to understand so they end up signing off on it but really not fully understanding what they're signing you know there was three content types well what's a content type you know they don't they might not know yeah and that's that's true actually actually this is a technical specs we split it like functional it's really like blocks and wireframes and yeah the level of detail the specification is quite complicated that's why on big projects we try to split down the specification to sprints and propose to the client specification before the spring so they don't have like five hundred pages but our usual specs are like six or five hundred pages for big project you have to explain we spent half a day for each spring to explain the specs and read them with a client so if he has questions he asked us so we make sure that he actually understands but again we asked to sign because otherwise well you cannot and if you don't put enough details then you will always have some extra discussions which are not very good in during the project to so well explain the specs and split them in small parts I think content types it's a quite easy to understand for some clients well if you explain show him and yeah yeah I had one other question so when when you talked about you guys use red mine for your ticketing system we use open atrium and we actually create accounts for our clients and they're actually in there in the issue queues you know resolving the tickets and responding and you know we ask for their feedback but that's sometimes problematic not having a sort of separate internal issue queue and sort of an external so I was wondering how you guys handle that why it's a problematic because then you know we like to be open and transparent with our clients but sometimes you know there are certain things you might not want the client to see or you know just sort of being able to you know have issue queues that are sort of more technical and maybe separate from the client but then also you have to communicate with the client as well so it's a tough balance well we use only one system that's what's often ask it but when we work for some agencies like subcontractors they ask us from separate red mines but when we do indirectly like 80 85% of our clients and direct we always they all everybody in the project everybody in red mine and usually guys who are not technical enough of here's not they are not interested they never login you see them like never login okay well being transparent and even if you have problems it's a good thing because what scares out the client when you stay we are delayed for example he he can understand you being late but he wants to understand why and how to avoid this so if you see all these people working nights and days and logging and commits if he even if he doesn't understand he see the activity of the project so he knows that you work hard on the resolving problems so we offer all access to the red mine everybody sees everything there it's quite easier to explain things like look it's inside there they are there if you don't care you don't you don't look inside but it's easier I think from my point of view I know Hi I'm Taco from go grill at the Netherlands thank you for your presentation I think you did a really good effort using statistical data to solve this problem and as you said 70% of the ICT projects in all our countries we can name one of you usually from the government they're very good at this that fail and I think this is one of the reasons why there's so many people here because it's not the you know all the tech guys and we can manage this but but real good project management and a good method of solving this issue is very important for any project to succeed so you've done this exercise by making these estimations do you have data on using this method for estimations and real project data and the delta data yes about the data yeah so the delta between what we estimated the real numbers yeah and and my fundamental question is especially for S3 projects we do a lot of this can you really estimate in front because I saw a lot of 15 plus yeah I know migrations that take like weeks so can you really make an estimation beforehand of how much work it takes to a project well usually what happens you can do real estimates mark how wrong they will be that the question actually is every estimates are good estimates at the beginning yeah so well that immigration is a good question it's a good example my question is should we do this like aren't we going down the rabbit hole and shouldn't we just move this to the past and go to a much more agile way of contracting which we did two years ago my life has become much better my developers life has become much better like our clients are much more satisfied yeah and I see you're taking like sprints and backlogs you're slowly moving into this agile way but but if you would take the step and lose the specifications go to agile design we can really do much better projects that are on time that are in budget and have a really good quality just we have this flexible scope yeah I totally agree that that's in theory in practical it's it's the truth the only thing is like how to explain I know that in northern Europe it's quite easier to to make this agile thing in south Europe or in US or or in France for example if you have a tender with like half a million tender everybody will have to answer the fixed budget they don't want to hear about like we'll see like agile will adjust the budget well it's not about this is the thing if you if you say this you do not understand agile because agile works with a fixed budget so you can promise your clients a fixed budget fixed time frame just not not to fix scope so every time somebody's talking about fixed price they don't say all agile projects are fixed price and this is the thing we should sell our clients you get a fixed price you get a fixed deadline and you get really good quality we just cannot make the promise of what you will get yeah but the perimeter then it's not fixed this is a tough sale I'm not I'm not saying this is easy because the trust is very important here you need a good track record but if you can make this case and if we all should do this like don't promise these things you cannot promise there's no way you can promise an integration with a third-party tool beforehand this is like even if you documented like 200 pages you'll still end up with things you cannot expect beforehand I mean why not stop this altogether lawyers aren't you know saying like okay I'm gonna do this for 10,000 euros beforehand now they say well we're just going to do the project and we'll see yeah the problem is that you have a lot of like you don't see the client before the RFP answers so you will write down yes we'll do that but you then we'll ask like confirm that this is included in the perimeter and you don't have even a call with the client before so you're not beat for that kind of projects well there are a lot of them like sending a bigger fp 500 page you don't have the permission to communicate with the client before you have to ask questions by mail that is sent to all the other all this business and then they ask to confirm the perimeter the exact that yes I will do that yes I will do that yes I will be these are not the clients we will take and you know because there's too risky and you know some just saying there's there is an alternative and I really like your effort on on doing data analysis and and trying to make conclusions and we should and we should do all the time walking but I think there's a much more better yes I know of doing I hope all the grants will move to that well we should all do it I mean let's start thank you you meant you mentioned about stop days yeah do you share the fact that you have stopped days with your clients and do you ever use that as an opportunity to assess reassess the quota maybe go back yeah that that the scope analysis of workshops of discovery workshops I if you can place that with the client is quite very good unfortunately on many RFP you have like one victory answer and even you don't have the time to set up this workshop we will we'll set we set up these workshops for very big projects but even this discovery workshops the scope analysis is still estimates because you still can't discover many problems within within the project but yeah that's definitely the we do all we try to do all the time this scope analysis with some workshops you may have misunderstood the question I was talking about the stop the stop day that you mentioned sorry the stop day scope guide the stop stop day with the stop day do you let clients know that that's in and do you ever use that no it's more and no no the stop is more internally with the team saying okay guys are we running out of limits or not it's just an internal to check out everything review the specs review what we've done and review what remains to be done so it's like a re-estimate in the middle of the project for us just to how to to know how to adopt our our behavior add more days on offered feature or or a stop down and slow down with offered features me more careful with evolution and changing questions just in the middle of projects stop and look and inside your numbers first I want to say thank you that you are sharing this knowledge to us I think that this is the key for giving better estimates just to share our knowledge how how you do it how how did thank you and then I have a couple of questions the first one I will try to be fast the first one in your company where is then between a bug fixing phase the acceptance phase and starting the support well when the when the sites goes live actually that's it's what's kind of good date if everybody see that well even if on some project life is not like but before life we are in bug faxing acceptance period and then once we've once we've gone live we start the support and warranty period but you mentioned something like one the site only with fixing of the critical books or or issues and then the fixing of the rest issues when the site is live yes and all these bugs goes in the support phase yeah and actually this is the only thing this is because the acceptance if you are late the acceptance may go very far away but you if you make accept your clients going live then by contract your guarantee period is fixed in the time so well it's quite easy then to discuss I'm guys we are three months after the release there are still a couple of bugs but now we have to signing MCO or support contract for going so it's just juridic things thank you and then one one more question how do you estimate the third-party integrations I know that this is something like there's no mantra about this but how you do it well it's usually depends of these third-party integrations very large yeah or you have data sources of content then usually you have to check out by content type like if you import events from an external system check out the system if it is documented if it is a standard protocol it's quite easy you would you check out how much time you will need with feeds to integrate this content type if this is SAP for example for e-commerce website well you have to check out how much interaction web services you have to call well going to split into details well this is kind of features well you have to split into details and check out this because it's so different it's external data systems and my last question sorry guys really the last one how you share this knowledge in your company because I see that you're yeah smart guy but but how you share this knowledge in oh well we we we have these estimates uploaded in red mine in the wiki with all with google docs with all estimates are shared with all guys in working in the project I mean knowledge of estimating ah well we talk a lot with the project actually estimates initial estimates are done by me but then project manager do all their changing question estimates so well we we talk a lot that we don't have any process of sharing yeah well so we talk like this okay thank you very much no problem and thank you for the presentation it's excellent and some really good jokes I wasn't sure if it was about the art of estimating or the art of replying to RFPs because one of the points that you made at the beginning was that senior people tend to do the estimations and junior people do the work and the principle in agile that the people doing the work of the people that should be doing the estimating yes that's true but that's that's exactly but if you receive again and like when how do you win projects sometimes guys call in and then you discuss and then you set up the project and then but actually what happens in in in in France for example I don't know in the US but in France is you receive in 300 pages RFP and you have two weeks to answer saying how good you are how good your guys are how much it cost that you confirm that everything is inside and then if I send this in French to my developers who not all speak French they will spend like hours to do that so the problem is that if I ask to the theme to estimate the templates and to the developer estimate teachers then then we'll probably have to spend like five weeks to estimate because so we need to go fast and that's why usually guys senior in senior position do the estimate because they know how to understand what the client actually wants behind the lines the developers probably will not see all the tricks inside the RFP that are not detailedly explain so this is why I don't think that actually developers actually developers are very good in the estimates when they know exactly what to do but at the stage of the RFP nobody knows exactly what to do you have to project your RFP into a website with everything split it out so this kind of complicated job and it takes time for developers who are actually developing other projects well that's why we do this way seems to me you might be at the best of both worlds you're using red mine you're creating high-level epics for everything you've put in the quote and then developers are having to break that down into tasks yeah and are they doing the estimating as they break that down as you go down to more detail yeah and once they did this estimate one so we are in the build phase they did this estimates we are able to check out oh fuck we like five days for in our initial code and the developers put like 25 days what happens so we can dig into the project manager spot this out can dig it well guys maybe misunderstanding on scope we can talk with the client to raise a little bit estimates or or slow down developers saying we don't need all that things inside so this is a good way to check out where the problems are thank you hi there I'm Gretchen from Drupal Squad and oftentimes when delivering estimates to clients I get a common response of they get out their red pen and they say oh I will do project management or no we will do QA or I will do content management strategy do you a walk away from those bids or do you B have a formula that you apply to your estimates that absorb the inefficiencies when that happens on a project not sure I get the question exactly so a client decides to do one of the roles themselves where oh yeah they want to have friends on the code you mean if you want not the code but project management or business analysis no no this is somebody can explain where the client decides to play the role of project management of a project manager or a BA do you accept those bids well or do you build in additional time for an official we always add additional project manager because I can't remember any of my client maybe a couple of them like they're able to manage actually external developers located far away and again the problem is that you've set up processes inside your company you have like your theme is otherwise don't have them as we have only we do for example we don't is fixed static HTML before doing templating all the guys doing a different way so when you have a new project manager like external they will apply his methods and then so well that's we always add a project manager so do you have a basic percentage that you work with or is it case by case well it's case by case of course because well again if the client is a tech guy with a good team and it's a small project we add lower figures if the guy is like marketing department with a crazy ideas and no idea of text and there is a very big project we add like 25 percent of project management well it depends of the size of the project because if we have only one guy in front of you with big project you have ten of them and everybody's talking and arguing so well you have more person thank you no problem thank you for your talk it was a lot of interesting facts and statistics however I have a couple of concerns and the most important is that if we are talking about s2 and s3 project size it by you I wonder if you have a chance to see how the project size correlate and numbers you provided correlate to team size because developing the projects with the two or three engineers in the end and with the team of ten or fifteen if we are talking about big projects it's a big challenge and usually there is a lot of overheads which should be estimated pretty well yeah but well that's the same if the project is big probably the team is big and then the then you have this overhead yeah that's right and and and I would say that the even most important is also the client type it's it's subjective so subjective you cannot decide like you are a good client you're a bad client but you know all of you like know this this this complicated client that cannot understand anything and don't want just cannot understand the agile they don't want to understand the specification in changing every time and then more technical organization so I think yes of course the the estimates based on the size of the projects are depending of the size of your team bigger project is probably bigger the team is so yes of course there's overhead though I I tried to put these numbers based on the size of the website but you can just replace size of the size of the team well okay thank you and one more small point about specification you said that you're trying to write specification on every project however who's paying for this work about the client is paying because we put line with specification what will you do if client you showed your estimation where there is a lot of numbers well descriptive stories then clients and head I don't need estimation I don't need specification I don't want to pay it well we split that we actually need yes yes well we don't need specification actually you well you need the say I don't want to pay it well this is important for you not for me well this is well it's important for me but it's also documentation if you can explain like a guy if you don't want to see if he wants to stop work with us after one two three years you will have a complete Bible of your project and then you have like 20 percent off and it's okay thank you thank you okay we're running of time thank you very much I guess it's more like as first of all thanks thanks for your presentation and opinion but I would also like to join to measure land guy yeah but I don't want to want to craft right yeah I'm easier your side like yeah I'm not there we are stupid in France like you know no no no no I would I would like to say this process we do describe this really good for minimizing risk for vendors yeah but actually like idea or a invitation from from this land guy which wasn't from undercut but yeah and ideas we actually would need to figure out how to fix problem of client that we can define project on detail level for whatever five hundred thousand our project yeah it's like impossible so this is the way how we can minimize our risk as a vendor but yeah doesn't actually help or fix the problem of the client and like project that I agree there is a session from ex-guy from not one I think about this kind of business-oriented things but well we're talking about estimates but yes it was really agree yeah thanks thank you bye