Creating an estimate for a software development project is hard, really hard. There are books and articles and speeches and academic papers, and you know none of them has got it completely right, because projects are still badly estimated. So what am I going to add to the mix? Nothing, really. Just a set of tools and techniques you can use to help you improve your estimations. But first off let’s dispel some myths.
Myth 1: Estimating isn’t necessary for Agile
Well, if you work in a company where they’re happy that the date of delivery and feature set be vague, then good for you. In the rest of the world we often have to present a quote to a customer that specifies what they’re going to get and how long it’s going to take. This means you’re going to have to estimate. Happily most estimation techniques work fine with Agile, they just require that you get an idea of your scope up front.
Even without a requirement for a full estimate, you still need to estimate the scope of the user stories to see which will likely make it into the sprint and which won’t.
Myth 2: Developers make good estimators
Sorry, but we really don’t. Look at our track record. We tend to underestimate tasks we consider “fun”, and overestimate those we consider “boring”. It would be great if that added up to the right amount, but it turns out that we suck even at overestimating, and generally forget about huge swathes of requirements when asked to “thumbsuck”. Oh, about giving a thumbsuck estimate: don’t, ever. It is about as reliable as throwing a dart at a board, so you might as well just do that instead. If you’re being pressured for one, consider a parable:
There was a pilot, who often flew from Paris to London
He was asked how much fuel he’d need for trip between Madrid and New York
He didn’t ask in which direction, nor the make of plane, nor the number of passengers
He just pulled a number out the air
And the plane ran out of fuel mid-ocean and everyone died
If you think people’s lives are not affected by poor estimating for software projects, think again. I’ve helped rescue some projects where poor estimating had badly affected code quality on systems affecting people’s medical aids, finances, and yes, the possibility of making an airplane fall out of the sky.
It does not take long to do a proper estimate, a few hours at most, so do it properly.
Myth 3: Estimates aren’t important
A proper estimate serves a broad array of functions:
- It lets you know when something is harder than expected, critically important knowledge in most projects
- It allows you to synchronize timelines with other departments. For our Document Management System we delivered the first version 4 months before the web site and marketing was ready. Ooops! Admittedly my software delivery estimate was spot on, but I hugely underestimated how long the marketing would take.
- It allows you to determine what your profitability would be on a project, which can mean the difference between costly failure and profitable success. You can walk away from projects that wouldn’t make you money. Otherwise you just don’t know. In the early days of Palantir, before I joined, there was a project like that, and it very nearly sank the company.
Okay, so let’s move on to some of the techniques:
Technique 1: Always keep records
This comes from Joel Spolsky I believe: always record your initial estimate, as well as the final result. Keep those records and use them in future estimation techniques. They allow you to build up a pattern of how various people estimate. Make sure you break up your estimates based on who gave which one. It is not embarrassing being off in your estimates, what is embarrassing is being off consistently and not factoring that into your future estimates.
Project 1: Estimated 50% below actual (ouch)
Project 2: Estimated 45% below actual, 50% below quoted (yay!)
We learn from our mistakes. Not doing so is insanity. You can build up an estimation factor to apply to people’s estimates to get a more accurate feel. Just make sure you update your estimation factor constantly, as their skills will change. Make sure to keep it isolated on project type. My estimation factor on Windows Services may be 80%, but on ASP.NET WebForms it could be 120%.
Technique 2: Multi-estimate
Have multiple people independently give estimates for the same item. One advantage of this technique is that you get a broader base of data for Technique 1. Probably the most important however is that it provides a level of confidence.
Assuming estimation factors already applied.
Item 1: Sean estimates 6 hours, Graeme estimates 7 hours, Craig estimates 5 hours. Standard deviation is 1. So, we can say the estimate is 6±1 hours with high confidence.
Item 2: Sean estimates 3 hours, Graeme estimates 7 hours, Craig estimates 14 hours. Standard deviation is 5.56. So, we can say the estimate is 8±6 hours with low confidence. With such wildly differing estimates they could all be way off.
This is a hugely important technique, and the least often applied. It offers huge benefits in accuracy as well as feeding nicely into Technique 1.
Technique 3: Constantly compare progress against the estimate
This is often done as part of standard project planning, but the real-world data is not fed back into your estimation. As you go, you can start calculating a new estimation factor which is specific to this project. You can then apply that back into your original estimates to get an updated idea of how your estimation might differ. This would mean you now have the following:
- Original Estimates
- Final Estimate – Calculated from Originals with factors applied
- Committed Timelines – Hopefully somewhere north of the Final Estimate
- “Current” Estimate – Recalculated from new estimation factors generated from progress
If your current estimate is creeping upwards towards the committed timelines, you need to raise that as a problem, before it becomes a problem. You might also find that one persons estimates for the project seem to be more in accord with reality, and give their estimates more weight, but be careful to keep the uncertainty values in place.
Technique 4: Be detailed
I get very suspicious of “2 week” items in an estimate. Sounds like a thumbsuck to me. Joel says you should break everything down to 16 hours at most. I prefer shorter even than that, but could live with 16 hours.
So why break it down? Well, it turns out that we’re really bad at estimating all the pieces that go into a big task, but pretty good at estimating small tasks. So, by forcing us to break it down into smaller items we’re required to think more about the makeup of each task. How much of a difference can this make to your timelines?
Oh, about 50%.
Oh, and make sure you include everything in your estimate, including wasted time waiting for third party vendors, holidays, sick leave, maternity leave, scope creep, project start delays, document signing delays, testing, debugging, the likelihood of having to rewrite a module or two, the lot. You can use the data you accumulate to feed back into future projects, improving their accuracy.
Take those variances you got from each developer in Technique 2, plug in their historical variances, and use Monte Carlo simulation to generate probability distributions. Now, you can confidently go along and say, “we have a 90% chance of hitting X months”.
If you find a great deal of variance on a task, it likely has not been scoped well. Consider investing a little more time in nailing down the requirements, and then re-estimate. Yes, still keep the original estimation data for historical reference.
If you find that some staff are more accurate, don’t use them in preference to everyone else. They could always have a bad day after all. Rather ask them to share the techniques that they use that make them so accurate.
Estimation must be seen as a high priority item, one of the highest. It’s more important than the project plan, more important than the specifications, more important than the actual development work. How can I say this? With a badly estimated project, you can do the development work perfectly and still make a loss.
Estimations are also not made just by padding them. Rather give the real numbers as accurate as you can to management. They can then use that as input on their decisions about whether to go for a project and how much to charge for it. If you pad your estimates too much, you could lose out on very lucrative business opportunities.
Do not get pressured into removing the error bars. Make sure you include the ± variance, or if you’re using Monte-Carlo (hopefully), the percentage probabilities. The nice thing about Monte Carlo is that you can give a 100% number if pressured for it, but it’s usually way more than the 90% figure. On some projects we go with 80%, on others 90%, on a few 95%. 100% is usually not worth using, unless millions ride on the delivery date.
Any others? Share them here.
This article has been recovered from an archive from my old blog site. Slight changes have been made.