Issue: May/June 2000 | PDF

Sitting on the Suitcase

Suppose you’re preparing for a trip and deciding which suitcase to take. You have a small suitcase that you like because it’s easy to carry and will fit into an airplane’s overhead storage bin. You also have a large suitcase, which you do not prefer because you’ll have to check it in, lengthening your trip. You lay your clothes beside the suitcase, and it appears that they’ll almost fit into the small suitcase. What do you do? You might try packing them very carefully, not wasting any space, and hoping they all fit. If that approach doesn’t work, you might try stuffing them into the suitcase with brute force, sitting on the top and trying to squeeze the latches closed. If that still doesn’t work, you’re faced with a difficult choice: leave a few clothes at home or take the larger suitcase.

Software projects face a similar dilemma. Project planners often find a gap between a project’s business targets and its estimated schedule and cost. If the gap is small, the planner might be able to control the project to a successful conclusion by preparing extra carefully or by squeezing the project’s schedule, budget, or feature set. If the gap is large, the project’s targets must be reconsidered.

Software Estimation

Some industry experts imply that the goal of estimation is to achieve pinpoint accuracy. They claim that effort estimates created using automated estimation tools can be accurate to within about 10% (see, for instance, Capers Jones’s Assessment and Control of Software Risks, Yourdon Press, 1994). Meanwhile, various reports about software industry practices suggest that real-world estimation accuracy falls far short of this ideal. Effort estimates that are accurate to within 50% are found in less than half of all projects (see The Standish Group’s 1994 report “Charting the Seas of Information Technology”).

Estimation accuracy is probably worse than it at first appears. The Standish Group’s 1994 “Chaos Report” found that “challenged projects” (those that experienced schedule and budget overruns) routinely threw out significant amounts of functionality in order to deliver the schedules and budgets they eventually did. Of course, their estimates weren’t for the abbreviated versions they eventually delivered; they were for the originally specified, full-featured versions. If these late projects had delivered all of their originally specified functionality, they would have overrun their plans even more.

These cost and schedule overruns are partly attributable to software developer optimism (see, for example, Michiel van Genuchten, “Why Is Software Late? An Empirical Study of Reasons for Delay in Software Development,” IEEE Transactions on Software Engineering, Vol. 17, No. 6, June 1991, pp. 582–590). They are partly attributable to the use of inefficient practices that fall short of expectations. And they are partly attributable to unrealistic target setting—targets are established, the targets become commitments, and the commitments are later reported as “estimates.”

Target Setting and Control

The purpose of a software estimate is not to make a perfect prediction about a project’s eventual cost or schedule. Changes occur throughout a project that invalidate many of the assumptions that went into early estimates, and even if we eventually achieve results within 10% of our original assessment, the results are usually a fluke. The project we end up with is rarely the same project we originally estimated, so the “accurate” estimate was really for some other project.

I propose a different goal for software estimation: An estimate should achieve accuracy sufficient to let the project manager control the project to meet its business targets. As Tom Gilb points out in Principles of Software Engineering Management (Addison-Wesley, 1988), estimates by themselves merely give us the ability to predict project outcomes, but we don’t need just prediction; we need control.

Estimation on software projects interplays with both target setting and control. Businesses have many important reasons to establish targets independent of software estimates, just as you might have important reasons to prefer a small suitcase to a large one. Sometimes software must be ready for a trade show, or for a holiday sales season, or for tax preparation season, or for some other externally imposed date. Sometimes a business is operating under tight cost constraints and will be penalized for exceeding cost targets. The business environment dictates these targets, and a company has little ability to influence them. What a business can influence are the parameters of its software projects. In this context, a pinpoint-accurate estimate that “we will deliver our software four weeks after the end of the holiday sales season” is of little use. The value to the business arises from being able to control the project enough to deliver desirable functionality within the business timeframe and cost desired.

Thus, the primary purpose of software estimation is not to predict a project’s outcome; it is to determine whether a project’s targets are realistic enough to allow the project to be controlled to meet its targets. Will the clothes I want to take on my trip fit into the small suitcase or will I be forced to take the large one? Can I take the small suitcase if I make minor adjustments? Business managers want the same kinds of answers. They often don’t want an accurate estimate that tells them that the desired clothes won’t fit into the suitcase; they want a plan for making as many of the clothes fit as possible.

Excess Baggage

Problems arise when the gap between the business targets and the schedule and effort needed to achieve those targets becomes too large. I assert (with no real proof other than my personal experience) that if the initial target and initial estimate are within about 20% of each other, the project manager will have enough maneuvering room to control the feature set, schedule, team size, and other parameters to meet the project’s business goals. If the gap between the target and what is actually needed is too wide, the manager will not be able to control the project to a successful conclusion by making minor adjustments to project parameters. No amount of careful packing or sitting on the suitcase will squeeze all my clothes into that smaller suitcase, and I’ll have to take the larger one even if it isn’t my first choice. The project targets will need to be brought into better alignment with reality before the manager can control the project to meet its targets.

Estimates don’t need to be perfectly accurate as much as they need to be useful. We should not let the pursuit of ever more accurate estimates blind us to the important but limited role that estimation plays. Control is the all-important suitcase. The business targets are the suitcase’s valuable contents. Accurate estimates just provide the wheels that make the suitcase a little bit easier to maneuver.

Editor: Steve McConnell, Construx Software  |  More articles