Issue: July/August 2002 | PDF

The Business of Software Improvement

The return on investment in improved software practices is well documented. In 1994, James Herbsleb reported that the average “business value” (roughly the same as ROI) for 13 organizations that took on CMM-SW-based improvement programs was about 5 to 1, with the best organizations realizing returns of 9 to 1 [1]. In 1995, Neil C. Olsen reported similar returns for organizations that made significant investments in staffing, training, and work environments [2]. In 2000, Capers Jones reported that the ROI from process improvement could easily go into double digits (meaning returns greater than 10 to 1) [3]. A recent analysis by Watts Humphrey found that the ROI for improved software practices could be in the neighborhood of 4 to 1 [4].

Indirect Benefits Are Even More Significant

The ROI figures in the published literature are based on operational savingsthat is, on reducing development cost per line of code written or per function point delivered. Although these savings are impressive, the greater business benefit might arise from the significant indirect returns that arise from improved software practices. Better software practices improve predictability of costs and schedules, reduce risk of cost and schedule overruns, provide early warning of problems, and support better management.

Many organizations that have focused on improving their software practices have reported improvements in predictability similar to the results in Figure 1 [5]. For a software products company, what is the business value of improving schedule estimation accuracy from plus or minus 100 percent to plus or minus 10 percent? What is the value of being able to make a commitment to customers six to 12 months in advance of a scheduled completion date, with high confidence of delivering on that commitment?

[missing image]

Figure 1. Project performance compared to estimated performance [5]. This example demonstrates different projects in the US Air Force.

For a company that develops custom software, what is the business value of being able to provide a fixed price bid with high confidence that the project will not significantly overrun the bid?

For a retail sales organization, what is the value of being able to plan cutover to a new system with pinpoint accuracy? What is the value of knowing with confidence that cutover will occur 1 October, as planned, with little risk of overrunning to 15 November or 1 December?

Unlike the operational benefits that most of the industry literature has focused on, these indirect benefits open the door to additional revenue opportunities. These benefits are based not on reducing costs, but on increasing access to additional business. For top decision makers in organizations, these indirect benefits are often more compelling than the direct, operational benefits.

Organizational Challenge

Considering the strongeven compellingcase for improving software practices, it might seem surprising that some organizations have not made a commitment to use best practices. I have recently been thinking a lot about why best practices aren’t used. Several factors seem to be in play.

First, there is a basic technology transfer issue. Many software development best practices have been available for decades, but only a few companies use them, and undergraduate programs have not generally taught these best practices. The scarcity of experienced users of these practices limits the rate at which current users can train new users. Although a person might reasonably assume that the average software organization’s capability is halfway between the worst organization’s capability and the best’s, in reality, the average software organization’s practices are much closer to the worst organization’s practices than the best’s. The result is that software developers who work in average organizationswhich includes most developershave never seen a really well-run software project, much less a really well-run software organization. The software industry faces the problem of bootstrapping best practices into common usage because of limited current usage of them.

A second factor is that recent economic circumstances have prevented software organizations from feeling any strong imperative to switch to better practices [6]. Throughout the 1990s, software-related companies rode a technology wave that rewarded companies just for being in the software industry. Companies didn’t need to focus on operational improvements because that would have shifted too much focus away from generating revenue. For a time, improved software practices seemed to be more of a distraction than a help.

A final factor is that many organizations push responsibility for software development improvement down to the project level. In reviewing the “effort multiplier” factors in the Cocomo II estimation model [7], I was struck by how few of the factors are under the control of an individual project manager. Of the 22 factors Cocomo II uses to fine-tune a project’s base effort estimate, in my judgment only three are typically under the individual project manager’s control: documentation, architecture and risk resolution, and development for reuse. Numerous factors are dictated by the nature of the company’s businessproduct complexity, required reliability, platform volatility, unprecedentedness of the software, and so on. A company cannot easily change these factors without changing businesses. The remaining factorsstaff capability, multisite development, personnel continuity, process maturity, and so oncan be influenced by the organization but not by individual projects.

What Can You do?

We could hope that upper management, sales, and marketing staff would read every issue of IEEE Software cover to cover or educate themselves about the finer nuances of software development some other way. But this isn’t likely to happen, so leading software practitioners have an ongoing responsibility: the education of nontechnical software project stakeholders. Software practitioners sometimes perceive upper management and other nontechnical staff to be blocking the use of better practices. We complain that they fail to support better practices or even undermine them. I’ve generally found, however, that upper management, sales, marketing, product support, and other personnel are receptive to improved software practices when I take the time to explain those practices to them. Indeed, they are acutely aware of the problems caused by current practices and are eager to hear how they can help improve software projects.

What have you done to educate executives about better software practices? What has worked well for you? I’d love to hear your comments at stevemcc@construx.com.

References

  1. J. Herbsleb, et al., Benefits of CMM Based Software Process Improvement: Initial Results, tech. report CMU/SEI-94-TR-13, Software Eng. Inst., Carnegie Mellon Univ., Pittsburgh,1994.
  2. N.C. Olsen, “Survival of the Fastest: Improving Service Velocity,”IEEE Software, vol. 12, no. 5,; Sept./Oct. 1995,, pp. 28-38.
  3. C. Jones,Software Assessments, Benchmarks, and Best Practices, Addison Wesley,Reading , Mass. ,2000.
  4. W. Humphrey,Winning with Software: An Executive Strategy, Addison Wesley,Reading , Mass.,2001.
  5. P.K. Lawlis , R.M. Flowe and J.B. Thordahl , “A Correlational Study of the CMM and Software Development Performance,”Crosstalk,Sept. 1995.
  6. S. McConnell,After the Gold Rush, Microsoft Press,Redmond , Wash. ,1999.
  7. B. Boehm, et al.,Software Cost Estimation with Cocomo II, Addison Wesley,Reading , Mass.,2000.

Editor: Steve McConnell, Construx Software  |  More articles