Issue: January/February 2002 | PDF

Closing the Gap

Years ago Fred Brooks commented years that “The gap between the best software engineering practice and the average practice is very wide—perhaps wider than in any other engineering discipline.” The past few years have seen a proliferation of books on project management, requirements, architecture, design, testing—nearly every area of software engineering. But within the companies I visit in my consulting business, I rarely see software engineering best practices being used. A question I’ve been asking myself with increasing frequency is, “Why aren’t people using the numerous good software engineering practices that are now so readily available?”

Classic Barriers to Innovation

A conventional answer to why these practices aren’t being used is that many of them simply aren’t yet mature. When presented with a new practice, software practitioners tend to ask tough questions like these:¹

• Do experimental results prove conclusively that the practice will work in the field?
• Are successes a result of the practice itself, or might they be the result of the people using it?
• Is the practice complete, or does it need to be adapted or extended before it can be applied?
• Does the practice have significant overhead (training, documentation) that offsets its value in the long run?
• If the practice was developed in a research setting, does it apply to real-world problems?
• Does the practice generally slow down the programmers?
• Can the practice be misapplied?
• Is information available about the risks involved with using the practice?
• Does the practice include information about how to integrate it with existing practices?
• Must the practice be applied in its entirety to realize significant benefits?

These are all fair questions, and I think it’s healthy for practitioners to ask them. Indeed, part of IEEE Software’s mission is to help our readers answer these questions. However, the practices I’m thinking of are hardly new, and, for many of them, I believe many of these questions have already been answered. Table 1 lists numerous practices that have been well-understood and deployed by leading organizations for decades.

In the management arena, we’ve had automated estimation tools since the early 1970s, but most projects don’t use them. Measurement has been a key topic for 25 years, but few organizations collect quantitative data on their projects. I still see software developers housed in open work bays or cubicles far more often than I see them working in private or semi-private offices—even though research about the effect of the physical environment on productivity has been conclusive for more than 15 years.

One of the most fundamental practices in software engineering is change control, especially as it relates to software requirements. I teach a two-day workshop based on my book Software Project Survival Guide (Microsoft Press 1998). When I originally developed the workshop, I included a module on change control because I could easily pull together the necessary materials, and I was working under some deadline pressure. I assumed that it would be too basic for most of my students and that I would need to replace that module as soon as I had time. To my surprise, three years later, after teaching the class about 20 times, I’ve had only one group of students in which more than 50% were already using change control. Change control has been described in the software engineering literature since 1978, however the basic practice has been employed in other branches of engineering and must be at least 50 years old. All of the tough questions described above were answered for change control decades ago. Considering its central role in software project control, I am puzzled about why this fundamental practice is not found on software projects universally.

Barriers to Software Innovations

Software presents some unique challenges to the adoption of better practices. One challenge is a lack of awareness that good practices exist. Where, ideally, should someone learn about fundamental software engineering practices? In most fields, we expect universities to provide education in the fundamentals of a field. Until very recently, however, most undergraduate degree programs related to computer programming have not including training in these basic practices. Software engineering is still a young field, additional university programs are coming online each year, and I think the lack of infrastructure is due simply to software engineering’s youth.

In the absence of university educational infrastructure, a person might expect software-producing companies themselves to provide training to supplement the university education system. In fact, a few leading companies do provide training to their software engineers, but not to an extent great enough to ameliorate industry wide software problems.

In less advanced companies, the lack of training has been more difficult to address. Before a manager can prescribe training, he needs to know that a field of knowledge is deep enough to need training. Managers who came up through the technical ranks 20 years ago, or even 10 years ago, may underestimate the depth of knowledge in modern software engineering. Many software managers are not themselves well trained enough to realize that their staff needs training.

Calling All Experts

These are all descriptions of what has not been done, but they still leave open a basic question: Why don’t software engineers—who are some of the brighter people on the planet—seek out better methods of doing their work? We’re all aware of the pain arising from not using these practices. So why don’t practitioners more actively seek them out and use them?

With all the advances during the past several years, it appears that the challenge for the software industry has shifted from good-practice development to good-practice deployment.

*What do you see as the barriers to deployment of good practices? How do you think good practices can be deployed more quickly? I invite your comments.

Table 1. Software best practices that are rarely used.

References

1. Raghavan, Sridhar A. and Donald R. Chand, 1989. “Diffusing Software-Engineering Methods,” IEEE Software, July 1989, pp. 81-90.

Editor: Steve McConnell, Construx Software  |  More articles