Issue: March/April 1999 | PDF
Software Engineering Principles
Many software practitioners think of software engineering knowledge almost exclusively as knowledge of specific technologies: Java, Perl, html, C++, Linux, Windows NT, and so on. Knowledge of specific technology details is necessary to perform computer programming. If someone assigns you to write a program in C++, you have to know something about C++ to get your program to work.
You often hear people say that software development knowledge has a 3 year half-life: half of what you need to know today will be obsolete within 3 years. In the domain of technology-related knowledge, that’s probably about right. But there is another kind of software development knowledge—a kind that I think of as “software engineering principles”—that does not have a three year half-life. These software engineering principles are likely to serve a professional programmer throughout his or her career.
Essence and Accident
In 1987, Fred Brooks published an influential paper, “No Silver Bullets—Essence and Accident in Software Engineering” (Computer, April 1987). The main contention of Brooks’ paper was that no single tool or methodology—no “silver bullet”— portended a 10-to-1 improvement in productivity over the next decade. The reasoning behind that claim is highly relevant to the distinction between technology knowledge and software engineering principles.
In using the words “essence” and “accident,” Brooks drew on an ancient philosophical tradition of distinguishing between “essential” and “accidental” properties. Essential properties are those properties that a thing must have to be that thing: A car must have an engine, wheels, and a transmission in order to be a car. These are essential properties. A car might or might not have a V8 or a V6, studded snow tires or racing slicks, an automatic or a manual transmission. These are “accidental” properties, the properties that arise by happenstance and do not affect the basic car-ness of the car.
According to Brooks, the most difficult work of software development is not representing the concepts faithfully in a specific computer programming language (coding) or checking the fidelity of that representation (testing)—that is the accidental part of software engineering.
Brooks argued that the essence of software engineering consists of working out the specification, design, and verification of a highly precise and richly detailed set of interlocking concepts. What makes software development difficult is its essential complexity, conformity, changeability, and invisibility. Computer programs are complex by their nature. Even if you could invent a programming language that operated at the level of the problem domain, programming would still be a complicated activity because you would still need to define relationships between real-world entities precisely, identify exception cases, anticipate all possible state transitions, and so on. Strip away the accidental work involved in representing these factors in some specific programming language within some specific operating system, and you still have the essential difficulty of developing and debugging the underlying real-world concepts.
Adding to its inherent complexity, additional complexity arises from the fact that software cannot be created in isolation, but must conform to real-world constraints—pre-existing hardware, third party components, government regulations, legacy data formats, and so on. The software designer often faces inflexible, external factors that limit the extent to which complexity can be reduced.
Another source of complexity is software’s changeability. The more successful a program is, the more uses people will find for it, and the more it will be adapted beyond the domain for which it was originally intended.
A final source of software complexity arises from software’s inherent invisibility. Software doesn’t exist in a way that can be represented using geometric models. Attempts to represent even simple, static logic flow geometrically quickly become absurdly complicated, as anyone who has ever tried to draw a full flow chart for even a relatively simple program will attest. Adding to the problem, software doesn’t exist meaningfully in static form; it only exists meaningfully when it’s executed. So even the absurdly complicated geometrical representations turn out to present software’s structure as simpler as it really is because they ignore the time dimension.
Brooks argued that the major gains to be realized from addressing the accidental elements of software engineering have already been made: the invention of high level languages, movement to interactive computing from batch processing, and development of powerful integrated environments. Any further order-of-magnitude improvements can be made only by addressing software’s essential difficulties—the complexity, conformity, changeability, and invisibility inherent to software development.
Software Engineering Principles
Knowledge that addresses what Brooks calls the essential difficulty of software engineering is what I think of as “software engineering principles.” During the past 30 years, since the first NATO conference on software engineering in 1968, the software industry has come a long ways in identifying the essential knowledge that a software engineer needs in order to develop software effectively.
One way to look at the state of this body of knowledge is to think of it in terms of the total body of knowledge needed to create a complex software system and the proportion of that body of knowledge that is relatively stable—the part that is not going to be obsolete within three years. As the figure below illustrates, at the time of the 1968 NATO conference, that stable core was relatively small.
How small was this body of knowledge at that time? Consider that the first fully correct binary search algorithm was published just six years before the NATO conference. Böhm and Jacopini published the paper that made elimination of the goto and structured programming possible only 2 years before the conference. Edsger Dijkstra wrote his famous “GoTo Statement Considered Harmful” letter to the editor the same year as the conference. Larry Constantine, Glenford Myers, and Wayne Stevens didn’t publish the first paper on structured design until six years after the conference. Tom Gilb published the first book on software metrics nine years after the conference. Tom DeMarco’s landmark book on structured analysis didn’t appear until 1979. Anyone who tried to identify a stable core of knowledge in 1968 would have had their work cut out for them. It’s hard to find much knowledge that would have been considered part of a “stable core” of software engineering knowledge in 1968 that is still in use today—perhaps only about 10 percent.
In contrast, I think that the same diagram drawn today would look more like this:
We still do not have perfect knowledge of what is needed to develop software effectively, so the stable core now occupies only perhaps 75 percent of the knowledge needed to develop a software system. But we probably don’t ever want a core that makes up 100 percent of the body of knowledge—that would indicate that software engineering had reached a steady state and was immune to future improvements.
Considered from the point of view of the body of knowledge of software engineering, I think we’ve made significant progress in the areas Brooks refers to as the “essential difficulties” of software development. We now have reference books that are adequate or good on the topics of requirements development, design, construction, testing, reviews, quality assurance, software project management, algorithms, and user interface design, just to name a few topics, and new and better books that codify the knowledge needed to be a software engineer are appearing regularly. An investment in learning software engineering principles is a particular good investment for a software professional to make because that knowledge will last a whole career—not be half obsolete within three years.
As a software development professional, you need knowledge of specific technologies to do your job. But you need knowledge of software engineering principles to do your job well. A continuing pursuit of such knowledge is one mark of a true professional.
Editor: Steve McConnell, Construx Software | More articles
