Issue: January/February 1999 | PDF
After the Gold Rush
In January 1848, James Marshall discovered gold in California’s American River, near a mill he was building for John Sutter. At first, he and Sutter dismissed the pea-sized nuggets as a nuisance; they believed the attention that gold would bring would spoil Sutter’s plans to build an agricultural empire. But within months, word spread, and by 1849 thousands of men and a handful of women from the United States and around the world (“49ers”) headed to California to make their fortunes in what became known as The California Gold Rush. The unprecedented rush west that followed would create something new to America: an economy driven by high-risk entrepreneurialism, fueled by dreams of striking it rich. Precious few 49ers actually realized that dream during the gold rush days, but the dream lives on in many modern software companies and individual software developers—with about the same chances of striking it rich and busting out as miners in California had in 1849.
The California gold rush was unique because the gold was found in riverbeds instead of embedded in hard rock. That meant that, at first, anyone with a tin pan and an entrepreneurial spirit had a chance to make a fortune. But by mid 1849, most of the easy gold had been found, which meant that a typical miner spent 10 hours a day in ice cold water, digging, sifting, and washing. As time passed, this backbreaking work yielded less and less gold. There were the occasional lucky strikes well into the 1850s, which provided just enough good news to encourage thousands to continue digging. Most failed every day, but they kept on—year after year.
After the early days of the gold rush, miners had to use more advanced techniques to extract gold. By the early 1850s, a single miner could no longer work his claim alone. He needed help, and he needed technology. At first, miners banded together informally to build dams, reroute rivers, and expose the gold. But soon even more capital-intensive techniques were needed to be successful, and the informal groups of miners were replaced by corporations. By the mid 1850s, most of the miners who remained were corporate employees rather than individual entrepreneurs.
Software Gold Rushes
The advent of a major new technology often means the beginning of what I think of as a “software gold rush.” I’ve personally seen gold rushes with the advent of the IBM PC and Microsoft DOS operating system, the migration from DOS to Windows, and the growth of Internet computing.
Gold rush software development is characterized by high-risk, high-reward development practices. Because few companies have established competitive presences in the marketplace, much of the technological gold seems to be just lying on the ground, waiting for anyone with the right mix of innovation and initiative to pick it up. Software 49ers rush into the new technology, hoping to strike it rich. The stereotypical technology goldrushers are two guys working in a garage who ultimately make a fortune—dynamic duos such as Bill Gates and Paul Allen (Microsoft), Steve Jobs and Steve Wozniak (Apple Computer), Bob Frankston and Dan Bricklin (VisiCalc), and so on.
Gold rush software development is a high risk activity. The practices employed during a software gold rush are usually associated with hacking rather than engineering: small team sizes, informal processes, long hours, little documentation, bare-bones quality assurance practices—practices I refer to collectively as “commitment-based development” (Rapid Development, Microsoft Press 1996). Use of these practices puts all but the smallest software projects at high risk of failure.
The odds of striking it rich during a software gold rush are about as good as they were during the California gold rush—for every success story, there are hundreds or even thousands of projects that bust out. But the failures aren’t nearly as interesting as the successes, and so we don’t hear very much about them. As with the California gold rush, projects run with commitment-based development are successful just often enough, and are so enormously lucrative when they do succeed, that they convince software developers that such high-risk practices can work, thus keeping the entrepreneurial dream alive.
After the Gold Rush
Post–gold rush software development is characterized by more methodical, lower-risk, capital-intensive development practices. Projects use relatively large teams, rely on more formal processes, adhere to more standards (compatibility with legacy code, industry-wide protocols, and so on), and work with much larger code bases. The emphasis is less on rushing software to market quickly and more on reliability, interoperability, usability, and other “esoteric” product characteristics that hardly matter during a gold rush but that matter a lot when a technology matures.
Companies most successful during one gold rush are likely to fail during the next gold rush.The archetypal post–gold rushers are the companies that became established during an earlier gold rush. These companies repeat Marshall and Sutter’s mistake of seeing new-technology gold as a nuisance that will interfere with their well-laid plans for extracting maximum value from the claims they staked during the last gold rush. Examples of companies that were slow to pick up new-technology gold nuggets include IBM during the early days of PC-DOS; Lotus during the early days of Windows; and Microsoft during the early days of the Internet. We’ll undoubtedly see this pattern repeated during the late days of the Internet by some of the companies that had the greatest successes in the early days—Netscape, Yahoo, Amazon.com—only time will tell which will successfully make the next great transition.
Gold rush style development practices have even lower odds of working in a post–gold rush phase. In the early days of a new technology, there are few established players or products. The technological barriers to entry are low, and early products can be small and still succeed. The first version of Microsoft Word for Windows consisted of just 249,000 lines of code. As with the California gold rush, fewer people and less capital are needed to stake a claim during the early days of a new technology. Two guys in a garage have a chance to compete against the major corporations when a successful product can be built with 249,000 lines of code. As the technology matures, however, the easy gold runs out, and successful companies have to compete on the basis of more capital intensive projects. The current version of Word for Windows, for example, consists of more than 5 million lines of code. One of the most damaging mistakes that successful gold rush companies make is to persist in using gold rush development approaches as the technology matures and their projects scale up. To compete successfully in the post–gold rush phase, the successful project needs to do a lot more than simply multiply the number of guys and get a bigger garage.
Gold rush economics are sometimes more sensible than they appear. It may be as hard for an established company to compete in a gold rush phase as it is for two guys in a garage to compete post–gold rush. During a gold rush, having thousands of individual software developers take on entrepreneurial risk voluntarily—with one in a thousand striking it rich and the rest chalking their losses up to experience—is tremendously beneficial from a macroeconomic point of view. No one but the individual entrepreneurs pays for the failures, and everyone has a chance to benefit by buying and using the products that succeed. But how can an individual company harness this dynamic? What company could possibly afford to fund thousands of individual entrepreneurs during a gold rush phase just to find the one or two that successfully develop new gold rush technology? The answer is, no company can afford to do that, which is one reason that software company acquisitions during a gold rush phase are more sensible than they might at first appear. Some industry observers thought Microsoft was crazy to pay $130 million to acquire Vermeer Technology, original creators of FrontPage, when it had only about $10 million in annual revenue. But from the entrepreneurial gold rush point of view, paying $130 million for the one success in a thousand is a cheap alternative to funding thousands of entrepreneurial experiments internally, nearly all of which would ultimately be dead ends.
California or Bust, Software Engineering Style
Gold rush software projects might be inherently risky, but the use of haphazard software development practices has made them riskier than they need to be. Developers working on gold rush projects have been saddled for decades with the methodological equivalents of tin pans and shovels. Historically, software engineering has focused its attention almost exclusively on post–gold rush projects. This needs to change. Gold rush software projects are essential to the forward march of technology and vital to the economy. More important, these are the projects that have the power to capture the imaginations of leading software practitioners. Modern software engineering needs to rise to the challenge of discovering and refining practices that can make more of these projects successful.
Editor: Steve McConnell, Construx Software | More articles
