Eyeless in Gaza

"Did you hear the one about the computer industry's progress?" one programmer says to another.

"No, how does it go?"

"If the car industry was anything like the computer industry," the first programmer says, "a Rolls-Royce would cost a penny, would get a million kilometers per gallon,"

"---and would crash once a week, killing all passengers," the other programmer finishes.

A computer program is a recipe. It tells us, or a machine, how to do something---bake a cake, do arithmetic, run a nuclear power plant---in a sequence of simple steps. Each step must be simple and clear, otherwise today's machines can't follow them. But each step may depend on the outcome of many other steps. And as there's no real limit to the number of possible steps, programs can be quite complicated.

Further, while we currently have a pretty good grip on the complexity of today's computer hardware, tomorrow's hardware may grow to be as complex as today's software. In fact, the two are already merging because the major cost in a new piece of computer hardware is no longer the raw materials or energy needed to build it but the imagination and knowledge of those who design it. Of course, by the time computer hardware is as complex as the most complex software today, software will be even more complex---because it's so much cheaper to make. Whether that software will be correct or not is another matter.

Today our biggest problems are too complex, diverse, and poorly understood for us to have any long-term hope of creating handmade computer solutions, as we did in the past. As our problems become even more complex and ill-defined in the years ahead, we'll fall ever further behind in our understanding and mastery of them. There seems to be nothing we can do about this situation, because the problem doesn't really lie in computers at all, it lies in complexity.

The Hubble space telescope launched in 1990, for example, took over ten thousand highly trained astronomers, bureaucrats, and computer and aerospace engineers two decades to design, develop, and deploy it. It cost 2,500 million dollars, not counting its 270-million-dollar yearly operating costs. Nor does that count the 450 million dollars for each shuttle flight to maintain it. Yet, after all that time and all that money, after all those people and all that skill, Hubble was continually plagued with problems, the worst one being a severe flaw in its main mirror. The more complex the system, the harder it is to get it right.

Another highly complex system is the space shuttle's ground-based software controller, which is about twenty-six million lines long. It took over twenty-two thousand programmer-years to develop at a cost of over 1,200 million dollars. It's a fine piece of work. But even after all that time and effort and money, we can't be sure it will handle all the situations the shuttle could encounter.

In 1986, the U.S. National Aeronautics and Space Administration was running a simulated reentry practice session of the space shuttle. Earlier that year the shuttle Challenger had exploded in flight, and NASA was under extreme pressure to upgrade its safety record. But there was a major computer failure during the simulation, and the controlling computer started sending flawed altitude data to the shuttle orbiter. The crew in the simulator, who thought they were in space, were actually already in the earth's atmosphere. The commands they gave sent the ship tumbling out of control. Had they been on a real flight, they would have died. Mistakes are inevitable.

Under the Bludgeonings of Chance