In order to provide some insight as to the critical nature of safety problems, we have included a list of examples that have occurred within the last few years (see List 1). We note that most of these systems were designed with safety issues very much in mind, and that despite the precautions used in these designs and implementations, errors occurred. Many of these errors were caused by a seemingly trivial mistake with extreme ramifications. As examples of software safety problems, we have extracted some interesting cases drawn largely from back issues of ACM SIGSOFT Software Engineering Notes. References are cited as (SEN vol no), where vol 10 = 1985. Some incidents are well documented, while others need investigation.
List 1 Legend ! = Loss of Life; * = Potentially Life-Critical; $ = Loss of Money/Equipment
! Arthritis-therapy microwaves set pacemaker to 214, killed patient(SEN 5 1) $* SAC/NORAD: 50 false alerts in 1979 (SEN 5 3), including a simulated attack whose outputs accidentally triggered a live scramble [9 Nov 1979] (SEN 5 3); $* WWMCCS false alarms triggered scrambles [3-6 Jun 1980] (SEN 5 3) * NORAD alert from BMEWS detecting the moon as incoming missiles (SEN 8 3) * 767 (UA 310 to Denver) four minutes without engines [August 1983] (SEN 8 5) * F18 missile thrust while clamped, plane lost 20,000 feet (SEN 8 5) * Mercury astronauts forced into manual reentry (SEN 8 3) * Cosmic rays cut Challenger comm in half for 14 hours [8 Oct 84] (SEN 10 1) *$ Frigate George Philip fired missile in opposite direction (SEN 8 5) *$ Mariner 1: Atlas booster launch failure DO 100 I=1.10 (not 1,10) (SEN 8 5) *$ Mariner 18: aborted due to missing NOT in program (SEN 5 2) *$ F18: plane crashed due to missing exception condition, pilot OK (SEN 6 2) *$ F14 lost to uncontrollable spin, traced to tactical software (SEN 9 5) *$ El Dorado: brake computer bug caused recall of all El Dorados (SEN 4 4) *$ First Space Shuttle backup launch-computer synch problem (SEN 6 5) * Second Space Shuttle operational simulation: tight loop upon cancellation
of an attempted abort; required manual override (SEN 7 1) * Second Shuttle simulation: bug found in jettisoning an SRB (SEN 8 3) * Gemini V 100mi landing err, prog ignored solar orbital motion (SEN 9 1) * UA 310 (767) four minutes without engines (SEN 8 5) * F16 simulation: plane flipped over whenever it crossed equator (SEN 5 2) * F16 simulation: upside-down deadlock over left vs. right roll (SEN 9 5) * Nuclear reactor design: bug in Shock II model/program (SEN 4 2) * Reactor overheating, low-oil indicator; two-fault coincidence (SEN 8 5) * SF BART train doors sometimes open on long legs between stations (SEN 8 5) * Santa Clara prison data system (inmate altered release date) (SEN 10 1) * FAA Air Traffic Control: many computer system outages (e.g., SEN 5 3) ! Korean Airlines 007 shot down [1 Sept 1983], killing 269; course off 10
degrees due to ten's digit of flight heading being miskeyed? (SEN 9 1) ! Air New Zealand crashed into mountain [28 Nov 1979]; computer course data
error detected and fixed, pilots not informed (SEN 6 3 & 6 5) *$ South Pacific Airlines, 200 aboard, 500 mi off course near USSR