Ganssle 1 MAPLD 2005/S110 Learning from Jack Ganssle Disaster

Ganssle 2 MAPLD 2005/S110 The Tacoma Narrows Bridge 4 months after opening, Nov 7, 1940

Ganssle 3 MAPLD 2005/S110 Forgotten Failures Montrose Bridge, Scotland 1838 Menai Strait Bridge, Wales, 1839 Basse-Chaine Bridge, 1850 Roche-Bernard Bridge, France, 1852 Wheeling Suspension Bridge, 1854 Dryburgh Abbey Bridge, Scotland, 1818 Niagara-Lewiston Bridge, 1864 Niagara-Clifton Bridge, 1889 Bronx-Whitestone, 1939 Deer Isle Bridge, 1939

Ganssle 4 MAPLD 2005/S110 Costs GeorgeGolden Bronx- Tacoma Washington Gate Whitestone Narrows Completed Span 3500 ft 4200 ft 2300 ft 2800 ft Cost $59.5m $35m $19.7m $6.4m

Ganssle 5 MAPLD 2005/S110 Lessons Cheaper is often more expensive Management decisions do not repeal the laws of physics Not learning from the past means repeating the past – endlessly Codes are a powerful way to insure projects are done correctly

Ganssle 6 MAPLD 2005/S110 Clementine Lessons learned: Schedules can’t rule Tired people make mistakes Error handlers save systems Never sacrifice testing

Ganssle 7 MAPLD 2005/S110 NEAR Lessons Learned: Tired people make mistakes. Use the VCS Test everything! Engineers rock! We must learn from disaster

Ganssle 8 MAPLD 2005/S110 Mars Polar Lander/Deep Space 2 Lessons learned: Tired people make mistakes Test everything! Test like you fly; fly what you test

Ganssle 9 MAPLD 2005/S110 Pathfinder Error handlers save systems Lessons learned: There’s no such thing as a glitch – believe your tests!

Ganssle 10 MAPLD 2005/S110 Mars Exploration Rover Lessons learned: Test like you fly; fly what you test We must learn from disaster Poor error handler

Ganssle 11 MAPLD 2005/S110 Titan IVb Centaur Lessons Learned: Test like you fly; fly what you test Use the VCS

Ganssle 12 MAPLD 2005/S110 Ariane 5 Lessons Learned: Improve error handling Assume software can fail Test everything! Be careful with ported code

Ganssle 13 MAPLD 2005/S110 Chinook Lessons Learned: Do reviews… before shipping! Test like you fly; fly what you test

Ganssle 14 MAPLD 2005/S110 Therac 25 Lessons Learned: Use tested components Use accepted practices Use peer reviews

Ganssle 15 MAPLD 2005/S110 Radiation Deaths in Panama May ‘01: Over 20 dead patients Possible to enter data in such a way to confuse machine; unit prints a safe treatment plan but overexposes. Lessons Learned: Test carefully Better Requirements Use a defined process & peer reviews

Ganssle 16 MAPLD 2005/S110 Pacemakers Lessons Learned: Test everything! Flash is not a schedule enhancer

Ganssle 17 MAPLD 2005/S110 Near Meltdown Lessons Learned: Test everything! Improve error handling

Ganssle 18 MAPLD 2005/S110 Lessons Learned: Be careful with ported code Blame the engineers Uwatec dive computer (1995) The Challenger

Ganssle 19 MAPLD 2005/S110 A Hot Day Lessons Learned: Test everything!

Ganssle 20 MAPLD 2005/S110 Lessons Learned: Choose your IP carefully

Ganssle 21 MAPLD 2005/S110 Forgotten Failures Ford Explorer recall Grand Prix leap-year glitch 1992 – Crash of only F-22 prototype 2003 – BMW traps Thai politician 2003 – BMW recalls is 2000 – Ford Explorer recall 747, 767, A340 avionics lockups 2003 – Slammer worm attacks nuke 1974 – Loss of a job for 7 years 1991 – Patriot missile failure

Ganssle 22 MAPLD 2005/S110 Our Criminal Behavior No Peer Reviews Implicated in the Chinook helicopter, Multidata Radiotherapy device, Therac 25. Average uninspected code contains bugs per 1000 LOC. Inspections find most of these. Cheaply.

Ganssle 23 MAPLD 2005/S110 Our Criminal Behavior Inadequate testing Implicated in the Clementine, NEAR, Mars Polar Lander, Pathfinder, Mars Expedition Rover, Titan IVb, Ariane, Sea Launch, Chinook, Therac 25, Multidata, pacemakers, Los Alamos incident, huge digital thermometer. Implicated in the NEAR, Pathfinder, Titan IVb, EFF, and FAA incidents. Ignoring or cheating the VCS

Ganssle 24 MAPLD 2005/S110 Our Criminal Behavior Lousy error handlers Implicated in the Ariane, Los Alamos incident, Clementine, Yorktown, Mars Expedition Rover, and many others This means adopting a culture of anticipating and planning for failures! And for FPGA users it means adopting a philosophy that things do fail!

Ganssle 25 MAPLD 2005/S110 Our Criminal Behavior The use of dangerous tools! C (worst)500 bugs/KLOC C (average) ADA (worst) 50 ADA (average) 25 SPARK (average) 4

Ganssle 26 MAPLD 2005/S110 The Boss’s Criminal Behavior Corollary: Tired people make mistakes Implicated in the Clementine, NEAR, Mars Polar Lander and many others Schedules can’t rule:

Ganssle 27 MAPLD 2005/S110 The Boss’s Criminal Behavior Be wary of financial shortcuts! Implicated in the Takoma Narrows Bridge, Ariane, MGM fire, and many others Reuse is extremely difficult. See “Confessions of a Used Program Salesman” by Will Tracz Implicated in the Ariane, Uwatec and many others. Reuse is not a panacea

Ganssle 28 MAPLD 2005/S110 Are we criminals? Or are we still in the dark ages? But there’s a lot we do know, so we’re negligent – and will be culpable – if we don’t consistently use best practices.