3. Complexity, Coupling, and Missile Defense Phil Varner Chapter 3 in Normal Accidents by Charles Perrow Software Aspects of Strategic Defense Systems by David Parnas Prediction: I will not get to missile defense today.

4. Definitions ● Four levels of a system ● Level One - part - valve ● Level Two - unit - steam generator ● Level Three - subsystem - secondary cooling system ● Level Four - system - plant ● Engineered safety devices between two and three prevent incidents from becoming accidents

5. Accidents and Incidents ● Accident - '' a failure in a subsystem, or the system as a whole, that damages one or more unit and in doing so disrupts the ongoing future output of the system'' ● Incident - ''damage that is limited to parts of a unit, whether the failure disrupts the system or not'' ● Heartless scheme - but essential – loss of life does not figure in here because focus is on parts – valve failing and killing someone - incident – Mars lander and no death - accident

6. Victims ● First - operators and ''glow boys'' ++risk ● Second - non-operating personnel or system users +risk ● Third - innocent bystanders -risk ● Fourth - fetuses and future generations --risk – ''we cannot suggest a good way to handle the issue in a safety-goal context'' - so ignore it

7. Accident types ● Component failure accidents - level 1-3 components that are linked in an anticipated sequence ● System accidents - unanticipated interaction of multiple failures ● Final accident - wing on airplane or earthquake – obvious cause (cause?) ● 3000 reports from ~70 plants – 300 accidents, 15-30 system accidents

8. Complex and Linear Interactions ● Linear – assembly lines ● Complex – multiple function – heater - heated a tank and cooled chemical reactor – if it fails, others fail too ● Rasmussen Report - PRA – main problem with systems is complexity

9. Proximity and Info ● Proximity and indirect information sources – Dauntless Colocotronis ● bad navigation info ● sliced hull at intersection of tank and pump room ● leaked into engine room ● boom, fire ● Proximity caused unexpected interaction ● Doors left open, water spreads fire, tank explosions ● Complex - not designed into the system and not anticipated

10. Complex and Linear ● Linear - 1% ● Complex - 10% ● Terminology problem – opposite of linear is non-linear – opposite of complex is simple – Kim? (as if I need to defer)

11. Complex Characteristics ● Proximity of parts or units that are not in a production sequence ● Many common mode connections ● Unfamiliar or unanticipated feedback loops ● Many control parameters w/ potential interactions ● Indirect or inferential information sources ● Limited understanding of same processes

12. Complex vs. Linear ● Proximity vs. Separation ● Common-mode connections vs. Dedicated ● Interconnected subsystems vs. Segregated ● Limited substitutions vs. Easy ● Feedback loops vs. Few feedback loops ● Multiple interacting controls vs. single purpose, segregated controls ● Indirect information vs. Direct Information ● Limited understanding vs. Extensive understanding

13. Complex vs. Linear ● We have complex systems because we don't know how to do it linearly ● ''If these complex systems also have catastrophic potential, then we had better consider alternative ways of getting the product, or abandoning the product entirely.''

14. Loose and Tight Coupling ● Loose coupling - allows certain parts of the system to express themselves according to their own logic or interest ● Tight coupling - restricts expression ● Which is preferable?

15. Tight vs. Loose ● Time dependent vs. delays possible ● Invariant sequences vs. Variant ● Only one way to reach production goal vs. many ways ● Little slack vs. lots of slack ● Buffers and redundancies are designed-in, deliberate vs. inherently everywhere ● Substitutions limited vs. substitutions abundant

16. Examples ● Dam - linear, tight ● Nuclear - complex, tight ● Post Office - linear, loose ● Universities - complex, loose ● SDI/NMD - very complex, very tight

17. SDI/NMD ● Bias alert - I personally think NMD is a bad idea from not only a technical standpoint, but also from a foreign policy standpoint ● Comforting to think of it as a giant, amusing corporate subsidy instead of an actual system that will be used

18. History of SDI ● 1983 - Regan began with speech – Strategic Defense Initiative aka Star Wars – space-based weapons – battle management satellites ● 1993 - Rebranded as National Missile Defense (NMD) ● Against 1972 ABM treaty, so in 2001 we withdrew ● Bush calls for accelerated development ● ''Tests'' so far have been rigged ● Equivalent of testing a car by making sure it starts

19. Patriot Missile ● Small scale version of NMD (hits flying stuff) ● Gulf War was first test! ● Three phases – platform turns toward incoming missile – platform radar + computer steers toward missile – onboard radar + computer then guides it to impact ● Heavy tight coupling between ground radar, ground guidance, missile radar, missile guidance, etc

20. Patriot Missile Effectiveness ● US Army claimed 80% (70) in Saudi and 50% (40) in Israel (Israel was harder because of populations) ● Several independent tests concluded that the effectiveness could be less than 10%, possibly 0 ● al-Hussein missiles would break up upon reentry - Patriots would lock onto the wrong section, and usually even miss that ● ''These data clearly indicate that the interceptor impacts were the result of software errors in the patriot system." (Statement of Theodore A. Postol before the U.S. House Of Representatives Committee on Government Operations, April 7, 1992)

21. SDI Requirements ● ''I call upon the scientific community, who gave us nuclear weapons, to turn their great talents to the cause of mankind and world peace; to give us the means of rendering these nuclear weapons impotent and obsolete.'' Ronald Reagan, May 1983 ● Like finding a needle in a stack of needles, located somewhere around the world, moving at Mach 3 ● Large system composed of large systems ● Large safety system of safety/reliability systems – can be divided into levels of systems that are themselves divided into levels

22. SDI Requirements ● Rapid and reliable warning of attack – tight coupling between complex radar, satellite systems, and correlation computers – false positives are big problem - during cold war a single missile was obviously not an attack, now? ● Determination of the source of attack (same) ● Determination of the likely targets of the attack – tight coupling between trajectory computers, sensor systems ● Determination of the missile trajectories (same)

23. SDI Requirements ● Coordination interception of the missiles or warheads during, boost, midcourse, and terminal phases, including assignment of responsibility for targets to individual weapons or sensors – coordination between sensors, trajectory computers, various weapons platforms (space-based lasers, air mounted lasers, interceptor missiles, smart pebbles) – Complex, tightly coupled action

24. More SDI Requirements ● Discrimination between decoys and warheads – complex, tightly coupled coordination between sensors and targeting computers ● Detailed control of individual weapons – complex, tightly coupled coordination between sensors, targeting computers, and guidance systems of weapons – Many weapons! ● Evaluation of the effectiveness of each attempt to destroy a target – complex, tightly coupled - sensors, targeting computers

25. Why Software is Unreliable ● Written in 1985, but the problems are the same ● Most engineered products work, software usually doesn't when released ● Unlike analog systems, digital systems have too many states and combinations of states (complex) to formally analyze ● Order of magnitude more complex ● Software failure is not the same as hardware failure – software - design or implementation problems – hardware - manufacturing problems

26. Why the SDI Software System Will Be Untrustworthy ● ''extremely high confidence that the system will work correctly the first time it is called upon'' ● Must identify, track, and direct weapons toward targets whose ballistic characteristics are unknown ● Computation by network of computers connected to sensors, weapons, etc, whose existence cannot be predicted because of countermeasures ● IMPOSSIBLE TO TEST

27. Characteristics ● Short service period - no time for human intervention ● Computational resources cannot be predicted ● System will include a large number of changing subsystems, they themselves being large systems ● System will be constantly changing with new sensors, weapons - all subject to independent modification – is loose coupling possible? – is linearization possible?

28. Implications ● Assumptions must be made about enemy weapons systems (tight coupling) ● ''fail-soft'' – predicted upon past history – component failures are unlikely and independent – system has excess capacity – real-time deadlines can be missed with out long-term effects

29. Implications ● No large scale system has ever been installed without extensive testing – approximate battle conditions – bugs still show up in actual battle (Patriot) – No faith in system under real conditions ● Software mods in field – in a 30-90 min war this is not possible – threat is slightly different now

30. Implications ● Must meet real-time deadlines reliably – prescheduling would work, but don't know how many processes need to be scheduled ● Difficulty with building system increase with the size of the system, num of independently modifiable subsystems, and number interfaces – Interfaces change - loose couplings break – subsystems modified - tight couplings break – subsystems added - complexity increases ● Most massive, difficult, extreme demands, untestable ● Victims are all levels

31. Complex vs. Linear ● Proximity vs. Separation ● Common-mode connections vs. Dedicated ● Interconnected subsystems vs. Segregated ● Limited substitutions vs. Easy ● Feedback loops vs. Few feedback loops ● Multiple interacting controls vs. single purpose, segregated controls ● Indirect information vs. Direct Information ● Limited understanding vs. Extensive understanding

32. Tight vs. Loose ● Time dependent vs. delays possible ● Invariant sequences vs. Variant ● Only one way to reach production goal vs. many ways ● Little slack vs. lots of slack ● Buffers and redundancies are designed-in, deliberate vs. inherently everywhere ● Substitutions limited vs. substitutions abundant