University of Virginia Department of Computer Science Complex Systems and System Accidents presented by: Joel Winstead

High-risk systems Many high-risk systems: airplanes, chemical plants, nuclear power, dams These systems are complex, with many interacting parts Many industrial accidents For some systems, we cannot tolerate failures Concern that risks appear faster than solutions

What is a system? Organizations, and organizations of organizations Set of interrelated components that act together as a whole to achieve a common goal

What is a system? A system is an abstraction or model A system has state an environment inputs outputs subsystems

Methodological Reductionism Analyze a system by breaking it into parts This assumes: Division into parts does not distort the system Components are the same when examined separately Principles governing assembly into whole are straightforward

Complexity Organized simplicity reductionism works Unorganized complexity e.g., ideal gas laws Organized Complexity systems analysis

Hierarchies and Emergence A complex system has a hierarchy of levels of organization Each level has its own rules and structure There are some properties that cannot be reduced to lower levels

Communication and Control Hierarchies separated by interfaces Control processes operate across interfaces Control processes impose constraints on lower levels in the hierarchy

History of Safety Design Factories not legally responsible for worker’s injuries Safety concerns often ignored A series of accident studies, pressure from labor unions, and legislation changed this Later, realization that production increases as safety increases

Safety Devices Machinery not initially designed for safety Accident-investigation-fix approach Guards attached to machinery to prevent some kinds of accidents Safety should be built into design This eventually led to universal safety standards

World War II Production Initially, focus shifted back to functionality over safety But, industrial accidents hurt war effort more killed in industrial accidents than battlefield Increased complexity means a posteriori methods no longer work People began to think in terms of systems

Systems Engineering and Analysis Large, complex, semi-automatic, unpredictable systems Must analyze system as a whole Needs analysis, feasibility studies, trade studies, architecture development, interface analysis

System Accidents Sometimes components fail Some events in systems are tightly coupled This leads to interactive complexity In order to understand the failure, we need to understand the system and not just the first component to fail

Normal Accidents Normal = inherent, not expected or frequent Multiple failures Tight coupling Interdependence of events not visible to operator Inherent property of systems, not components

Perrow’s Day in the Life The story begins with a coffee pot left on Many seemingly unrelated things fail, resulting in our hero being unable to get to an important appointment What was the primary cause of this?

Complexity is to blame There was coupling where it wasn’t expected Redundant paths don’t help when there are multiple failures or tight coupling Some components not normally considered individually important had large consequences

Aren’t real systems designed? This “system” consists of many separately designed components stuck together in an ad-hoc way It is not how this particular system was designed, but the kinds of failures and couplings that occurred in it that are interesting Jumbo jets have coffee pots too

What can we do about this? Adding new safety systems just adds new systems to the mix We need to avoid the properties that make these systems complex We won’t always be able to do this We need to consider what systems we really need

Are Perrow and Leveson talking about the same thing? Leveson focuses on how systems are built and designed Perrow focuses on how systems fail Are they talking about the same “systems”?