1. Mission Success Starts With Safety 1 Probabilistic Risk Assessment: NASA Strategy for Capability Enhancement Dr. Michael G. Stamatelatos Manager, Risk Assessment NASA Headquarters Office of Safety and Mission Assurance mstamate@hq.nasa.gov (202) 358-1668

2. Mission Success Starts With Safety 2 Outline Introduction PRA History at NASA NASA PRA Capability Improvement Strategy On-Going Efforts

3. Mission Success Starts With Safety 3 Introduction

4. Mission Success Starts With Safety 4 PRA Simply Described

5. Mission Success Starts With Safety 5 Types of Risk and Related Consequences Deaths, injuries, illness Contamination, loss of use Money lost Mission, schedule, etc.

6. Mission Success Starts With Safety 6 Risk Sources in Safety Risk Assessment Hardware Failures Hardware Failures Organizational Factors Organizational Factors External Events, Acts of Nature External Events, Acts of Nature Human Error Increasing degree of complexity in modeling

7. Mission Success Starts With Safety 7 Risk Assessment & Management Qualitative Risk Assessment FMEA FTA Qualitative Risk Assessment FMEA FTA Quantitative Risk Assessment Initiating Events Scenario Modeling (MLD, ESD, ETA, FTA) Risk Quantification Uncertainty Evaluation Quantitative Risk Assessment Initiating Events Scenario Modeling (MLD, ESD, ETA, FTA) Risk Quantification Uncertainty Evaluation Inputs Mission Success Criteria Technical Data Cost Schedule Management Procedures Other Improved Safety & PerformanceImproved Performance

8. Mission Success Starts With Safety 8 Benefits of PRA Probabilistic Risk Assessment (PRA) has proven to be a systematic, logical, and comprehensive tool to assess risk (likelihood of unwanted consequences) in modern technological applications (e.g., nuclear power, electric power generation, chemical processing industry), for:  Improving system performance & mission success  Increasing safety in design, operation & upgrade  Saving money in design, manufacturing, assembly and operation

9. Mission Success Starts With Safety 9 When Should PRA Be Performed?  When important decisions must be made about complex systems under uncertainty  When information is not sufficient to comprehensively assess strengths and weaknesses of complex systems by other means  When important complex jobs must be performed successfully for the first time  In all life cycle phases of a complex system

10. Mission Success Starts With Safety 10 Brief History of PRA at NASA

11. Mission Success Starts With Safety 11 Beginning with the Challenge of Apollo... “I believe that this nation should commit itself to achieving the goal, before this decade is out, of landing a man on the moon and returning him safely to the earth.” -- President John F. Kennedy, May 25, 1961  Early Apollo program estimate of mission success probability was approximately 0.20--not what people wanted to hear.  On July 20, 1969, the human race accomplished its single greatest technological achievement of all time when a human first set foot on another celestial body.  5 additional successful Moon missions (out of 6 attempts) occurred between 1969 and 1972 = 6 out of 7 = 0.86 demonstrated mission success.  So much for PRA in NASA for a long time to come!  Instead NASA would rely on FMEAs

12. Mission Success Starts With Safety 12 …Then Challenger  January 28, 1986, after 25 successful flights, the Space Shuttle Challenger explodes.  October 29, 1986, “Investigation of the Challenger Accident,” by the Committee on Science and Technology, House of Representatives:  “Without some means of estimating the probability of failure of the various [Shuttle] elements it is not clear how NASA can focus its attention and resources as effectively as possible on the most critical systems.”  January 1988, “Post-Challenger Evaluation of Space Shuttle Risk Assessment and Management,” by the Slay Committee:  “The Committee recommends that probabilistic risk assessment approaches be applied to the Shuttle risk management program at the earliest possible date. Data bases derived from STS failures, anomalies, and flight and test results, and the associated analysis techniques, should be systematically expanded to support probabilistic risk assessment, trend analysis, and other quantitative analyses relating to reliability and safety.”

13. Mission Success Starts With Safety 13 The Return to PRA  Dec. ‘87 -- Space Shuttle Proof-of-Concept Study  Feb. ‘88 -- Space Shuttle Main Propulsion Pressurization System PRA  Nov. ‘88 -- Enhanced Hazard Analysis for Space Systems  Apr. ‘89 -- Independent Assessment of Shuttle Accident Scenario Probabilities for the Galileo Mission  Jul. ‘90 -- Space Station Freedom -- External Maintenance Task Team -- Final Report -- the “Fisher-Price Study”  Dec. ‘90 -- Safety of the Thermal Protection System of the Space Shuttle Orbiter -- Quantitative Analysis and Organizational Factors  Jun. ‘91 -- 8’ High Temperature Tunnel PRA(@ LaRC)  ‘92 -- Advanced Solid Rocket Motor Field Joint Case Sealing System Leak Check vs. No Leak Check

14. Mission Success Starts With Safety 14 The Return to PRA (cont’d)  Jul. ‘92 -- Risk Assessment of the ASRM Propellant Manufacturing Facility  Jul. ‘93 -- Reliability and Requirements Analysis for Space Exploration Initiative Vehicles: Comparative Risk Assessment -- the Space Shuttle and an All Rocket Single Stage to Orbit Vehicle  Sep. ‘93 -- An Analysis of Selected Risk Factors and Uncertainties for Space Station Assembly Up to Human Tended Condition for Space Station Transition Options A1 and A2  Jan. ‘94 -- An Investigation of the Risk Implications of Space Shuttle Solid Rocket Booster Chamber Pressure Excursions  Feb. ‘95 -- PRA of the Space Shuttle -- A Study of the Potential of Losing the Vehicle During Normal Operations  And others including PRA work done in support of Ulysses and Cassini missions.

15. Mission Success Starts With Safety 15 The NASA Administrator Steps In...  July 29, 1996, the NASA Administrator, Dan Goldin:  “Since I came to NASA [1992], we’ve spent billions of dollars on Shuttle upgrades without knowing how much they improve safety. I want a tool to help base upgrade decisions on risk.”  Earlier “paper PRAs” prepared by NASA contractors would not serve the purpose.  October 1997, an early version of the NASA Quantitative Risk Assessment System (QRAS) is demonstrated to the Administrator.  February 1998, Version 1.0 of QRAS is baselined.  Two other intermediate version have been tested  March 2001, Version 1.6 of QRAS will be delivered. It will have full PRA capabilities.

16. Mission Success Starts With Safety 16 Space Shuttle » Johnson Space Center and Marshal Space Flight Center have been modeling their Shuttle elements. » Space Shuttle Program has begun to factor risk into their Upgrades Program.

17. Mission Success Starts With Safety 17 International Space Station  1999 -- The NASA Advisory Council recommended, the NASA Administrator concurred, and the ISS Program has begun a PRA. » First portion of PRA (through Flight 7A) delivered in Dec. ‘99; 2nd portion (through Flight 10A) expected in Dec. ‘00. » Using the SAPHIRE software application for conducting PRA.  Objectives of ISS PRA: » Provide a quantitative look at ISS operations risk » Provide a model for future ISS safety decision-support activities » Provide a model for safety related operations planning » Provide a model for trading marginal safety enhancements versus cost

18. Mission Success Starts With Safety 18 Mars Sample Return Mission  Mission must meet a Planetary Protection Program criterion of <10 -6 probability of Earth contamination upon return of sample  Use of PRA is being seriously considered as a means to evaluate mission compliance with the PPP criterion

19. Mission Success Starts With Safety 19 The Risk Management Picture at NASA NASA Procedures and Guidelines 7120.5A, “NASA Program and Project Management Processes and Requirements,” April 3, 1998 – Requires NASA Program & Project Managers to manage risk formally – We are seeing evidence of real risk management in numerous NASA projects – Risk management is a factor in high-level program/project decision- making “Continuous Risk Management” training course developed and pilot- tested on numerous NASA project teams – To be picked up by NASA’s APPL in FY 01 Risk-Based Acquisition Management (R-BAM) – Interim rule entitled “Risk Management,” -- published in Federal Register June 14, 2000; effective July 14, 2000 – Changes NASA Supplement to the Federal Acquisition Regulations (FAR) to emphasize considerations of risk management in the acquisition process

20. Mission Success Starts With Safety 20 Strategy for PRA Capability Enhancement

21. Mission Success Starts With Safety 21 Current PRA Status at NASA Good News:  Strong management interest and support for PRA  In-house experience with traditional FMEA and some FTA Bad News:  Scarce and scattered PRA resources (people, tools, data)  No corporate memory on PRA past work and data  Inadequate communication and cooperation on PRA among Centers and with HQ

22. Mission Success Starts With Safety 22 NASA Objective For PRA Develop a world-class in-house capability to perform, manage, and use Probabilistic Risk Assessment (PRA) methods at NASA

23. Mission Success Starts With Safety 23 Ingredients for Success from Experience  In-house expertise to perform, manage and use PRAs to make sound decisions  In-house ownership and corporate memory of PRA methods, tools, databases and results  Lowest dependence on outside help to manage, perform, understand, and use PRA methods and results to make management decisions

24. Mission Success Starts With Safety 24 NASA PRA Capability Growth Model Total Dependence on External Consultants NASA is Only a Customer of Its PRA Projects NASA on a PRA Learning Curve; Takes Leadership in PRA Consultants Provide Transfer of Technology and Support in PRA NASA With PRA Experts; Manages/Performs PRA Projects Outside PRA Consultants Just Provide Support Today Near-TermLong-Term 1-2 years 2-5 years Time frame RoleShiftRoleShift PRA RenaissanceAge of PRA Enlightenment

25. Mission Success Starts With Safety 25 Essential Elements for PRA Strategy  Develop NASA personnel skilled in PRA to transfer the state-of-the-art PRA technology to  Develop/adopt NASA-wide PRA policy, procedures, computer tools  Develop, coordinate & maintain PRA databases (reliability, maintenance, initiating events, etc.)  Foster communication and cooperation of efforts throughout NASA  Integrate and coordinate PRA efforts throughout NASA

26. Mission Success Starts With Safety 26 Strategy Implementation Roles NASA HQ  Organize, coordinate and conduct awareness and practitioner PRA training  Develop PRA policy and procedures  Mentor and provide specialized PRA assistance  Coordinate use of PRA computer tools and consultants  Manage the PRA process and oversee PRA quality NASA Centers  Work with HQ to implement PRA strategy  Manage and perform PRA projects  Provide local PRA expertise and support in training, consulting and mentoring PRA projects  Become local custodians of PRA data and software

27. Mission Success Starts With Safety 27 We are Up and Running...  Initiated work on PRA policy and PRA awareness and practitioner training  Started effort on PRA Procedures Guide for aerospace applications  Began cooperation with NRC, USG agency most experience in PRA  Acquired SAPHIRE PRA computer program and conducted training at HQ and Centers; Trained 50 people to date.  Organized PRA information exchange workshop (this workshop) and the development of NASA-wide PRA working group  Started cooperation with ESA on PRA policy and procedures  Planning additional workshops on PRA and on risk-based decision making for management applications

28. Mission Success Starts With Safety 28 Conclusion NASA needs your help, support and cooperation to reach our PRA objective for the benefit of NASA and its Centers