1. Futron Corporation 2021 Cunningham Drive, Suite 303 Hampton, Virginia 23666 Phone 757-262-2074 Fax 757-262-2173 www.futron.com Results You Can Trust Assessing Uncertainty HRA – INCOSE November 8 & 9 2007

2. Results You Can Trust 2 Overview Interactive Exercise Definitions Why Assess Uncertainty Interactive Exercise Types of Uncertainty Sources of Uncertainty Assessing (Bounding) Uncertainty Interactive Exercise Summary

3. Results You Can Trust 3 Interactive Exercise

4. Results You Can Trust 4 Definitions: Uncertainty Uncertainty: may range from a falling short of certainty to an almost complete lack of conviction or knowledge especially about an outcome or result. Lack of clear definition

5. Results You Can Trust 5 Definitions: Risk Risk: possibility (chance) of loss or injury (Likelihood, Consequence. Ordered Pair ) Probability: (1) : the ratio of the number of outcomes in an exhaustive set of equally likely outcomes that produce a given event to the total number of possible outcomes (2) : the chance that a given event will occur (3) A numerical measure of the likelihood of an event relative to a set of alternative events. Likelihood: probability

6. Results You Can Trust 6 Definitions: Confidence Interval Confidence Interval: a group of continuous or discrete adjacent values that is used to estimate a statistical parameter (as a mean or variance) and that tends to include the true value of the parameter a predetermined proportion of the time if the process of finding the group of values is repeated a number of times Used to quantify/communicate amount of certainty 90% confidence interval (limit/bounds) mean

7. Results You Can Trust 7 Why Assess Uncertainty? We manage risks to ensure we meet our objectives We have tasks, projects, and programs to accomplish  They all have objectives  We all make decisions and commit resources to meet these objectives  Decision are choices between alternatives and involve uncertainty  This uncertainty leads to risk Managing uncertainty is a part of our job  Involves understanding our decisions  Managing the uncertainty in those decisions  Identifying the risks to meeting our objectives that come from uncertainty  Make better decisions / select better mitigation strategies

8. Results You Can Trust 8 Interactive Exercise

9. Results You Can Trust 9 Aleatory  Randomness or variability that remains when the system is well known  Inherent Variability  You can calculate the odds  Example: Drawing a queen of diamonds from a standard deck of 52 playing cards on the first draw.  Not reducible Types of Uncertainty: Aleatory

10. Results You Can Trust 10 Epistemic  Stems from limitations of fundamental knowledge about a phenomenon of interest.  Model Uncertainty  Can be revealed by divergences in the opinions of experts who may disagree.  Example: Drawing a queen of diamonds from a deck of playing cards on the first draw when you have no idea how many cards are in the deck  This type of uncertainty is reducible: Improvements of current knowledge Improvements of corresponding models Collection of additional data (test results) Types of Uncertainty: Epistemic

11. Results You Can Trust 11 Sources of Uncertainty Insufficient data  Measurement errors  State not directly observed (inferred)  Imperfect understanding of processes  Not enough data to be statistically significant Natural variability  Varies spatially (with space)  Varies temporally (with time) Model imperfections Underestimating uncertainty and the illusion of control - We tend to underestimate future uncertainty because we tend to believe we have more control over events than we really do. We believe we have control to minimize potential problems in our decisions.

12. Results You Can Trust 12 Six levels of “sophistication” Simply establish the existence of uncertainty (risk) Identifying a “worst-case” scenario (unbiased bounds – one tailed or two tailed) Computing the effects of the quasi-worst case scenario or plausible upper bound Computing a maximum-likelihood estimate of the failure probability of losses Modeling the probability distribution of potential outcomes (risk curve) Modeling explicitly the uncertainties about the risk, separating epistemic from aleatory Bounding Uncertainty

13. Results You Can Trust 13 Interactive Exercise

14. Results You Can Trust 14 Summary It is not easy We think we know more than we really do Understand, assess and communicate your own uncertainty Understand how others assess and communicate their uncertainty Focus Mitigation Strategies

15. Results You Can Trust 15 Bibliography Ang, Alfred H-S. and Tang, Wilson H. “Probability Concepts in Engineering Planning and Design, Volume I – Basic Principles” Ang, Alfred H-S. and Tang, Wilson H. “Probability Concepts in Engineering Planning and Design, Volume II – Decision, Risk and Reliability.” John Wiley, New York 1984 Roberds, William J. (1990). “Methods for Developing Defensible Subjective Probability Assessments,” Transportation Research Record No. 1288, 183-190. CE311S Elementary Probability and Statistics for Civil Engineers class notes, University of Texas – Austin Futron Technical University (FTU) Fundamentals of Risk Management course notes Decision Analysis Training for ESMD Risk Management Officers, by Dr. Douglas Stanley, Georgia Institute of Technology, National Institute of Aerospace The Epistimic Uncertainty Project, Sandia National Laboratories http://www.sandia.gov/epistemic/ Wikipedia (interactive exercises) http://en.wikipedia.org/wiki/Main_Page http://en.wikipedia.org/wiki/Main_Page Encyclopedia Britannica Online http://www.britannica.com/http://www.britannica.com/