Uncertainty & Decision Making James K. Hammitt Harvard Center for Risk Analysis.

Slides:



Advertisements
Similar presentations
Design of Experiments Lecture I
Advertisements

Outline input analysis input analyzer of ARENA parameter estimation
Statistics review of basic probability and statistics.
Simulation with ArenaAppendix C – A Refresher on Probability and StatisticsSlide 1 of 33 A Refresher on Probability and Statistics Appendix C.
Reading the Dental Literature
Sensitivity and Scenario Analysis
Engineering Economic Analysis Canadian Edition
Approaches to Data Acquisition The LCA depends upon data acquisition Qualitative vs. Quantitative –While some quantitative analysis is appropriate, inappropriate.
©GoldSim Technology Group LLC., 2004 Probabilistic Simulation “Uncertainty is a sign of humility, and humility is just the ability or the willingness to.
CropLife America & RISE 2014 Spring Conference Arlington, VA Finding Common Ground in the Pesticide Risk Assessment Process Bruce K. Hope, Ph.D.
Markus Amann The RAINS model: Modelling of health impacts of PM and ozone.
RAINS review 2004 The RAINS model: Health impacts of PM.
Chapter 12 Simple Regression
1/55 EF 507 QUANTITATIVE METHODS FOR ECONOMICS AND FINANCE FALL 2008 Chapter 10 Hypothesis Testing.
The Simple Regression Model
Understanding the relevance of climate model simulations to informing policy: An example of the application of MAGICC to greenhouse gas mitigation policy.
Inference.ppt - © Aki Taanila1 Sampling Probability sample Non probability sample Statistical inference Sampling error.
Lecture 10 Comparison and Evaluation of Alternative System Designs.
Using ranking and DCE data to value health states on the QALY scale using conventional and Bayesian methods Theresa Cain.
Value of Information for Complex Economic Models Jeremy Oakley Department of Probability and Statistics, University of Sheffield. Paper available from.
Environmental Risk Analysis
Chapter 10 Hypothesis Testing
Confidence Intervals and Hypothesis Testing - II
Math 647 March 18 & 20, Probability Probability: Probability: Probability of exceedance: Pr[ X > x ] or Pr [X x ] or Pr [X < x ] Relative frequency/proportions:
Qualitative and Quantitative Sampling
Air Quality Health Risk Assessment – Methodological Issues and Needs Presented to SAMSI September 19, 2007 Research Triangle Park, NC Anne E. Smith, Ph.D.
Estimation of Statistical Parameters
Statistics & Biology Shelly’s Super Happy Fun Times February 7, 2012 Will Herrick.
Lecture 12 Statistical Inference (Estimation) Point and Interval estimation By Aziza Munir.
Applied Welfare Econ & Cost Benefit Analysis
CS433 Modeling and Simulation Lecture 16 Output Analysis Large-Sample Estimation Theory Dr. Anis Koubâa 30 May 2009 Al-Imam Mohammad Ibn Saud University.
Value of information Marko Tainio Decision analysis and Risk Management course in Kuopio
Simon Thornley Meta-analysis: pooling study results.
Engineering Economic Analysis Canadian Edition
Geo597 Geostatistics Ch9 Random Function Models.
The Scientific Method Probability and Inferential Statistics.
Predicting the Benefits and Costs of Criminal Justice Policies TAD Conference, August 23, 2013 David L. Weimer La Follette School of Public Affairs University.
Statistics Introduction 2. The word Probability derives from the Latin probabilitas, which can also mean probity, a measure of the authority of a witness.
Brian Macpherson Ph.D, Professor of Statistics, University of Manitoba Tom Bingham Statistician, The Boeing Company.
Delivering Integrated, Sustainable, Water Resources Solutions Monte Carlo Simulation Robert C. Patev North Atlantic Division – Regional Technical Specialist.
Decision making Under Risk & Uncertainty. PAWAN MADUSHANKA MADUSHAN WIJEMANNA.
Section 9-1: Inference for Slope and Correlation Section 9-3: Confidence and Prediction Intervals Visit the Maths Study Centre.
FIN 614: Financial Management Larry Schrenk, Instructor.
1 Components of the Deterministic Portion of the Utility “Deterministic -- Observable -- Systematic” portion of the utility!  Mathematical function of.
Inference: Probabilities and Distributions Feb , 2012.
Chapter 15.3 Risk Assessment 2002 WHO report: “Focusing on risks to health is the key to preventing disease and injury.” risk assessment—process of evaluating.
Air Toxics Risk Assessment: Traditional versus New Approaches Mark Saperstein BP Product Stewardship Group.
Environmental Risk Analysis Chapter 6 © 2007 Thomson Learning/South-WesternCallan and Thomas, Environmental Economics and Management, 4e.
Global and Regional estimates of the Burden Due to Ambient Air Pollution: results from GBD ST AFRICA/MIDDLE-EAST EXPERT MEETING AND WORKSHOP ON THE.
Part 1d: Exposure Assessment and Modeling Thomas Robins, MD, MPH.
Various Topics of Interest to the Inquiring Orthopedist Richard Gerkin, MD, MS BGSMC GME Research.
RISK DUE TO AIR POLLUTANTS
Learning Objectives After this section, you should be able to: The Practice of Statistics, 5 th Edition1 DESCRIBE the shape, center, and spread of the.
Hypothesis Testing. Statistical Inference – dealing with parameter and model uncertainty  Confidence Intervals (credible intervals)  Hypothesis Tests.
“ Building Strong “ Delivering Integrated, Sustainable, Water Resources Solutions Uncertainty & Variability Charles Yoe, Ph.D.
Confidence Intervals and Hypothesis Testing Mark Dancox Public Health Intelligence Course – Day 3.
A Decision Framework for Vehicle Inspection and Maintenance Programs Using Health Benefit Analysis Ying Li University of North Carolina at Chapel Hill,
Monte Carlo Methods CEE 6410 – Water Resources Systems Analysis Nov. 12, 2015.
Outline Historical note about Bayes’ rule Bayesian updating for probability density functions –Salary offer estimate Coin trials example Reading material:
Exposure Prediction and Measurement Error in Air Pollution and Health Studies Lianne Sheppard Adam A. Szpiro, Sun-Young Kim University of Washington CMAS.
Stats 242.3(02) Statistical Theory and Methodology.
Uncertain Judgements: Eliciting experts’ probabilities Anthony O’Hagan et al 2006 Review by Samu Mäntyniemi.
Theoretical distributions: the Normal distribution.
Prior beliefs Prior belief is knowledge that one has about a parameter of interest before any events have been observed – For example, you may have an.
Uncertainty and controversy in environmental research
Uncertainty Analysis in Emission Inventories
Chapter 5 STATISTICAL INFERENCE: ESTIMATION AND HYPOTHESES TESTING
Uncertainty Analysis in Emission Inventories
Al-Imam Mohammad Ibn Saud University Large-Sample Estimation Theory
NAPLEX preparation: Biostatistics
Presentation transcript:

Uncertainty & Decision Making James K. Hammitt Harvard Center for Risk Analysis

2 Outline Uncertainty aversion & value of information Representing uncertainty as probability Policy evaluation –Components of uncertainty –Examples Diesel-vehicle emissions Mercury from power plants Expert judgment

3 Aversion to Risk, Uncertainty, Ambiguity, & Ignorance Humans dislike absence of certainty –Risk: "objective" probabilities –Uncertainty: subjective probabilities –Ambiguity: unknown probabilities –Ignorance: unknown possible outcomes Should we take greater precaution when risks are more uncertain? How should we describe uncertainty?

4 Perils of Prudence ( Nichols & Zeckhauser 1986) Conservative assumptions, worst-case analysis, uncertainty aversion can increase harm Technology DeathsProbabilityExpected deaths Uncertain , Certain Using upper-bound risk estimates, Certain would be preferred to Uncertain

5 Perils of Prudence If decision is repeated for 10 pairs of technologies (and risks are independent) Technology DeathsProbability Uncertain < 1, Certain 1, Policy of choosing Certain (with smaller upper- bound risk) is almost sure to kill more people

6 Value of Information For each of 10 technologies, learn true number of deaths for ambiguous type –Choose Uncertain if it causes 1 death –Choose Certain otherwise ChoiceExpected deaths Uncertain (always) 110 Certain (always)1,010 Perfect information 20 Expected value of information 90 lives saved

7 Value(s) of Information Increase chance of choosing decision that is best for actual conditions –"Expected value of information" in decision theory Overcome burden of proof needed to depart from status quo policy or default assumption –Compensate for decision rule that does not maximize expected value of outcome Reassure decision makers and affected public that decision is appropriate –Enhance compliance, minimize opposition & legal challenges –Incorporate compliance and challenges as factors in analysis?

8 Quantifying Uncertainty with Probability Probabilities of health risks are subjective –Often extrapolated from animal experiments or observational human data –Quantitative measure of degree of belief –Individuals can have different probabilities for same event There is no "true" or "objective" probability All probabilities are subjective –"Objective randomness" is not random but chaos (e.g., coin toss, roulette wheel) Deterministic process Sensitively dependent on initial conditions (butterfly flapping wings in China may cause hurricane in Atlantic) –Insufficient information about initial conditions

9 Disagreement Among Experts Individuals can hold different probabilities –When evidence to choose among them is inadequate As evidence accumulates –Experts should update their probabilities "When somebody persuades me that I am wrong, I change my mind. What do you do?" - John Maynard Keynes –Ultimately, probabilities should converge Coin toss, roulette wheel "In the long run we are all dead."- John Maynard Keynes

10 Quantifying Uncertainty About Policy Outcomes Use simulation model to combine multiple inputs –Inputs: releases to environment, fate & transport, human exposure, dose-response function –Outputs: adverse health events, benefits and costs Represent uncertainty about each component of model as probability distribution Calculate probability distribution of output using Monte Carlo analysis (or alternatives)

11 Components of Uncertainty "Model uncertainty" –Functional form –Causality "Parameter uncertainty" –Sampling variation in data (estimation error) –Relevance of data to application May be helpful to distinguish, but can combine using "super-model" –Weighted sum of alternative models, weights are uncertain parameters Note: statistical confidence intervals are not sufficient; exclude many important sources of uncertainty

12 Example: Low-Dose Extrapolation Estimate risk at high dose, where risk is measurable (e.g., 1/10, 1/100) Extrapolate to risk at low dose Extrapolation can be sensitive to choice among models that fit observed data equally well

Probability of Response Dose Response X – Linear ExtrapolationL – Logit Model M – Multi-Stage ModelG – Gamma Multi-Hit Model W – Weibull ModelP – Probit Model Low-dose extrapolation for 2-acetylaminofluorene under several mathematical models. Dose d (ppm) Excess Risk Low Dose Extrapolation XMWL, GP

14 Policy-Evaluation Examples Retrofit diesel trucks & buses in Mexico City Benefits of reducing mercury emissions from electric power plants

15 Primary PM SO 2 HC Primary PM Ammonium Sulfate Secondary Organic PM Death Capital O/M Inspection Catalyzed DPF Self-Regenerating DPF Removable DPF DOC Diesel Retrofit: Benefit-Cost Model

16 Annual Deaths Averted (per 1000 vehicles) (Error bars show interquartile range)

17 Net Benefits of Catalyzed Filter v. Alternatives (US$ millions per 1000 vehicles, model year ≥ 1994)

18 Relative importance of uncertainty about variables (Interquartile range, annual net benefits of retrofitting all vehicles with active regeneration filters holding other variables fixed at medians)

19 Mercury from Power Plants IQ loss? Heart attack? Deposition Emissions Fate & Bioaccumulation Exposure

20 Summary of Benefits

21 Benefits of Reducing MeHg Intake 10% (US)

22 Relative importance of uncertainty about variables (Correlation of input and output)

23 Benefits & Sensitivity to Key Parameter

24 Expert Judgment Risk assessment models incorporate many assumptions –Choices usually made by modelers, informed by scientific literature –Meta-analysis can be used when literature is rich Alternative (or complement): expert elicitation –Experts provide probability distributions for key parameters –Rigorous, replicable process Selection of experts Preparation Interview

25 Key Elicitation Question (Mortality Effect of PM 2.5 ) "What is your estimate of the true percent change in annual, all-cause mortality in the adult U.S. population resulting from a permanent 1µg/m 3 reduction in annual average PM 2.5 " 5 th, 25 th, 50 th, 75 th, and 95 th percentiles of cumulative density function

26 Source: EPA PM NAAQS RIA 2006

27 Source: EPA PM NAAQS RIA 2006

28 Performance: Expert Predictions of Ambient Benzene Concentrations NHEXAS Result Means90th Percentiles Source: Walker et al. 2003

29 Conclusions Outcomes of any policy alternative are uncertain ex ante Characterize uncertainty as probability distributions Propagate uncertainties about model inputs using Monte Carlo analysis Agreement on probabilities may not exist Pattern of precautionary policies may be costly