Uncertainty & Decision Making James K. Hammitt Harvard Center for Risk Analysis

2 Outline Uncertainty aversion & value of information Representing uncertainty as probability Policy evaluation –Components of uncertainty –Examples Diesel-vehicle emissions Mercury from power plants Expert judgment

3 Aversion to Risk, Uncertainty, Ambiguity, & Ignorance Humans dislike absence of certainty –Risk: "objective" probabilities –Uncertainty: subjective probabilities –Ambiguity: unknown probabilities –Ignorance: unknown possible outcomes Should we take greater precaution when risks are more uncertain? How should we describe uncertainty?

4 Perils of Prudence ( Nichols & Zeckhauser 1986) Conservative assumptions, worst-case analysis, uncertainty aversion can increase harm Technology DeathsProbabilityExpected deaths Uncertain , Certain Using upper-bound risk estimates, Certain would be preferred to Uncertain

5 Perils of Prudence If decision is repeated for 10 pairs of technologies (and risks are independent) Technology DeathsProbability Uncertain < 1, Certain 1, Policy of choosing Certain (with smaller upper- bound risk) is almost sure to kill more people

6 Value of Information For each of 10 technologies, learn true number of deaths for ambiguous type –Choose Uncertain if it causes 1 death –Choose Certain otherwise ChoiceExpected deaths Uncertain (always) 110 Certain (always)1,010 Perfect information 20 Expected value of information 90 lives saved

7 Value(s) of Information Increase chance of choosing decision that is best for actual conditions –"Expected value of information" in decision theory Overcome burden of proof needed to depart from status quo policy or default assumption –Compensate for decision rule that does not maximize expected value of outcome Reassure decision makers and affected public that decision is appropriate –Enhance compliance, minimize opposition & legal challenges –Incorporate compliance and challenges as factors in analysis?

8 Quantifying Uncertainty with Probability Probabilities of health risks are subjective –Often extrapolated from animal experiments or observational human data –Quantitative measure of degree of belief –Individuals can have different probabilities for same event There is no "true" or "objective" probability All probabilities are subjective –"Objective randomness" is not random but chaos (e.g., coin toss, roulette wheel) Deterministic process Sensitively dependent on initial conditions (butterfly flapping wings in China may cause hurricane in Atlantic) –Insufficient information about initial conditions

9 Disagreement Among Experts Individuals can hold different probabilities –When evidence to choose among them is inadequate As evidence accumulates –Experts should update their probabilities "When somebody persuades me that I am wrong, I change my mind. What do you do?" - John Maynard Keynes –Ultimately, probabilities should converge Coin toss, roulette wheel "In the long run we are all dead."- John Maynard Keynes

10 Quantifying Uncertainty About Policy Outcomes Use simulation model to combine multiple inputs –Inputs: releases to environment, fate & transport, human exposure, dose-response function –Outputs: adverse health events, benefits and costs Represent uncertainty about each component of model as probability distribution Calculate probability distribution of output using Monte Carlo analysis (or alternatives)

11 Components of Uncertainty "Model uncertainty" –Functional form –Causality "Parameter uncertainty" –Sampling variation in data (estimation error) –Relevance of data to application May be helpful to distinguish, but can combine using "super-model" –Weighted sum of alternative models, weights are uncertain parameters Note: statistical confidence intervals are not sufficient; exclude many important sources of uncertainty

12 Example: Low-Dose Extrapolation Estimate risk at high dose, where risk is measurable (e.g., 1/10, 1/100) Extrapolate to risk at low dose Extrapolation can be sensitive to choice among models that fit observed data equally well

Probability of Response Dose Response X – Linear ExtrapolationL – Logit Model M – Multi-Stage ModelG – Gamma Multi-Hit Model W – Weibull ModelP – Probit Model Low-dose extrapolation for 2-acetylaminofluorene under several mathematical models. Dose d (ppm) Excess Risk Low Dose Extrapolation XMWL, GP

14 Policy-Evaluation Examples Retrofit diesel trucks & buses in Mexico City Benefits of reducing mercury emissions from electric power plants

15 Primary PM SO 2 HC Primary PM Ammonium Sulfate Secondary Organic PM Death Capital O/M Inspection Catalyzed DPF Self-Regenerating DPF Removable DPF DOC Diesel Retrofit: Benefit-Cost Model

16 Annual Deaths Averted (per 1000 vehicles) (Error bars show interquartile range)

17 Net Benefits of Catalyzed Filter v. Alternatives (US$ millions per 1000 vehicles, model year ≥ 1994)

18 Relative importance of uncertainty about variables (Interquartile range, annual net benefits of retrofitting all vehicles with active regeneration filters holding other variables fixed at medians)

19 Mercury from Power Plants IQ loss? Heart attack? Deposition Emissions Fate & Bioaccumulation Exposure

20 Summary of Benefits

21 Benefits of Reducing MeHg Intake 10% (US)

22 Relative importance of uncertainty about variables (Correlation of input and output)

23 Benefits & Sensitivity to Key Parameter

24 Expert Judgment Risk assessment models incorporate many assumptions –Choices usually made by modelers, informed by scientific literature –Meta-analysis can be used when literature is rich Alternative (or complement): expert elicitation –Experts provide probability distributions for key parameters –Rigorous, replicable process Selection of experts Preparation Interview

25 Key Elicitation Question (Mortality Effect of PM 2.5 ) "What is your estimate of the true percent change in annual, all-cause mortality in the adult U.S. population resulting from a permanent 1µg/m 3 reduction in annual average PM 2.5 " 5 th, 25 th, 50 th, 75 th, and 95 th percentiles of cumulative density function

26 Source: EPA PM NAAQS RIA 2006

27 Source: EPA PM NAAQS RIA 2006

28 Performance: Expert Predictions of Ambient Benzene Concentrations NHEXAS Result Means90th Percentiles Source: Walker et al. 2003

29 Conclusions Outcomes of any policy alternative are uncertain ex ante Characterize uncertainty as probability distributions Propagate uncertainties about model inputs using Monte Carlo analysis Agreement on probabilities may not exist Pattern of precautionary policies may be costly