Expert elicitation Jouni Tuomisto THL

Outline What is expert elicitation An example: ERF and health impacts for PM2.5 Key concepts: Calibration Informativeness Equal-weight decision-maker Performance-based decision-maker

What is expert elicitation In decision analysis, you want to rely on data. However, often data does not exist or is not feasible to get. Then, you can/must rely on subjective estimates. Expert elicitation is about getting subjective estimates in a systematic, controllable, and testable way.

Expert elicitation is a scientific instrument You must know to which area it may be applied. The process of using it must be repeatable. The measurements must depend on the issue measured and not on the properties of the instrument. The measurement process must be acceptable in the eyes of the society. The measurements must hold against validation. It must be calibrated.

What should be controlled The process of selecting experts. The performance of experts in the particular field. The clarity of questions (clairvoyant test). Understand the mental models of experts about the issue.

PM2.5 expert elicitation What is the exposure-response function of PM2.5 on mortality? What are the ERFs in the USA and Europe? Specifically, what are the health impacts of PM2.5 emissions from the Kuwait oil fires in 1991 in the Kuwaitis (citizens of Kuwait)?

Expert selection Based on peer nominations: PM literature search revealed authors in the area. These authors were asked to nominate experts. Top 10 nominees were asked to participate. Six accepted. Dr. Bert Brunekreef (Ph.D., Prof) Environmental Epidemiology, IRAS, UU, Netherlands. Dr. Annette Peters (Ph.D., Assistant prof) Assistant Professor and Head of the Epidemiology of Air Pollution Health Effects Research Unit of the Institute of Epidemiology at the GSF National Research Center for Environment and Health, Neuhenberg, Germany. Dr. Nino Ku¨ nzli (MD, Ph.D.), formerly Assistant Professor at the Institute for Social and Preventive Medicine at the University of Basel (Switzerland), is Associate Professor at the University of Southern California Keck School of Medicine, USA. Dr. H. Ross Anderson (MD, Ph.D.) is Professor and Head of the Department of Public Health Sciences at St. George’s Hospital Medical School at the University of London, England. Dr. Ken Donaldson (Ph.D.) is Professor of Respiratory Toxicology on the Faculty of Medicine at the University of Edinburgh, Scotland. Dr. Juha Pekkanen (MD, Ph.D.) is Head of the Environmental Epidemiology Unit in the Department of Environmental Health at the National Public Health Institute, Finland.

Questions asked Twelve questions, typically the following format Q1. What is your estimate of the true, but unknown, percent change in the annual, non-accidental mortality rate in the adult US population resulting from a permanent 1 mg/m3 reduction in long-term annual average PM2.5 (from a population-weighted baseline concentration of 18 mg/m3) throughout the USA?

Results: experts mostly agree

Results: experts disagree