1. Stakeholder Preference Modeling with Probabilistic Inversion Roger M. Cooke Resources for the Future Dept Math, TU Delft June 16, 2011

2. Foundations Health states Risks nano-enabled food

3. Expert Judgment for Uncertainty Quantification: PM 2.5 Uncertainty in Mortality Response to Airborne Fine Particulate Matter: Combining European Air Pollution Experts Jouni T. Tuomisto, Andrew Wilson, John S. Evans, Marko Tainio (RESS 2008)

4. Fundamental Theorem of Decision Theory For Rational Preference UNIQUE probability P which represents degree of belief: DegBel(France wins worldcup) > DegBel(Belgium wins worldcup)  P(F) > P(B) AND a Utility function, unique up to 0 and 1, that represents values: ($1000 if F, $0 else) > ($1000 if B, $0 else)  Exp’d Utility (($1000 if F, $0 else)) > Exp’d Utility (($9000 if B, $0 else)) BUT….

5. UNLIKE Expert Judgment: There is no Updating utilities on observations Convergence of utilities via Observations Empirical control on Utilities Community of ‘Utility Experts’ Rational consensus on Utilities

6. Why is Preference Modeling Impoverished? AHP MAUT MCDM ELECTRA REMBRANT OUTRANKING THURSTONE BRADLEY TERRY PROBIT LOGIT NESTED LOGIT PSYCH’L SCALING Validation ???

7. What means Validation? Goal = find ‘true Utility values’ for alternatives? Fools’s Errand

8. Condorcet’s Paradox of Majority Preference 1/3 prefer Mozart > Hayden > Bach 1/3 prefer Hayden > Bach > Mozart 1/3 prefer Bach > Mozart > Hayden THEN 2/3’s prefer Bach > Mozart Mozart > Hayden Hayden > Bach

9. What can we do? Random Utility Theory Each (rational) stakeholder has a utility function over alternatives  characterize population as distribution over utility functions

10. Probabilistic Inversion Domain: utility functions Of stakeholders Range : choices of stakeholders G maps utilities into choices Invert G at this distribution Observe Stakeholders Preferences

11. Used for stakeholder Preference Modeling: Risks of Nano enabled foods (Flari, WHO, CIS) Valuing impaired health states (Flari, FDA) Valuing fossil fuel policy options (RFF) Prioritizing ecosystem threats (NCEAS) Prioritizing zoonose threats (RIVM) Modeling wiring failure (Mazzuchi) Prioritizing vCVJ options (Aspinall Health Canada) UK Research Council (Aspinall) Aus. Univ. Fac Sci reviews (Aspinall).

12. steps 1.Get discrete choice data from stakeholders for choice alternatives A1,…An – “Which of (A,B) do you prefer” – “Rank your top 3 of (A, B, C, D, E, F)” 2.Find dist’n over utilities on [0,1] n which reproduces stakeholders preferences 3.If utility is function of covariates, validate out of sample.

13. Valuation of impaired Health states Flari et al 17 health states 6 criteria Each criteria has 3 values, described in narrative 19 Experts ranked 5 groups of 5 health states

14. First, find dist’n over utilities for the 17 Health States which recover Observed Frequencies of rankings (i.e. wo criteria)

15. Build MAUT model for HS utilities, based on the 6 criteria Each stakeholder has a weight vector that determines his/her preference Population of stakeholders = population of weights Characterize population based on all rankings involving at least 7 (30%) experts (= 28 rankings). Validate on remaining rankings (= 77 rankings)

16. Average weights

17. Preference dependence emerges from fitting

18. Average weights per group of 5 health states

19. Predict out-of-sample rankings First time in HISTORY that a multi attribute model has been WRONG!!!

20. Average of predictions vs Out-of- Sample observed rankings, Not SOOO bad

21. Risks of Nano-Enabled Foods Flari et al

22. Nano enabled food risks (VLARI) rank top and bottom 5

25. Correlation of criteria weights

26. Conclusion Stakeholder preference modeling is empirical science ‘preference for criteria’ inferred from data, not elicited (in) dependence in choices inferred from data, not assumed THANK YOU