1. Adapting de Finetti's Proper Scoring Rules for Measuring Subjective Beliefs to Modern Decision Theories of Ambiguity Gijs van de Kuilen, Theo Offerman, Joep Sonnemans, & Peter P. Wakker June 23, 2006 FUR, Rome Topic: Our chance estimates of various soccer-teams to become world-champion. E: Brasil will win. not-E: other team.

2. Imagine following bet: You choose 0  r  1, as you like. We call r your reported probability of Brasil, and 1–r your reported probability of not-Brasil. You receive E not-E   1 – (1– r) 2 1 – r 2 What r should be chosen? 2

3. Rational model: Subjective expected utility (SEU). Moderate amounts: U is linear. So: SEV. After some algebra:...... Optimal r = your true subjective probability of Brasil winning. !!! Wow !!! 3

4. "Bayesian truth serum" (Prelec, Science, 2005). Medicine against "frequentism." Superior to elicitations through preferences . Superior to elicitations through indifferences ~ (BDM). Widely used: Hanson (Nature, 2002), Prelec (Science 2005). In accounting (Wright 1988), Bayesian statistics (Savage 1971), business (Stael von Holstein 1972), education (Echternacht 1972), medicine (Spiegelhalter 1986), psychology (Liberman & Tversky 1993; McClelland & Bolger 1994), experimental economics (Nyarko & Schotter 2002). We want to introduce these very nice things into the FUR-nonEU world. 4

5. Survey Part I. Deriving r from theories (SEV, SEU, RDU for probabilistic sophistication, RDU for ambiguity ("CEU"). Part II. Deriving theories from observed r. In particular: Derive beliefs/ambiguity attitudes. Will turn out to be surprisingly easy. Proper scoring rules Nonexpected utility: Mutual benefits. Part III. Implementation of our method in an experiment. 5

6. Part I. Deriving r from Theories (SEV, and then 3 deviations). 6

7. Let us assume you very strongly believe in Brasil (Ronaldinho …) Your "true" subj. prob.(Brasil) = 0.75. SEV: Then your optimal r E = 0.75. 7

8. 0.25 0.50 0.75 1 0 p 8 Reported probability R(p) = r E as function of true probability p, under: nonEU 0.69 R(p) EU 0.61 r EV EV r nonEU r nonEUA r nonEUA : nonexpected utility for unknown probabilities ("Ambiguity"). (c) nonexpected utility for known probabilities, with U(x) = x 0.5 and with w(p) as common; (b) expected utility with U(x) =  x (EU); (a) expected value (EV); r EU next p. go to p. 11, Example EU go to p. 15, Example nonEU 0 0.50 1 0.25 0.75 go to p. 19, Example nonEUA

9. So far we assumed SEV (as does no-one at FUR, but as does the whole ocean of literature that uses proper scoring rules...) Deviation 1 from SEV. What if you want to bet on Brasil with larger stakes [SEU with U nonlinear]? Now optimize pU ( 1 – (1– r) 2 ) + ( 1 – p )U (1 – r 2 ) 9

10. 10 U´(1–r 2 ) U´(1 – (1–r) 2 ) (1–p) p + p r = Reversed (and explicit) expression: U´(1–r 2 ) U´(1 – (1–r) 2 ) (1–r) r + r p =

11. How bet on Brasil? [ Expected Utility ]. EV: r EV = 0.75. Expected utility, U(x) =  x: r EU = 0.69. You now bet less on Brasil. Closer to safety. (Winkler & Murphy 1970.) 11 go to p. 8, with figure of R(p)

12. Deviation 2 from SEV : nonexpected utility for probabilities ( Allais 1953, Machina 1982, Kahneman & Tversky 1979, Quiggin 1982, Schmeidler 1989, Gilboa 1987, Gilboa & Schmeidler 1989, Gul 1991, Tversky & Kahneman 1992, etc.) 12 For two-gain prospects, virtually all those theories are as follows: For r  0.5, nonEU(r) = w(p)U ( 1 – (1–r) 2 ) + ( 1–w(p) ) U(1–r 2 ). r < 0.5, symmetry; soit! Different treatment of highest and lowest outcome: "rank-dependence."

13. p w(p) 1 1 0 1/3 Figure. The common weighting function w. w(p) = exp(–(–ln(p))  ) for  = 0.65. w(1/3)  1/3; 13 1/3 w(2/3) .51 2/3.51

14. Now 14 U´(1–r 2 ) U´(1 – (1–r) 2 ) ( 1–w(p) ) w(p) + w(p) r = U´(1–r 2 ) U´(1 – (1–r) 2 ) (1–r) r + r p = Reversed (explicit) expression: w –1 ( )

15. How bet on Brasil now? [nonEU with probabilities ]. EV: r EV = 0.75. EU: r EU = 0.69. Nonexpected utility, U(x) =  x, w(p) = exp(–(–ln(p)) 0.65 ). r nonEU = 0.61. You bet even less on Brasil. Again closer to safety. 15 go to p. 8, with figure of R(p) Deviations from EV and Bayesianism were at level of behavior so far; were not at level of beliefs. Now for something different; more fundamental.

16. 3 rd violation of EV: Ambiguity (unknown probabilities; belief/decision-attitude? Yet to be settled). No objective data on probabilities. How deal with unknown probabilities? Have to give up Bayesian beliefs descriptively. According to some even normatively. 16

17. 17 Instead of additive beliefs p = P(E), nonadditive beliefs B(E) (Dempster&Shafer, Tversky&Koehler, etc.) All currently existing decision models: For r  0.5, nonEU(r) = w(B(E))U ( 1 – (1–r) 2 ) + ( 1–w(B(E)) ) U(1–r 2 ). Don't recognize? Write W(E) = w(B(E)): i s just Schmeidler's Choquet expected utility! Can always write B(E) = w –1 (W(E)). For binary gambles: Pfanzagl 1959; Luce ('00 Chapter 3); Ghirardato & Marinacci ('01, "biseparable").

18. 18 U´(1–r 2 ) U´(1 – (1–r) 2 ) ( 1–w(B(E)) ) w(B(E)) + w(B(E)) r E = U´(1–r 2 ) U´(1 – (1–r) 2 ) (1–r) r + r B(E) = Reversed (explicit) expression: w –1 ( )

19. How bet on Brasil now? [Ambiguity, nonEUA]. r EV = 0.75. r EU = 0.69. r nonEU = 0.61 (under plausible assumptions). Similarly, r nonEUA = 0.52. r's are close to always saying fifty-fifty. "Belief" component B(E) = w –1 (W) = 0.62. 19 go to p. 8, with figure of R(p)

20. B(E): ambiguity attitude  /=/  beliefs?? Before entering that debate, first: How measure B(E)? Our contribution: through proper scoring rules with "risk correction." 20

21. 21 We reconsider reversed explicit expressions: U´(1–r 2 ) U´(1 – (1–r) 2 ) (1–r) r + r p = w –1 ( ) U´(1–r 2 ) U´(1 – (1–r) 2 ) (1–r) r + r B(E) = w –1 ( ) Corollary. p = B(E) if related to the same r!! Part II. Deriving Theoretical Models from Empirical Observations of r

22. 22 If - for event E, subject has r E = r; - for probability p, subject has R(p) = r; then B(E) = p. Need not measure w, W, U! We simply measure the R(p) curves, and use their inverses: B(E) = R –1 (r E ) follows. Applying R –1 is called risk correction. Directly implementable empirically. We did so in an experiment, and found plausible results.

23. 23 Our proposal takes the best of several worlds! Need not measure U,W, and w. Get "canonical probability" without measuring indifferences (BDM …; Holt 2006). Calibration without needing many repeated observations. Do all that with no more than simple proper- scoring-rule questions.

24. 24 We bring the insights of modern nonEU to proper scoring rules, making them empirically more realistic. (SEV in 2006 is not credible …) We bring the insights of proper scoring rules to modern nonEU, making B very easy to measure and analyze.

25. Part III. Experimental Test of Our Correction Method 25

26. Method Participants. N = 93 students. Procedure. Computarized in lab. Groups of 15/16 each. 4 practice questions. 26

27. 27 Stimuli 1. First we did proper scoring rule for unknown probabilities. 72 in total. For each stock two small intervals, and, third, their union. Thus, we test for additivity.

28. 28 Stimuli 2. Known probabilities: Two 10-sided dies thrown. Yield random nr. between 01 and 100. Event E: nr.  75 (etc.). Done for all probabilities j/20. Motivating subjects. Real incentives. Two treatments. 1. All-pay. Points paid for all questions. 6 points = €1. Average earning €15.05. 2. One-pay (random-lottery system). One question, randomly selected afterwards, played for real. 1 point = €20. Average earning: €15.30.

29. 29 Results

30. 30 Average correction curves.

31. 31 0.8 0.9 1 -2.0-1.5-0.50.00.51.01.5 ρ F(ρ) treatment one treatment all Individual corrections

32. 32

33. Summary and Conclusion Modern decision theories: proper scoring rules are heavily biased. We correct for those biases, with benefits for proper-scoring rule community and for nonEU community. Experiment: correction improves quality; reduces deviations from ("rational"?) Bayesian beliefs. Do not remove all deviations from Bayesian beliefs. Beliefs seem to be genuinely nonadditive/nonBayesian/sensitive-to- ambiguity. 33