Aiming for the Impossible: Preference Measurements that Upset neither Economists nor Psychologists Peter P. Wakker (& Cathleen Johnson, Aurélien Baillon, Han Bleichrodt, Zhihua Li, & Dennie van Dolder) We introduce the prior incentive system (Prince): An improved method for measuring individual preferences. Hoped to be both economically acceptable (real incentives) & psychologically acceptable (understandable to subjects).

2 Prince combines tractability of matching with transparency/validity of binary choice. Thus it revives matching. And, thus, it resolves - C lassical preference reversals; - reliance on isolation for random incentive system; - the income effect; - strategic behavior in adaptive/chained experiments.

Survey: 1.History; 2.Definition of Prince; 3.Implementation 1: a small experiment (WTA); 4.Implementation 1: a big experiment (WTA); 5.Implementation 3: a chained/adaptive experiment, measuring utility; 6.Conclusion. 3

1. History 4

 s have challenged economics throughout history (Edwards 1954). Biggest challenge: Preference reversals (Lichtenstein & Slovic’71; Lindman’71; Birnbaum, Coffey, Mellers, & Weiss 1992). Kahneman & Tversky’s (1979) prospect theory challenged a.o. through many framing effects: loss aversion/endowment effect; WTA-WTP discrepancy; inconsistent probabilities; inconsistent utilities. Is birth of behavioral economics. 5 Dutch notation for “psychologists”

Reactions came from experimental economists: Economic experiments should be done “better:” No deception; real incentives; learning of stimuli. Led to big improvements. But many anomalies remained: preference reversals (Grether & Plott 1979), nonadditive probabilities, nonexpected utility, loss aversion, WTP-WTA discrepancy. Sometimes controversial (Plott & Zeiler 2005). 6

Also, there still are problems with real incentives. Are not yet very understandable to subjects. First we consider matching questions. Those ask directly for indifference points. Example: 7 Willingness to accept (WTA) of Which cash amount is equally preferable as this mug?

Economists usually incentivize matching by Becker-DeGroot-Marschak (BDM) mechanism: 1. Randomly a prize is selected. 2. If prize > WTA, subject gets prize. 3. If prize  WTA, subject gets mug. Is incentive compatible. (Right !?)  s : Is hard to understand for subjects. Random prize is vague future concept to them. Conditioning on it is even vaguer. Hard to understand! Other problem: bargaining instincts distort. 8

Nowadays: choice lists are more popular. Example: 9 1 Subject is given a list of prices; 2 For each line, the subject chooses the preferred item.

10 Then WTA = €4. For example, Random incentive system (RIS): At end of experiment, 1 choice is randomly selected, and implemented for real. If the choice in row 8 is the one selected for real, then the subject gets €7.50. RIS is incentive compatible (right!?).

Drawbacks of choice lists: Crude; biases (central tendency); Drawbacks of RIS: requires the questionable isolation assumption to avoid perception of meta-lottery. Yet RIS is by far most used today, e.g. because it avoids income effects. P.s.: RIS was first proposed by Savage (1954, p. 29, crediting Wallis). Prince: improves RIS. Enhances isolation. 11

2. Definition of Prince 12

Definition of Prince: 1)For each subject, one Real Choice Situation is selected prior to the experiment. Denoted RCS. 2)The RCS is inserted in a sealed envelope, given to the subject. Is called prior envelope. 3)The prior envelope describes the whole RCS. 4)Questions to subjects never refer to abstract hypothetical choice situations, but always to the RCS in the prior envelope held in hand. 5)Questions to subjects never refer to abstract hypothetical choice acts, but always to the selection from the prior envelope by the experimenter at the end. Subjects give instructions to the experimenter about that selection. 6)& 7) on adaptive experiments: discussed later. 13

3. Implementation 1: a small experiment (WTA) 14

We only measured one WTA for one mug. N=30 subjects. €10 showup-fee, + mug or money. Prior endowment: They immediately received the mug (and later could sell). 50 sealed numbered envelopes. Each subject randomly took one, their prior envelope. They received explanations and then the following questionnaire. 15

16 Option 1: Keep your mug Option 2: Give up your mug for €x In each of the 50 envelopes, one option is to keep your mug, and the other option is to give up your mug for a money amount. The note in each envelope is as follows. The money amount x varies between €0 and €10 in different envelopes. Five of the envelopes contain a randomly generated amount between €0 and €1, five envelopes contain a randomly generated amount between €1 and €2, five contain a randomly generated amount between €2 and €3, and so on, with finally five envelopes containing a randomly generated amount between €9 and €10. Thus the amount in your envelope can be any amount, in cents, between €0 and €10. Please give us instructions, for each possible envelope that your envelope may be, whether we should let you keep your mug, or we should give you that money amount in exchange for your mug. Do so by specifying a threshold (in cents). My threshold is € ……,…… If the money amount x in my envelope is equal to or above the threshold, then give me that money amount in exchange for my mug. If the money amount x in my envelope is below the threshold then let me keep my mug.

4. Implementation 2: a big experiment 17

N=80 subjects. €10 showup-fee, + mug or chocolate or money. 18

We measured: 1a. WTA of mug; matching & no prior endowment; 1b. WTA of mug; choice list & no prior endowment; 2. WTA of chocolate; matching & no prior endowment; 3. WTA of chocolate; choice list & no prior endowment; 4. Choice between P- and $-lottery; 5. Cash equivalent of P-lottery; 6. Cash equivalent of $-lottery; 7. Matching probability of event Matching probability of event Matching probability of union of events 1&2. 19

Each of the above 10 questions corresponds with one “type” of envelope. Of each type there are 10 envelopes, totalling 100 envelopes. There were two groups of students, one answering questions 1a, 2, …, 9, and the other answering questions 1b, 2, …, 9. For each group there were 90 envelopes, and each student chose one randomly, being their prior envelope and containing their RCS (real choice situation). During experiment, subjects filled out a questionnaire, providing instructions to the experimenter. At the end, the experimenter opened their envelope, and then implemented the relevant instructions of choice from the questionnaire. 20

Now we look at questions and results in detail. Question 1a: WTA for mug through matching as you saw before in small experiment. Only difference: now no prior endowment. 21

Question 1b: Again, WTA for mug; again no prior endowment. Difference: now no matching, but choice list. Here is the corresponding question in the questionnaire: 22

23 Type  Option 1: The mug Option 2: €x In each of the 10 envelopes of type , one option is the mug you just saw, and the other option is a money amount. The money amount x varies between €0.50 and €9.50 in different envelopes (see below). The note in each envelope of type  is as follows. In the following list, each line describes the content of one envelope of type δ. On each line, cross out the square before the option that we should give you if that line describes the two options in your envelope.

Results for WTA for mug (all between-subjects): Questions 1 (matching with prior endowment), 1a (matching without prior endowment), & 1b (choice list without prior endowment) 24 GroupsNStd. Dev.Mean Question 1 Question 1a Question 1b prior endow- ment or not Questionstreatmentdf mean difference 1 – 1a 1a – 1b 1.81 – t p matching vs. choice –

Conclusion: Endowment effect is confirmed by Prince. Choice-matching discrepancy is disconfirmed: it is resolved (further disconfirmed by Questions 2 & 3, within-subjects with chocolate, N=80.) Our claim: Endowment effect is a genuine property of preference. The choice-matching discrepancy is not a genuine property of preference, but is a bias in measurement ( Tversky, Slovic, & Kahneman 1990 ). 25

Testing preference reversal 26

27 Question 4 (Choice between P- and $-bet) concerned the following question in the questionnaire: Type  Option 1: €4 with a probability of 97% Option 2: €16 with a probability of 31% In the 10 envelopes of type , both options are probability- contingent money. The note in each envelope of type  is as follows. Please give us instructions whether we should give you option 1 or option 2 if your envelope is of type . Give me option …..

28 Question 5 (Matching; CE of P-bet) was as follows: Type Option 1: €4 with a probability of 97% Option 2: €x In the 10 envelopes of type, one option is probability- contingent money, and the other option is a sure money amount. The note in each envelope of type is as follows. The money amount x varies between the envelopes. Please give us instructions for each possible envelope of type that your envelope may be, whether we should give you the sure money amount or the probability-contingent money. Do so by specifying a threshold (in cents). My threshold is € ……,…… If the money amount x in my envelope is equal to or above the threshold, then give me that money amount. If the money amount x in my envelope is below the threshold then give me the probability-contingent money.

29 Question 6 (Matching; CE of $-bet) Type  Option 1: €16 with a probability of 31% Option 2: €x In the 10 envelopes of type , one option is probability- contingent money, and the other option is a sure money amount. The note in each envelope of type  is as follows. The money amount x varies between the envelopes. Please give us instructions for each possible envelope of type that your envelope may be, whether we should give you the sure money amount or the probability-contingent money. Do so by specifying a threshold (in cents). My threshold is € ……,…… If the money amount x in my envelope is equal to or above the threshold, then give me that money amount. If the money amount x in my envelope is below the threshold then give me the probability-contingent money.

Results: Normal preference reversals: 11% Opposite preference reversals: 7% No difference (p = 0.55). Such numbers can be expected under usual choice errors. Conclusion: Prince resolves preference reversals. Once again, choice and matching have been reconciled. Traditional discrepancies do not reflect genuine preferences, but are due to biased measurements. 30

Side result: Matching with no range provided gave more anomalies than matching with. Providing context can reduce biases! See: Birnbaum, Michael H. (1992) “Should Contextual Effects in Human Judgment Be Avoided?,” Book Review of: E. Christopher Poulton (1989) “Bias in Quantifying Judgments,” Erlbaum, Hillsdale NJ; Contemporary Psychology 37, 21–23. 31

Questions 7-9 measured subjective probabilities, replicating part of Baillon & Bleichrodt (2013) but with Prince & matching rather than BB’s choice lists. Found the same results, in particular, the same violations of additivity. Confirms again validity of matching under Prince. 32

Nice further result: in 119 choice lists only 1 irrational preference switch. Conclusions from big experiment: Prince works well. Revives matching. Sheds better light on which preference challenges are real in preference, and which are just biases in measurement 33

5. Implementation 3: a chained/adaptive experiment, measuring utility 34

N=25 subjects hypothetical. N=55 subjects with real incentives. Stimuli were, roughly, as follows. 35

36 €1?€1? ½ ½ ½ ½ ~ €8€8 €1€1 €10 €2?€2? ½ ½ ½ ½ €8€8 €1€1 €1€1 €3?€3? ½ ½ ½ ½ €8€8 €1€1 €2€2 €4?€4? ½ ½ ½ ½ €8€8 €1€1 €3€3 ~ ~ ~ For chaining, have to solve: - indeterminacy overlap problem; - exclusion overlap problem. Solving those are the other two defining properties of Prince.

Thus Prince has made the tradeoff method for measuring utility incentive compatible. 37

6. Final conclusion 38

39 Biggest con of Prince: not for lazy experimenters. Pros: Prince combines tractability of matching with transparency/validity of binary choice. Thus it revives matching. And, thus, it resolves - C lassical preference reversals; - reliance on isolation for random incentive system; - the income effect; -strategic behavior in adaptive/chained experiments.

The end 40