1. Experimental Economics: Short Course Universidad del Desarrollo Santiago, Chile December 17, 2009 Dr. Jonathan E. Alevy Department of Economics University of Alaska Anchorage afja@uaa.alaska.edu

2. Field Experiments Field experiments combine elements of lab experiments – Clean counterfactual – Randomization to treatment – Ability to collect auxiliary data (e.g. risk preferences) – Salient incentives (pay based on performance) And traditional field studies – Relevance to participants & natural setting “Home-grown” values Salient incentives

3. Field Experiments in the Empirical Toolkit AFE_____FFE______NFE Lab field experiments Uncontrolled Field data conventional lab experiment (Lab) – employs a standard subject pool of students, an abstract framing, and an imposed set of rules artefactual field experiment (AFE) – same as a conventional lab experiment but with a non-standard subject pool framed field experiment (FFE) – same as an artefactual field experiment but with field context in the commodity, task, information, stakes, time frame, etc. natural field experiment (NFE) – same as a framed field experiment but where the environment is the one that the subjects naturally undertake these tasks, such that the subjects do not know that they are in an experiment

4. Learning from artefactual experiments Subject pool effects – Relevance seen in lab as well. Students from different schools, different majors etc. Field Setting: – Selected samples Access to “high-level” decisionmakers – Corporate executives play a variant of the trust game (Costa Rica). » Fehr & List 2004 – Financial market professionals: information processing and cascades. » Alevy, Haigh & List, 2007 – Representative samples Risk & time preferences in Denmark Relevant for interpreting welfare effects of policy – Harrison et al. 2002, 2007

5. CEO’s, Trust, and Punishment Two person trust game: – Player 1 has 10 units of currency – First move: Can pass x  {0, 1, …,10} to player 2 – Amount passed is tripled. – Second move: Player 2 with 3x can pass any amount, y  {0, 1, …,3x}, back to player 1

6. CEOs, Trust, and Punishment Fehr & List (FL) variant – Player 1: Along with x, requests a specific amount, y*. – Can punish for deviations – Can waive right to punish in advance of transfer Waive punishment  FL Variant = original trust game Key findings – Waiving right to punishment is reciprocated  more returned – CEOs significantly more likely to waive right than baseline (students) and in general more trusting/trustworthy. Selection into CEO status  different profile than students Notes: – Empirical strategy: Random effects tobit model (censored data) – Protocol uses neutral language “conditional pay cut” not “punish” “Decision problem” not “trust game”

7. Further Afield: Framed Experiments Introduce context relevant to the subjects – E.g. vary the media in Holt/Laury risk elicitation. Cash used as a control in each case. Harrison, List & Towe, 2007 Econometrica – Coin collecters, treatment varies what is known about quality of coins. – Coins with low uncertainty similar to cash. Alevy, Cristi, & Melo, 2009 – Elicit risk preferences using lotteries for water. – Elicited attitudes using water treatment help explain bidding behavior in separate water auctions.

8. Chilean Water Market Experiments How to sell multiple heterogeneous items? – Estate sales – Sale of condos within a new complex Bidder’s choice or ‘Right-to-choose’ (RTC) auctions are a candidate institution – Rather than bid on specific commodity Compete for the right to select preferred item from available set – Idea: If little interest in any particular good Could competition across goods bolsters prices?

9. Chilean Water Market Experiments II Early evidence on RTC from uncontrolled field data – Ashenfelter Genesove 1992 AER – Nonexperimental evidence on condominium prices – RTC versus bilateral bargaining Price premium in RTC Experimental approach to understanding RTC – A cleaner counterfactual is possible – Good-by-good or sequential (SEQ) auction is benchmark – Observe impact of relevant changes in auction rules

10. RTC vs. Sequential Theory: a two-bidder, two-good model – Theory based on Burguet (2007) – With risk-aversion RTC auctions raise more revenue than SEQ – Intuition: Possibility that preferred good is chosen in early rounds  risk averse bid more in RTC Previous laboratory findings – Goeree et al. 2004 – Eliaz et al. 2008 – mixed evidence on theory: none control for risk attitudes

11. Field Experiments Designs Protocol – Salient payoffs, winner pays for good – Second price auction rule – Independently elicit risk attitudes & time preferences Chilean Auctions – 1 RTC Auction 20 bidders – 1 SEQ Auction 18 bidders – Subjects are farm owners and managers in Limari Valley study area

12. Fase1 del Ejemplo de la Subasta Resultado de la Subasta LicitadorPreciosProducto ID 1 25,0003Silla ID 2 27,0001Bicicleta ID 3 36,0003Silla ID 4 50,0002Computadora ID 5 8,0002Computadora ID 6 35,0001Bicicleta Resultado de la Subasta LicitadorPreciosProducto ID 4 50,0002Computadora ID 3 36,0003Silla ID 6 35,0001Bicicleta ID 2 27,0001Bicicleta ID 1 25,0003Silla ID 5 8,0002Computadora Resultado de la Subasta LicitadorPreciosProducto ID 4 50,0002Computadora ID 3 36,0003Silla ID 6 35,0001Bicicleta ID 2 27,0001Bicicleta ID 1 25,0003Silla ID 5 8,0002Computadora 1 2 3 Resultado de la Subasta LicitadorPreciosProducto ID 4 50,0002Computadora ID 3 36,0003Silla ID 6 35,0001Bicicleta ID 2 27,0001Bicicleta ID 1 25,0003Silla ID 5 8,0002Computadora Ganador de la Subasta Precio PagadoProducto ID 436,0002Computadora

13. Fase 2 del Ejemplo de la Subasta Resultado de la Subasta LicitadorPreciosProducto ID 1 25,0003Silla ID 2 27,0001Bicicleta ID 3 36,0003Silla ID 4 15,0002Silla ID 5 5,0002Silla ID 6 35,0001Bicicleta Ganador de la SubastaPreciosProducto ???? Resultado de la Subasta LicitadorPreciosProducto ID 3 36,0003Silla ID 6 35,0001Bicicleta ID 2 27,0001Bicicleta ID 1 25,0003Silla ID 4 15,0002Silla ID 5 5,0002Silla Resultado de la Subasta LicitadorPreciosProducto ID 3 36,0003Silla ID 6 35,0001Bicicleta ID 2 27,0001Bicicleta ID 1 25,0003Silla ID 4 15,0002Silla ID 5 5,0002Silla Ganador de la SubastaPreciosProducto 3 35,000 Silla

14. Experimental Results - Chile Auction Type1234 SEQRevenue50602025 Bid7.5413.407.988.30 RTCRevenue100 Bid43.5436.0427.4824.83 Revenues and Bids (Pesos per cubic meter) Bids in RTC greatly exceed SEQ

15. Tobit Estimates of bidding behavior VariableModel 1Model 2 RTC39.456***4.464 Pref 2-7.736-7.656 Good 3-19.657**-19.425** Good 4-31.478***-30.983*** RTC x Good 20.4030.322 RTC x Good 3-8.315-8.595 RTC x Good 4-6.039-6.465 Experience19.030** Risk-2.414 0.349 RTC x Risk6.170** 0.041 _cons13.35923.481** N164 Log likelihood582.62578.97

16. Ongoing opportunities for framed & natural field experiments Sports memorabilia shows, open air markets etc. Benefits: – Subjects who select into the market place in specific roles. – Variety of experience levels – Pay their own money Some useful protocols for valuation in this setting – nth price auction: number of goods sold is determined randomly Provides low value bidders with incentives for truthful revelation (List and Shogren, 1998) – BDM mechanism: one person at a time

17. Interesting Results from Sportscard Studies Many behavioral anomalies are mitigated when going from lab to field – Endowment effects eliminated among superexperienced traders List 2003, 2004 – Anchoring of values possible only for new market participants – Alevy, Landry and List, 2007 – Framing effects on valuation mitigated but still strong – Alevy, List, and Adamowicz, 2006

18. Gift Exchange in Sportscard Markets Artefactual, framed, and natural experiments combined – List 2006 JPE Behavioralist Meets the Market Examine whether social preferences influence outcomes in markets. Methodology: – 1. Artefactual field experiment abstract framing, subjects are sports card buyers and sellers (placed in their usual roles) Gift exchange exists: Buyers pay high prices, reciprocated by high quality.

19. Gift Exchange in Sportscard Markets Methodology 2. Framed Field Experiment – Replace abstract good with sportscards – Gift exchange result 3. Natural Field Experiment – Anonymously visit sellers who had participated in experiment – Offers of high prices not reciprocated by high quality Exception: Local dealers Explanation reputation not reciprocity

21. Characterizing the Entrepreneur Using Field Experiments (Elston et al. 2005) Use entrepreneurs (full- and part-time) vs control groups to test four alleged characteristics of entrepreneurs: Entrepreneurs are risk loving? – Test based on Holt and Laury (2002) Entrepreneurs have a joy of winning and/or suffer from (optimistic) judgmental errors? – Winners curse in common value auctions Holt and Sherman (1994) Entrepreneurs are overconfident? – Entry game Camerer and Lovallo (1999)

22. RISK AVERSION – based on Holt and Laury (2002). Figure3 shows raw responses which suggest: FT Entrepreneurs less risk averse than the PT Entrepreneurs or Others PT Entrepreneurs may be more risk averse than the nonentrepreneurs, particularly for the last three problems measuring extreme risk aversion.

23. ….. But figure3 fails to condition on observed differences in samples. Therefore, use Interval Regression Model (results in table 2): Full-Time Entrepreneurs (FT) have lower aversion to risk than Part-Time Entrepreneurs (PT). PT do not have significantly different risk attitudes than Non-Entrepreneurs (default category)

24. Judgmental Error and Joy of Winning Common Value Auctions Judgmental Error: The winners curse – Receive signal on value of good, value is common to all – Those receiving high value win but pay too much Joy of winning – Another reason for overbidding Protocol distinguishes between these reasons for overbidding – But yields no strong results. – No evidence of judgmental errors for any subject pool

25. Overconfidence and Entry Camerer and Lovallo (1999) - Protocol. Subjects in groups of 5 decide whether to enter a market or not. – Endowed with $10 – Pay $10 to enter – The most skilled among entrants receives $35 Subjects that chose not to enter would be allowed to keep the initial stake. No subject knew their score on the skill test prior to entry, so entry was determined in part by the subject’s belief about their skill level in relation to the others that might enter. Entry is also affected by risk attitudes Beliefs about number entering are also elicited

26. Conclusions Experiments an important new tool in economics Replication and extension leads to accumulation Field experiments an important link – to traditional empirical methods

27. Replication and Progress through Experiments Attitudes to risk are a primitive in economics – Gollier 2001 “It is quite surprising and disappointing to me that almost 40 years after the establishment of the concept … by Pratt and Arrow, our profession has not yet been able to attain a consensus about the measurement of risk aversion.” – Since 2001 enormous progress thanks to Holt, Laury, Harrison and others.

28. Psychology & Economics: Thriving Important collaborations/investigations of ideas from psychology – Ideas & methods generated by psychologists Some withstand economists’ scrutiny – Preference reversals, framing effects Some do not – Methods: Lack of salience & deception – Salience seems to weaken importance some results related to prospect theory.

29. Lab & Field Experiments Large debate on role of lab & field experiments Scrutiny in lab versus anonymity in field – Particularly relevant for social preference models “The basic strategy underlying laboratory experiments in the physical sciences and economics is similar, but the fact that humans are the object of study in the latter raises special questions about the ability to extrapolate experimental findings beyond the lab, questions that do not arise in the physical sciences.” – Levitt & List, 2007 Falk & Heckman, 2009 reply to Levitt & List – People are also under observation in field settings. – In laboratory settings the amount and type of scrutiny (observation by experimenter ) can be a treatment variable.

30. Summing up: Checklist to run an experiment… What is the question of interest? – Why is it interesting (theory, policymaker, fact finding)? Stakes sufficient? – Flat-payoff? Preferences induced? – What homegrown values remain uncontrolled? Appropriate subject pool? Theory cleanly tested (assumptions, etc.)? Within vs. between group design? Instructions appropriate? – Test these out with colleagues Need administrative approval? Pilot? Run!

31. Resources for experimentalists John List’s www.fieldexperiments.comwww.fieldexperiments.com Charles Holt’s http://people.virginia.edu/~cah2k/research.htmlhttp://people.virginia.edu/~cah2k/research.html Georgia State University – EconPort http://www.econport.org/http://www.econport.org/ Al Roth’s webpage http://kuznets.fas.harvard.edu/~aroth/alroth.html http://kuznets.fas.harvard.edu/~aroth/alroth.html Economics Science Association (ESA) www.economicscience.org www.economicscience.org Latin America Field Experiments Network (LAFEN) http://lafen.uniandes.edu.co/ http://lafen.uniandes.edu.co/ University of Alaska Anchorage – Experimental Economics Laboratory http://econlab.uaa.alaska.eduhttp://econlab.uaa.alaska.edu Many more resources …. Feel free to email me for further information and discussion at jalevy@cbpp.uaa.alaska.edujalevy@cbpp.uaa.alaska.edu

32. THANKS! Feel free to email me for further information and discussion at jalevy@cbpp.uaa.alaska.edujalevy@cbpp.uaa.alaska.edu