On the Autonomy of Experiments in Economics Martin Jones School of Social Sciences, University of Dundee

Autonomous Research Programmes Will argue that experiments can be used to independently investigate phenomena. Will argue that experiments can be used to independently investigate phenomena. There is no requirement to test a particular theory. There is no requirement to test a particular theory. There is no requirement to externally validate using field studies. There is no requirement to externally validate using field studies.

Preliminaries Aim of experimentation is to discover causal relationships. Aim of experimentation is to discover causal relationships. Use of controls to isolate causes. Use of controls to isolate causes. Causation not explicit in economics but implicit in e.g. consumer theory, rational choice theory etc. Causation not explicit in economics but implicit in e.g. consumer theory, rational choice theory etc. Causes in experiments often linked to subject attitudes. Causes in experiments often linked to subject attitudes.

Taxonomy of Experiments Theory testing experiments; e.g. tests of EU, Cumulative Prospect Theory etc. Theory testing experiments; e.g. tests of EU, Cumulative Prospect Theory etc. Domesticated System experiments (Harré 2003) e.g. market based tests of “Winner’s Curse” Domesticated System experiments (Harré 2003) e.g. market based tests of “Winner’s Curse” Causal Control Experiments (CCEs) e.g. event- splitting effects (Starmer 2000) Causal Control Experiments (CCEs) e.g. event- splitting effects (Starmer 2000) Note: latter two not devoid of theoretical concerns- simply not theory testing. Note: latter two not devoid of theoretical concerns- simply not theory testing.

Instruments Instruments in an experiment enhance and constrain knowledge gained (c.f. Hacking 1983). Instruments in an experiment enhance and constrain knowledge gained (c.f. Hacking 1983). Not necessarily physical- can be experimental procedures (e.g. binary lottery mechanism) Not necessarily physical- can be experimental procedures (e.g. binary lottery mechanism) Problem with instruments- an experimental effect is produced jointly by behaviour and instruments. An effect may be an artefact of the instruments. Problem with instruments- an experimental effect is produced jointly by behaviour and instruments. An effect may be an artefact of the instruments.

Manipulation Reality of phenomenon ensured by manipulation. Reality of phenomenon ensured by manipulation. Biology- microscopes (Hacking 1983); Economics- equitable division in Ultimatum games (Camerer 2003). Biology- microscopes (Hacking 1983); Economics- equitable division in Ultimatum games (Camerer 2003). Manipulation involving different instruments and tests of instruments ensures independence of phenomenon. Manipulation involving different instruments and tests of instruments ensures independence of phenomenon. With multiple treatments and experiments one can ensure the reality and independence of phenomena- especially CCEs. With multiple treatments and experiments one can ensure the reality and independence of phenomena- especially CCEs.

External Validity Guala (2005) Experiments act as a screening device for theories. Guala (2005) Experiments act as a screening device for theories. External validity assured by Field Studies. External validity assured by Field Studies. This obviously works for “Domesticated System” type experiments. May work for theory- testing experiments This obviously works for “Domesticated System” type experiments. May work for theory- testing experiments

External Validity Guala’s framework does not work for CCEs Guala’s framework does not work for CCEs CCEs disrupt natural systems CCEs disrupt natural systems CCEs may produce effects that do not exist in the external world. CCEs may produce effects that do not exist in the external world. Therefore CCEs cannot be made externally valid through Field Studies. Therefore CCEs cannot be made externally valid through Field Studies.

How can CCEs be Externally Valid? CCEs are counterfactual but have many elements in common with external world (human subjects, money etc.). When causal mechanisms correspond a CCE is externally valid. When causal mechanisms correspond a CCE is externally valid. If additional causes postulated in external world then (in principle) can be tested in laboratory. If additional causes postulated in external world then (in principle) can be tested in laboratory. Example- money payments higher in external world? (or arbitrage etc.) Example- money payments higher in external world? (or arbitrage etc.)

How can CCEs be Externally Valid? Similarly if too many causes in lab. Similarly if too many causes in lab. Aim of CCEs not to reproduce phenomena but to find explanations. Does not have to correspond completely- otherwise counterfactual experiments not possible. Aim of CCEs not to reproduce phenomena but to find explanations. Does not have to correspond completely- otherwise counterfactual experiments not possible. Source of knowledge for additional causes could come from previous experiments, theoretical knowledge, previous field studies or a combination of these. Source of knowledge for additional causes could come from previous experiments, theoretical knowledge, previous field studies or a combination of these.

Conclusions CCEs are useful as autonomous research tools. Independent of both theory testing and necessity for field studies. CCEs are useful as autonomous research tools. Independent of both theory testing and necessity for field studies. CCEs useful as part of a programme of experiments which establish the reality, independence and external validity of the phenomenon being studied. CCEs useful as part of a programme of experiments which establish the reality, independence and external validity of the phenomenon being studied. Repetition, Pilot studies, practice runs essential- eliminate artefacts from experiment. Repetition, Pilot studies, practice runs essential- eliminate artefacts from experiment.

Conclusions Is this restricted to CCEs? Is this restricted to CCEs? No- same arguments apply to other experiments as well. All investigate causal connections. No- same arguments apply to other experiments as well. All investigate causal connections. Arguments for theory testing and field studies are additional arguments for specific types of experiment. Arguments for theory testing and field studies are additional arguments for specific types of experiment.