1. CONNECTING SIMULATIONS WITH HUMAN SUBJECT EXPeriments
Sera Linardi
Graduation School of Public and International Affairs
University of Pittsburgh

2. Models of human behavior
Agents Humans
What is this tutorial about?
Models of human behavior
Simulation
Actual human behavior
Experiments

3. Today’s goals
Discuss ways to integrate actual human behavior into agents based modeling.
Provide examples from economics.
Experience a human subjects experiment.
Think about how you can integrate human behavior into your modeling projects.

4. An overview of related fields
Economics: mathematical models with closed-form, analytic solutions often involving rational agents interacting with one another globally.
Goal: comparative static exercises, generalization (Nash (1950), Akerlof (1970)
Experimental economics (ExpEcon): actual humans interacting with each other in a controlled environment (can be global or local)
Goal: generate data (including dynamics of aggregate norms and comparative statics) from actual human behavior (Roth and Erev (1995, 1999))
Agent-based computational economics (ACE): evolving, heterogeneous, boundedly rational agents interact with one another locally
Goal: observe aggregate norms that emerge (Schelling(1978), Axelrod(1984))
ACE bottom up, inductive, heterogeneous, simulation
Epstein and Axtell (1996)) How agents adapt are less important than the aggregate outcomes that emerge.
bottom up, inductive, heterogeneous, put humans in a certain situation

5. An overview of related fields
Natural allies! Bottom up, inductive, heterogeneity central
Economics: mathematical models with closed-form, analytic solutions often involving rational agents interacting with one another globally.
Goal: comparative static exercises, generalization (Nash (1950), Akerlof (1970)
Experimental economics (ExpEcon): actual humans interacting with each other in a controlled environment (can be global or local)
Goal: generate data (including dynamics of aggregate norms and comparative statics) from actual human behavior (Roth and Erev (1995, 1999))
Agent-based computational economics (ACE): evolving, heterogeneous, boundedly rational agents interact with one another locally
Goal: observe aggregate norms that emerge (Schelling(1978), Axelrod(1984))
ACE bottom up, inductive, heterogeneous, simulation
Epstein and Axtell (1996)) How agents adapt are less important than the aggregate outcomes that emerge.
bottom up, inductive, heterogeneous, put humans in a certain situation

6. Three approaches 1. Humans -> Agents
Use ABM to explore hypothesis from extend human subject experiments.
Gode and Sunder (1993, 2004), Crawford (1991, 1995), Roth and Erev (1995, 1999), Camerer and Ho (1999a, 1999b)

7. Three approaches 2. Agents -> Humans
Use human subject experiments to “horse race” predictions of ABM models
Arifovic, McKelvey, Pevnitskaya (2016), Linardi (2017)

8. Three approaches 3. Humans interacting with agents.
Have agents play with human in experiments to explore how behavior change as markets scale up.
Prisoner’s dilemma (Roth and Murnighan, 1978), school choice (Chen et al, 2017)

9. Example 1: humans -> agents
Setting:
Economics = the study of behavior under constraints (goods to sell, money to buy)
Many institutions supposed to work as predicted (converge to equilibrium prices) assuming rational agents. But rationality may not be realistic / necessary.
Double auctions is a multilateral process that is used to organize some of the most common economic institution (stock, commodity, currency and other markets)
Buyers enter bids and sellers enter asks. When bid and ask cross a transaction occurs (say at average price, or the price that is the earlier of the two).
Let’s set one up!
Is double auction theoretically supposed to converge to the equilibrium?

10. A double auction experiment
Setting up a human subject experiment (Anderson and Holt, 1997, Smith, 1962):
Recruitment: 9 sellers and 9 buyers. each buying/selling a single item.
Induce valuation:
Buyers get resale value of the item – you want to pay less than this.
Sellers get cost for selling the item – you want to charge more than this
Introduce rules of the game:
Buyers you call out bids “Buyer 4 want to buy at $X”. ($X < resale value)
Sellers you can out asks “Seller 3 want to sell for $Y”. ($Y > cost of selling)
Once there’s an overlap: I will call out “Seller 3 sold to Buyer 4 at $ (X+Y)/2”. The buyer and seller that matched is then out of the market, outstanding bids and asks are wiped clean.
After the experiment, actually pay your subjects according to the induced value and game rules.

11. Double auction experiment
Asks (Seller)
Bids (Buyer)
Seller #
Buyer #
Transacted
prices

12. Theoretical Prediction
No.
Buyer valuation
Seller
cost 1 10 2 3 8 4 7 5 6 9

13. Theoretical Prediction vs Actual Behavior
Transacted
prices

14. Gode and Sunder (JPE, 1993)
Predicted
equilibrium prices

15. Gode and Sunder (JPE, 1993) Predicted equilibrium prices
Zero intelligence: no memory, bids & asks drawn randomly
Predicted
equilibrium prices

16. Gode and Sunder (JPE, 1993) Predicted equilibrium prices
Random draw: bids < resale value + asks > cost of selling
Predicted
equilibrium prices

17. Findings:
Human subject experiment = prices do converge to equilibrium prices
Agent-based model = rationality is not required for price convergence– in fact, this can be achieved with zero intelligence with well specified market trading rules.

18. Example 2: agents -> humans
Turns out convergence doesn’t always happen. E.g.. when valuation has to be computed and when market is thin (not many buyers and sellers).
Convergence is “hard” and actual prices stay far from equilibrium prices.
What behavior model can explain partial convergence?

19. Linardi (GEB, 2017)

20. Linardi (GEB, 2017)

21. Example 3: agents and humans together
School choice problem: students indicating their preferences over schools. Schools have a quota. Schools have priorities over students.
Two algorithms to match students to schools:
Boston mechanism: students submit preferences, each school consider only those who rank it first by priority, remaining school consider remaining students who ranked it kth.
Deferred acceptance (DA) mechanism: students apply to one school, each school tentatively accept by priority, remaining students reapply, schools re-rank applicants.
Large market properties unknown.

22. Findings
All humans
Humans + empirical robots
Properties of both matching algorithm appears stable to scaling.
Humans play similarly against empirical robots as they do with humans.
Viable way to cheaply scale up human subject experiments. .

23. Summary Can more closely integrate humans and agents:
1. Human experiments -> agents
2. Competing agent based models -> human experiments
3. Human interacting with agents in an experiment

24. Thank you! This tutorial borrows heavily from:
Leigh Tesfatsion, ACE Research Area: Mixed Experiments with Real and Computational Agents“ (webpage, accessed May 2018)
John Duffy, "Agent-Based Models and Human-Subject Experiments“, in Leigh Tesfatsion and Kenneth L. Judd (editors), Handbook of Computational Economics, Vol. 2: Agent- Based Computational Economics, North-Holland/Elsevier, Amsterdam, Spring 2006.
D. K. Gode and S. Sunder, "Allocative Efficiency of Markets with Zero Intelligence Traders: Market as a partial substitute for individual rationality“, Journal of Political Economy, Vol. 101, Number 1, 1993, pp
Linardi, Sera. "Accounting for noise in the microfoundations of information aggregation." Games and Economic Behavior 101 (2017):
Chen, Yan, et al. "Matching in the large: an experimental study." (2017).
Papers from Jasmina Arifovic