Linking multi-agent simulation to experiments in economy Re-implementing John Duffy’s model of speculative learning agents

Experimental economics [Hayek, "The use of knowledge in Society", 1945 ] Production of a setting where individuals face an economic situation that is similar to real ones – controlled and easy to reproduce. Limited possibilities of action Control of motivation of agents / interaction mode / initial information Observation of behaviours of real humans Interpretation of results Comparison of behaviours with the theoretical setting Drawing hypothesis explaining the differences

Simulations to design experiments " Usual protocol: humans interact only through computers to get an absolute control of communication. " Choice of parameters to design experiments by running tests with only computers " Test of behavioural hypothesis with learning IA in place of humans following the experiments to reproduce the results " Mixing humans and AI in the same environment Duffy, J, 2001, Learning to Speculate: Experiments with Articial and Real Agents, JEDC, 25, pp

The Kiyotaki and Wright model: how to induce speculative behaviours? The aim of KW is to find an economy in which the production and consumption rules would Force the agents to exchange Have them use a good as an exchange value (enable them to store a good that has no direct value to them) Discretisation of action – best response Knowing possession of the whole population Kiyotaki N., Wright R., 1989, On money as a medium of exchange, Journal of Political Economy 97,

Situation of the agents Production > exchange > consumption Agent i consumes good i Agent i produces good i +1 Consumption > gain in utility = u. Agent produces iff it has just consumed 3 types of agents : 1, 2, 3 3 goods : 1, 2, 3

A time-step At the end of a time-step: an agent 1 possesses only good 2 or 3 Only stores ONE unit of non-divisible good Storage costs 0< c1 <c2 < c3 < u “discount factor”  : 0<  < 1 – end of the exchange rounds storage

A time-step A B , 3 2 C exchange Bilateral decentralised negotiation - choose to exchange or not. No « simple exchange » (A is forbidden) > at least three agents have been involved when everyone is satisfied

Fundamental and speculative behaviours Expected gain for agent i (imagine that it will get good i at time t + 1 by exchanging against the good it possesses now): Possesses good (i+1): γ i +1 = - c i +1 +  u. Possesses good (i+2): γ i +2 = - c i +2 +  u. Fundamental = when facing an exchange accepts to store the good that gives the best expected gain. Speculative = when facing an exchange accepts to store the good that gives the worst expected gain if there is a higher chance to perform the exchange to get i at the next time-step. Fundamental for agents 1 and 3 is to refuse to exchange i+1 for i+2 Fundamental for agents 2 is to accept to exchange i+1 for i+2 Speculative: depends on actual probabilities of exchange

Fundamental and speculative behaviours Notation: Proportion of agents i possessing the good they produce: pi, and possessing the other good: (1-pi) (s1,s2,s3) the set of strategies by agents 1, 2 and 3, Si = 0 if agents refuses to get i+2 when facing the opportunity Si = 1 if agents accepts to get i+2 when facing the opportunity Solution by KW: the agent decides if it will speculate or not by anticipating its ability to exchange at the next time-step. >> Strategic equilibrium depending on u, c1, c2, c3,  and (p1,p2,p3) - either (0,1,0) ou (1,1,0) Issue for Duffy: agents have no complete knowledge and learn anticipated gain through experience experience is individual – not all agents of the same type choose the same

Reproduction of the speculative setting Several production of this model put in a distributed setting (multi- agent type models) Marimon, R., E.R. McGrattan and T.J. Sargent, 1990, Money as a medium of exchange in an economy with artificially intelligent agents, Journal of Economic Dynamics and Control 14, Basci, E., 1999, Learning by imitation, Journal of Economic Dynamics and Control 23, Several production of this model put in an experimental setting Duffy, and J. Ochs, 1999a, Emergence of money as a medium of exchange: An experimental study, American Economic Review 89, Duffy, J. and J. Ochs, 1999b, Fiat money as a medium of exchange: Experimental evidence, working paper, University of Pittsburgh.

Duffy: getting close to experimental setting for comparison and extension Limited number of agents 16 or 24 Short simulation: only repetition of 10 games in a row Chooses setting where (1,1,0) is the solution profile agents of type 1 “should” play speculative agents of type 2 and 3 “should” play fundamental Same setting is used for artificial agents and humans, Manipulation of the setting to influence learning change proportion for different probabilities of meeting automate some of the behaviours >> once the simulation shows how interesting the setting is, reproduce it with real humans / mix artificial and real agents

Learning algorithm for the agents A simulation is a set of 10 games with probability β to stop at each time-step, agents cannot exchange when it stops and they start with their production good again when an agent A meets another agent (B) – if B proposes the good A owns: no exchange – B has good i: A proposes the exchange – otherwise depends on memory ν i+1 = Σ (I i+1 ) * γ i +1 - Σ (I i+2 ) * γ i +2 ν i+2 = Σ (I i+2 ) * γ i +2 - Σ (I i+1 ) * γ i +1 Where “I i+1 = 1 for a time-step where i succeeded in getting i +1 with i at start and I i+1 = –1 in the opposite case” x = ν i+1 - ν i+2 And probability to refuse: exp x / (1 + exp x)

Reproducing the learning algorithm in simulations Homogeneous Simulations Ambiguity to interpret the algorithm: “I i+1 = 1 for a time-step where i succeeded in getting i with i+1 at start and I i+1 = 0 in the opposite case” Algo1: any time the agents has possessed the good Algo2: any time he could have exchanged and it was refused (=if it had had the other good, the exchange would have been accepted) Algo3: Algo1 but doesn’t’ exchange back to get its production good Constrained simulations Only agents of type 1 have to learn and the others are automated

Experimental data (8-8-8) Agents type 1 offers 2 for 3 Agents type 2 offers 3 for 1 Agents type 3 offers 1 for 2 first half of the experiment second half of the experiment first half of the experiment second half of the experiment first half of the experiment second half of the experiment R R R R R Average on R1- R

Agents type 1Agents type 2Agents type 3 Time-step for the first half Time-step for the second half Time-step for the first half Time-step for the second half Time-step for the first half Time-step for the second half Average on A1-A50,190,320,770,990,220,04 SIM 1 Rational agents Average speculation rate MSD SIM 2 var- rational agents Average speculation rate MSD SIM 3 Stable agents Average speculation rate MSD

Agents type 1Agents type 2Agents type 3 Time-step for the first half Time-step for the second half Time-step for the first half Time-step for the second half Time-step for the first half Time-step for the second half Duffy: Average on 5 sessions Average speculation rate SIM 1’ Rational agents Average speculation rate MSD SIM 2 ‘ var- rational agents Average speculation rate MSD SIM 3 ‘ Stable agents Average speculation rate MSD

Agents of type 1 – algo1

Agents of type 1 – algo2

Agents of type 1 – algo3

Reasons why comparison hasn’t work (but will) " My mistake in reproducing the setting " Dependence to random generator " Not enough meetings or agents to establish comparison with experiments " My simulations: same results " Not enough meetings, possibility to exchange with such a simple algorithm " Comparison at the macro level is not enough > use the actions one-by-one (need of the whole set of experimental data).