Talk, but speak clear Improving the rigour in agent-based social simulation Matteo Richiardi Università Politecnica delle Marche and Collegio Carlo Alberto – LABORatorio R. Revelli ESSA 2007, September 10-15, 2007, Toulouse

2 Outline  What is Agent-based Computational Economics (ACE)  Main features  When to use an agent-based model  The dual problem of micro-macro relation  A dynamic system representation of agent-based models  The notion of equilibrium in agent-based models  Analysis of the models  Estimation / calibration of the parameters  Description of the models

3 References Leombruni, Richiardi, Saam, Sonnessa (2006), “A Common Protocol for Agent Based Social Simulation”, JASSS, 9(1) Richiardi, “Agent-based Computational Economics. A Short Introduction”, mimeo …. downloadable from

4 What is ACE Agent-based computational models are models in which: (i) a multitude of objects interact with each other and with the environment (ii) the objects are autonomous, i.e. there is no central, or “top down” control over their behavior; (iii) the outcome of their interaction is numerically computed. “ACE is the computational study of economic processes modeled as dynamic systems of interacting agents.” (L. Tesfatsion)

5 A methodological remark Methodological individualism: explaining society as the aggregation of decisions by individuals (Austrian School of Economics).  Reductionism: the whole is nothing but the sum of its parts. ACE Holism: the proprierties of a system cannot be deduced by the properties of its components alone. Indeed, the system as the whole determines how the parts behave: the whole is more than the sum of its parts (Aristotle, Metaphysics).  Organicism (Ritter, 1919): the organization, not the composition, of organisms is what counts.

6 Main features of ACE models  Heterogeneity  Explicit space  Local interaction  Cognitive foundations:  Bounded rationality  Limited / asymmetric information  Non equilibrium dynamics

7 When to use ACE  To get a quick intuition of the dynamics a given system is able to produce (scrap paper use)  To thoroughly investigate models that are not susceptible of a more traditional analysis, or are susceptible of a more traditional analysis only at too a high cost: 1.Numerical computation of analytical models 2.Robustness analysis of analytical models 3.Stand-alone simulation models

8 When to use ACE  Analytical models: little interaction, little dynamics, sophysticated behaviors (often, not always)  Simple behaviors can produce complex patterns (e.g. Langton’s ants) [GO]GO  Simple choices can lead to complex and diversified behaviors (e.g. El Farol bar problem, W. Brian Arthur, “Inductive reasoning and bounded rationality”, American Economic Review, n. 84, p. 406, 1994)

9 The dual problem The dual problem of the micro-macro relation: a)FROM MICRO TO MACRO: Find the aggregate implications of given individual behaviors b)FROM MACRO TO MICRO: Find the conditions at the micro level that give raise to some observed macro phenomena - ACE as generative social science: “If you didn’t grow it, you didn’t explain it” (Epstein, 1999)

10 Our real problem a growing community of modellers a disappointing publication rate audience is more often interested in the methodology than in the topic  autoreferentiality ABM papers seem to be confined only to specialized journals like the Journal of Economic Dynamics and Control, the Journal of Artificial Societies and Social Simulation, Computational Economics, the Journal of Economic Interaction and Coordination, Advances in Complex Systems and few others

11 The real problem How can the rate of acceptance of papers with agent-based methodology in the top journals be increased? Talk to the mainstream !!! Break the auto-referentiality circuit !!!

12 Traditional analytical modelling Traditional analytical modelling relies on a very well established, although implicit, methodological protocol, both with respect to: the way models are presented and to the kind of analyses that are performed.

13 Traditional analytical modelling E.g., in most papers: detailed reference to the literature; the model often adopts an existing framework and extends, or departs from, well-known models only in limited respects; this allows a concise description, and saves more space for the results, which are finally confronted with empirical data; when estimation is involved measures of validity and reliability of the estimates are always presented, in a very standardized way.

14 Agent based modelling E.g., in most papers: very limited reference to the literature; the model often departs radically from the existing literature and adopts or extends no existing, well-known framework. This allows only a superficial description of the model, and even leaves no space for the results, which are only rarely confronted with empirical data. When estimation is involved few measures of validity and reliability of the estimates are presented, often in a very unstandardized way.

15 ACE is mathematics  Computer simulation as a third symbol system, aside verbal description and mathematics (Ostrom, 1988)  “Simulation is neither good nor bad mathematics, but no mathematics at all” (Gilbert et al., 1999)  “Any theory that can be expressed in either of the first two symbol systems can also be expressed in the third symbol system.”  “There might be verbal theories which cannot be adequately expressed in the second symbol system of mathematics, but can be in the third”

16 ACE models as recursive systems behavioral rules state variables structural parameters

17 I-O transformation function initial conditions ACE models as recursive systems structural parameters

18 2 notions of equilibrium:  AT A MICRO LEVEL, when individual strategies are constant  AT A MACRO LEVEL, when some relevant aggregate statistics of the system are stationary: Equilibrium

19  g is unknown  get an inductive evidence about g, by performing multiple (thousands, millions) runs and recording inputs and outputs of each run Interpretation of results

20  sensitivity analysis around some default values of the parameters and initial conditions: keep all parameters and initial conditions fixed and change only one at a time Local analysis of g  equivalent to exploring the partial derivatives of g  choice of the default values is an issue

21  Let all parameters change between the different simulation runs  Get an estimate of g (metamodel, response surface, compact model, emulator, ecc.) Global analysis of g

22 Thank you for your attention! Matteo Richiardi

23 “the distinction drawn between calibrating and estimating the parameters of a model is artificial at best. Moreover, the justification for what is called “calibration” is vague and confusing. In a profession that is already too segmented, the construction of such artificial distinctions is counterproductive” (Hansen and Heckman, 1996) Choice of structural parameters:  some parameters have real counterparts and their value is known: no need for calibration/estimation  otherwise: choice of the values that make the artificial data produced via simulation as close as possible to the real data Calibration / estimation

24  Gouriereux and Monfort, 1997  Mariano et al, 2000  Train, 2003  define some target variables (e.g. moments)  compute them both in the artificial and in the real data  keep changing the parameters to estimate (the  ) until the distance between the values of the target variables in the artificial data and in the real data is minimized  Indirect Inference: the targets are the estimates of an auxiliary model Simulation based estimation

25  code availability  the sequence of events in the simulation must be carefully described:  pseudocode  time-sequence diagrams Replicability

26 The pseudo-code … Source: Neugart (2006)

27 Time-sequence diagrams Source: Richiardi (2007)