Daniel Ariosa Ecole Polytechnique Fédérale de Lausanne (EPFL) Institut de Physique de la Matière Complexe CH-1015 Lausanne, Switzerland and Hugo Fort Instituto de Física, Facultad de Ciencias Universidad de la República Montevideo, Uruguay Statistical Mechanics Applied to Social Sciences ____________________________________

Introduction n Estimating Utilities: Magnetic Systems and Games People Play n... and Adaptive Self-Interested Agents n Extended Estimator Approach for 2x2 Games and its Mapping to the Ising Hamiltonian

Outline n Self-organization into cooperative equilibrium states n The extended estimator formulation n Mapping the iterated game into an Ising model n Classifying Markovian Strategies n Stability of cooperation - Asynchronous random dynamics -Synchronous fully connected system n Thermodynamics of Ising mappable strategies n Discussion

Self-organization into cooperative equilibrium states n How populations of self interested agents cooperate, or manage in order to satisfy this goal globally or collectively ? n A few examples: - electrons in a superconductor - local magnetic moments in a ferromagnet - molecules that cooperate to form cells, cells that cooperate to form living creatures that in turn cooperate to form societies... n Different approaches: n Different approaches (paradigms and extremal principles): - Biology  Darwin’s evolution  fitness maximization - Economics  « Homo economicus »  profit maximization - Physics  statistical thermodynamics  free energy minimization

n Game theory and the Prisoner’s dilemma - The Prisoner’s dilemma models the social behavior of "selfish" individuals - 2x2 game in normal form: i) 2 players, each confronting 2 choices : to cooperate (C) or to defect (D) ii) each player makes his choice without knowing what the other will do iii) there is a 2  2 matrix specifying the payoffs of each player for the 4 possible outcomes: With the condition and

Elementary Markovian strategies and estimates ( aspiration levels ) for iterated games _____________________________________________ -The player updates its behavior (C or D) according to the outcome of the preceding run. - A simple updating rule (= strategy) consists in comparing the obtained utilities (  ) with a given estimate (   ) for the expected income. - Example: The “PAVLOV” strategy Behavior: “win-stay, lose-shift” Estimate: P <   < R Character: 

-more examples: “Retaliator” strategy: -Behavior: retaliates when the other player defects - Estimate: S <   < P - Character “Tit-for-Tat” strategy: -Behavior: cooperate on the first move an then reproduce what the other player did In the preceding move. - Estimate (conditional): S P * ; R > R * ; T < T * - Character: S* T* R* P* Retaliator

The extended estimator formulation - Behavior (state):  “SPIN”: -Estimate  Estimator payoff matrix(EPM): - Updating rule:player(i) FLIPS 

Mapping the iterated game into an Ising model ______________________________ Ising spin Story line:two-valued variable  Ising spin Metropolis updating rule for c  Metropolis algorithm algorithm (T=0) Flipping condition:

The Ising Hamiltonian: Energy density associated with the flip: The link:

Explicit form of the utilities: - when playing against a single player (j): - when playing against z nearest neighboring (nn) players: - In terms of the Ising variables: and

The mapping: 11 22

Classifying Markovian strategies

Stability of cooperation __________________________________ Average cooperation: Fraction c of agents in the C-state a) Steady state cooperation (asymptotic): c* b) Ground state cooperation (equilibrium at T=0): c eq Synchronous Dynamics: All agents simultaneously update their states in one round. Asynchronous Dynamics: The update is carried out by the subset of agents who just played.

A pair of players is randomly chosen for each round. Stability in Asynchronous Random Dynamics (ARD): Example for the PAVLOV ( )strategy: Only c* = 1/2 is a stable solution (for all but one cooperating agents, the system is rapidly driven away from c = 1.)

Asynchronous Random Dynamics (ARD):

Stability in the Synchronous Fully Connected System (SFC): A) The FES case The equilibrium sate strongly depends upon the initial configuration. Obtained utilities as a function of c : A stable configuration is reached when all players get a payoff  greater than the estimate  or, in other words, when  is lower than the cooperator’s utilities. Marginal stability is reached also when all players defect and  is lower than the defector’s utilities.

Phase diagram for the FES case

steady state Phase diagram for the steady state cooperation in IMS

Thermodynamics of Ising-mappable strategies Fully connected system(z=N-1): Free energy functional in terms of mapping parameters:

Mean field approximation: Partition function: Average magnetization:

ground state Phase diagram for the ground state cooperation in IMS

Summary All the relevant elementary Markovian strategies for the Prisoner’s dilemma have been formulated in terms of an extended (conditional) estimate. Another subset (AD ; TFT ; CON ; AC) has been mapped on an Ising Hamiltonian with 2 parameters. The remaining (7) strategies can also be formulated in terms of estimators involving more than two parameters (3, 4). A straightforward application of the thermodynamic approach of IMS consists of finding the ground state cooperation of complex systems of interacting agents, which is clearly different from the steady state of the iterated game. Finite temperature leads to “generous” strategies allowing more efficient equilibrium states in its “fitness” landscape. A subset of these strategies (FR ; RET ; PAV ; AMB ; ALT) admits a fixed (single) estimate.

Future work: The mapping on Hamiltonian systems can be exploited in many ways: - Replacing the two valued state [C or D] by a continuous variable (more rich systems as the XY model) - Evolution (Darwinian) and learning models: Instead of cooperation, consider the strategy (eg. the FES  *) as the site variable (order parameter?). Within this context, thermal fluctuations represent spontaneous “mutations” and the Boltzman energy factor will operate the selection. - Heterogeneous estimates varying from site to site (eg. spin glass model)