1. 851-0585-04L – Modeling and Simulating Social Systems with MATLAB
L – Modeling and Simulating Social Systems with MATLAB
Lecture 5 – Agent-based Modeling / Game Theory
Karsten Donnay and Stefano Balietti
Chair of Sociology, in particular of Modeling and Simulation
© ETH Zürich |
© ETH Zürich |

2. Schedule of the course Introduction to MATLAB
Schedule of the course
Introduction to MATLAB
20.02.
27.02.
05.03.
12.03.
19.03.
26.03.
02.04.
23.04.
30.04.
07.05.
14.05.
21.05.
28.05.
Research Plan is due NEXT WEEK
Introduction to social-science modeling and simulations
Working on projects (seminar thesis)
Handing in seminar thesis and giving a presentation

3. Goals of Lecture 5: students will
Consolidate their knowledge of cellular automata, through brief repetition of the main concepts introduced in the previous session.
Understand the important concept of Agent-based Modeling (ABM) as a powerful tool to capture complex system dynamics.
Get a brief introduction to Game Theory as a mean to formalize complex interaction dynamics such as human cooperation and coordination.
Get further information on the use of GIT and begin working their Research Projects discussing first ideas for implementations etc. with the lecturers.

4. Repetition
Cellular automata (CA) are abstract representations of interaction dynamics in discrete space and time
Reduce complexity of interaction to simple microscopic interaction rules; canonical way to formalize interaction range are Neighborhoods (Moore, van Neuman)
Approaches using cellular automata have a few natural limitations: only discrete time space and time; results depend on updating scheme (simultaneous vs. sequential); system dynamics can not always easily be reduced to microscopic interaction rules
Can be abstract test bed for complex systems: same results from CA and dynamical systems approach (Ex. 2)

5. Motivation for microscopic modeling
A number of real world processes may be reduced to simple local interaction rules (e.g. Swarm Intelligence)
Key for correctly capturing complex system dynamics with local individual interactions is that dynamic components of system are ‘atomizable’ in the context under consideration (e.g. pedestrians ‘behave’ like particles)
Represents reductionist (or mechanistic) point of view in contrast to a systemic (or holistic) approach; in line with the tradition that has fueled the success of natural science research in the past 200 years!
Game Theory provides great framework to formalize and reduce complex (strategic) interactions

6. Swarm Intelligence (1989)
Ant colonies, bird flocking, animal herding, bacterial growth, fish schooling…

7. Swarm Intelligence Key Concepts: decentralized control
interaction and learning
self-organization

8. Boids (Reynolds,1986)
A boid is a simulated individual particle or object moving about in a virtual space.

9. More information: http://www.red3d.com/cwr/boids/
Separation: steer to avoid crowding local flockmates
Alignment: steer towards the average heading of local flockmates
Cohesion: steer to move toward the average position of local flockmates

10. Game Theory
Mathematical framework for strategic interactions of individuals
Formalizes the notion of finding a ‘best strategy’ (keyword: Nash equilibrium) when facing a well-defined decision situation; conceptualization through ‘games’
Underlying assumption is that individuals optimize their ‘payoffs’ (or more precisely: ‘utility’) when faced with strategic decisions
NOTE: huge difference between game theoretic results for one-shot games vs. repeated interactions!

11. Game Theory - Human Cooperation
Prisoner’s dilemma
15,15
20,0
0,20
19,19

12. The Evolution of Cooperation (Axelrod,1984)
Tit for tat
being nice (cooperate first)
pay back immediately
easy to be recognized
Shadow of the future (discount parameter)
if the probability of meeting again is large enough, it is better to be nice…

13. Game Theory - Coordination Games
1,1
0,0

14. Evolutionary Game Theory
Classical Game Theory suffers from a number of conceptual weaknesses (hyper-rational players, selection problem of strategy, static theory)
Evolutionary Game Theory introduces evolutionary dynamics for strategy selection, i.e. the evolutionary most stable strategy dominates the system dynamics
Players are rational about their own decisions but do not need to know the strategies (and reasoning) of all other players!
Very suited for agent-based modeling approaches as it allows for learning and adaption of agents!

15. Evolutionary Learning
The main assumption underlying evolutionary thinking is that the entities which are more successful at a particular time will have the best chance of being present in the future.
Selection
Replication
Mutation

16. Evolutionary Learning
Selection is a discriminating force that favors some specific entities rather than others.
Replication ensures that the entities (or their properties) are preserved, replicated or inherited from one generation to another
Selection and replication work closely together, and in general tend to reduce diversity
the generation of new diversity is the job of the mutation mechanism

17. How agents can learn
Imitation : People copy the behavior of others, especially behavior that is popular or appears to yield high payoffs.
Reinforcement : People tend to adopt actions that yielded a high payoff in the past, and to avoid actions that yielded a low payoff.
Best reply : People adopt actions that optimize their expected payoff given what they expect others to do. Subjects choose best replies to the empirical frequency distribution of their opponents’ previous actions, i.e. “Fictitious Play”. Agents may also update their beliefs about others’ behavior.

18. From Factors to Actors (Macy,2002)
Simple and predictable local interactions can generate familiar but enigmatic global patterns, such as the diffusion of information, emergence of norms, coordination of conventions, or participation in collective action

19. From Factors to Actors (Macy,2002)
Simple and predictable local interactions can generate familiar but enigmatic global patterns, such as the diffusion of information, emergence of norms, coordination of conventions, or participation in collective action
Agents are autonomous
Agents are interdependent
Agents follow simple rules
Agents are adaptive and backward-looking

20. From Factors to Actors (Macy,2002)
From Factors to Actors (Macy,2002)
Simple and predictable local interactions can generate familiar but enigmatic global patterns, such as the diffusion of information, emergence of norms, coordination of conventions, or participation in collective action
Agents are autonomous
Agents are interdependent
Agents follow simple rules
Agents are adaptive and backward-looking
reduction/ atomization problem!
can be a very problematic assumption!!

21. Programming a simulator
Programming a simulator
Agent-based simulations
the models simulate the simultaneous operations of multiple agents, in an attempt to re-create and predict the actions of complex phenomena.
agents can be pedestrians, animals, customers, internet users, etc… 21 21

22. Programming a simulator
Programming a simulator
Agent-based simulations
State update
According to set of behavioral rules
Time t
Time t + dt
Characterized by a state (or states)
(location, dead/alive, color, level of excitation)
New state
Time line T1 T2 dt 22 22

23. Programming a simulator
Programming a simulator
Agent-based simulations
State update
According to set of behavioral rules, including neighborhood
Time t
Time t + dt
Characterized by a state
(location, dead/alive, level of excitation)
New state
Time line T1 T2 23 23

24. Programming a simulator
Programming a simulator
Programming an agent-based simulator often goes in five steps
Initialization
Initial state; parameters; environment
Time loop
Processing each time steps
Agents loop
Processing each agents
Update
Updating agent i at time t
Save data
For further analysis
Initialization
Time loop
end
Agents loop
end
Update state
Save data 24 24

25. Programming a simulator
Efficient Structure
define and initialize all variables in a script file that serves as the main of your program
execute the actual simulation as function with the pre-defined variables as inputs
automated analysis, visualization etc. of the simulation output may then be implemented in the main file using the outputs of the function that runs the actual program

26. Programming a simulator
Programming a simulator
Step 1: Defining the initial state & parameters
% Initial state
t0=0; %begining of the time line
dt=1 ; %time step
T=100; %number of time steps
%Initial state of the agents
State1 = [ ];
State2 = [ ];
etc… 26 26

27. Programming a simulator
Programming a simulator
Step 1: Defining the initial state & parameters
% Initial state
t0=0; %begining of the time line
dt=1 ; %time step
T=100; %number of time steps
%Initial state of the agents
State1 = zeros(1,50);
State2 = rand(1,50); 27 27

28. Programming a simulator
Programming a simulator
Step 2: Covering each time step
% time loop
For t=t0:dt:T
What happen at each time step?
- Update environment
- Update agents
end 28 28

29. Programming a simulator
Programming a simulator
Step 3: Covering each agent
% agent loop
For i=1:length(States)
What happen for each agent?
end 29 29

30. Programming a simulator
Programming a simulator
Step 4: Updating agent i at time t
% update
% Each agent has 60% chance to switch to state 1
randomValue=rand();
if (randomValue<0.6)
State(i)=1;
else
State(i)=0;
end
Use sub-functions for better organization!! 30 30

31. Programming a simulator
Programming a simulator
Step 4: Updating agent i at time t
% update
% Each agent has ‘proba’ chance to switch to state 1
randomValue=rand();
if (randomValue<proba)
State(i)=1;
else
State(i)=0;
end
Define parameters in the first part!! 31 31

32. Programming a simulator
Programming a simulator
Step 4: Updating agent i at time t
% Initial state
t0=0; %begining of the time line
dt=1 ; %time step
T=100; %number of time steps
proba=0.6; %probability to switch state
Define parameters in the first part!! 32 32

33. Programming a simulator
Programming a simulator
Step 5: Final processing
% Outputs and final processing
propAlive = sum(states)/length(states);
And return propAlive as the output of the simulation. 33 33

34. Programming a simulator
Running simulation
>> p=simulation()
p =
0.54
0.72

35. Programming a simulator
Programming a simulator
Alternatively : a framework for the simulator
% Running N simulations
N=1000 %number of simulation
for n=1:N
p(n)=simulation();
end
Use a framework to run N successive simulation and store the result each time. 35 35

36. Programming a simulator
Programming a simulator
Alternatively : a framework for the simulator
>> hist(p) 36 36

37.
Exercise 1
There is some additional material on the iterated prisoner’s dilemma available on the course website
If you are not familiar with the game, the ready- to-use implementation of the game will help you get a better understanding of its dynamics… 37 37

38.
Exercise 2
Start thinking about how to adapt the general simulation engine shown today for your project
Problems:
Workspace organization (files, functions, data)
Parameters initialization
Number of agents
Organization of loops
Which data to save, how frequently, in which format
What kind of interactions are relevant … 38 38

39. References Fudenberg & Tirole, Game Theory, MIT Press (2000)
References
Fudenberg & Tirole, Game Theory, MIT Press (2000)
Meyerson, Game Theory, Harvard University Press (1997)
Herbert Gintis, Game Theory Evolving, Princeton Univ. Press, 2nd Edition (2009)
Uhrmacher, Weyns (Editors), Multi-Agent Systems – Simulation and Applications, CRC Press (2009)