1. Game Theory and Cooperation
Jeff Schank
UC Davis

2. Origin of Game Theory
John von Neumann and Oskar Morgenstern (1944) Theory of Games and Economic Behavior, Princeton University Press.
John F. Nash, Jr. (1950) Equilibrium Points in N-Person Games. PNAS, 36: 48–49.
John Maynard Smith and George R. Price (1973) The logic of animal conflict. Nature, 246: 15–8.

3. What is a Game?
A game involves at least two players, not necessarily human
For example:
Player 1 = a person, Player 2 = a stock market;
Player 1 = an animal, Player 2 = an animal (possibly from different species)
Player 1 = a firm, Player 2 = consumers
Each player has strategies whose payoff depends on the strategy used by (an)other player(s)
Payoffs can be in terms of money, resources, happiness, or more generally what economists call utility
It can be very difficult to precisely calculate the payoffs for a strategy
The best cases for calculating payoffs precisely involve money
Although calculating payoffs for strategies are often difficult or effectively impossible, if we can calculate or estimate payoffs, then the theory of games can help us find the best or at least good strategies

4. Prisoner’s Dilemma

5. –1, –1
–1, -1
–5, –.5
–.5, –5
–3, –3

6. Normal Form of the Prisoner’s Dilemma
Don’t Confess (cooperate)
Confess (defect)
–1, –1
–5, –.5
–.5, –5
–3, –3

7. Explicit Assumptions
There are choices of strategies available to both players
The success of a chosen strategy depends on the choice of strategy by an opponent
Players behave rationally: each player chooses the strategy that maximizes their individual payoff

8. Nash Equilibrium
A pure-strategy Nash equilibrium is a choice of strategies by all players such that no player can unilaterally improve his/her payoff by changing strategies
Calculation:
Assume: Strategies are fixed
Question: If each player is given the strategy of the other, can that player improve his/her payoff by switching?
Solution: If so, then the strategy set is not a Nash Equilibrium, otherwise it is

9. Normal Form of the Prisoner’s Dilemma
Don’t Confess (cooperate)
Confess (defect)
–1, –1
–5, –.5
–.5, –5
–3, –3
Suppose Player 1 and 2 both choose “Don’t Confess”
Player 1 knows that player 2 didn’t confess and sees that she can improve her payoff by changing from don’t confess to confess
Player 2 know that player 1 didn’t confess and sees that he can improve his payoff by changing from don’t confess to confess
Thus, [Don’t Confess, Don’t Confess] is not a Nash equilibrium
The same reasoning leads to the conclusion that [Confess, Confess] is a Nash equilibrium

10. Stag Hunt Game

11. Normal Form of the Stag Hunt Game
Cooperate
Defect
2, 2
0, 1
1, 0
1, 1
Suppose Player 1 and 2 both choose “Cooperate”
Player 1 knows that player 2 cooperated and sees that she cannot improve her payoff by changing from cooperate to defect
Player 2 knows that player 1 cooperated and sees that he cannot improve his payoff by changing from cooperate to defect
Thus, [Cooperate, Cooperate] is a Nash equilibrium
Is [Defect, Defect] a Nash equilibrium?

12. Hawk-Dove Game

13. Normal Form of the Hawk-Dove Game
–2, –2
4, 0
0, 4
1, 1
Suppose Player 1 and 2 both choose “Hawk”
Player 1 knows that player 2 played Hawk and sees that she can improve her payoff by changing from Hawk to Dove
Player 2 knows that player 1 played Hawk and sees that he can improve his payoff by changing from Hawk to Dove
Thus, [Hawk, Hawk] is not a Nash equilibrium
Is [Dove, Dove] a Nash equilibrium?

14. Calculating a Mixed Strategy for the Hawk-Dove Game

15. Normal Form of the Hawk-Dove Game
(V–C)/2, (V–C)/2
V, 0
0, V
V/2, V/2
Suppose Player 1 and 2 both choose “Hawk”
Player 1 knows that player 2 played Hawk and sees that she can improve her payoff by changing from Hawk to Dove if C > V
Player 2 knows that player 1 played Hawk and sees that he can improve his payoff by changing from Hawk to Dove if C > V
Thus, [Hawk, Hawk] is not a Nash equilibrium
Is [Dove, Dove] a Nash equilibrium?

16. Calculating a Mixed Strategy for the Hawk-Dove Game

17. Different Games Model Different Aspects of Cooperation
Prisoner’s Dilemma: Models the problem of defecting/cheating in cooperative behavior
Stag Hunt: Models the problem of coordinating behavior to achieve cooperation
Hawk-Dove: Models the problem of competing vs. cooperating/sharing resources

18. Repeated Games What happens if such games are played repeatedly?
What if a population of individuals played games such as the Prisoner’s Dilemma, Stag Hunt, or Hawk-Dove games, would they cooperate?
These are more difficult questions to answer

19. Evolutionary Game Theory
We can answer these questions in evolutionary contexts (biological or cultural) by asking whether strategy I (e.g., cooperate or dove) can be invaded by strategy J (e.g., defect or hawk) in small numbers (e.g., by mutation or strategy switching)?
If not, strategy I is evolutionarily stable

20. Maynard-Smith & Price (1973)
They introduced two conditions for a strategy to be evolutionarily stable (ESS)
The first is the strict Nash equilibrium condition:
E(I, I) > E(J, I), which means that the expected payoff for strategy I against itself is greater than the expected payoff of any strategy J ≠ I against I
Or if E(I, I) = E(J, I)  E(I , J) > E(J, J), which means that if J does as well against I as I against itself, then I much do better against J than J does against itself for J ≠ I

21. In General, Cooperation is Not an ESS
Assuming that any player in a population can play any other player (random assortment assumption, like in Kalick and Hamilton’s Mate Choice model)
Cooperation in the Prisoner’s Dilemma is not an ESS
Cooperation in the Stag Hunt game may be an ESS depending on the size of the cooperative payoff
In the Hawk-Dove Game, a mixed ESS only exists if the cost of fighting is greater than the reward (i.e. C > V)

22. Implicit Assumptions Disembodiment assumption
Infinite populations
No spatial structure
No movement strategies
No individual characteristics such as personality
Averaged payoff assumption
Most clearly evident in the Hawk-Dove game
Even distribution of payoff among cooperators
Assumptions can interact

23. Averaged Hawk-Dove Game
(V–C)/2, (V–C)/2
V, 0
0, V
V/2, V/2

24. Classic Hawk-Dove Game
V or –C, V or –C
V, 0
0, V
V or 0, V or 0

25. Sharing Doves
Hawk
Dove
V or –C, V or –C
V, 0
0, V
V/2, V/2

26. Equal Hawks
Hawk
Dove
(V–C)/2, (V–C)/2
V, 0
0, V
V or 0, V or 0

27. Hawk Dove Games
For each of the four versions above, they each have the same ESS for the same set of payoff values
PH = V/C
For example, if C = 18 and V = 24, PH = 24/18 4/3  No mixed ESS, Doves cannot persist!
For example, if C = 24 and V = 12, PH = 12/24 1/2  there is a mixed ESS, Doves can persist!

28. Let’s Consider Agents that Play the Hawk-Dove Game in Space
Agents are located in a discrete 2D space
Agents accumulate resources by playing the the Hawk-Dove game with agents next to them
Agents can reproduce if they accumulate sufficient resources
Offspring disperse locally

29. Aggregate vs. Random Shuffle

30. Aggregation & Random Shuffle
frequencyOfDoves 0.5
startEnergy 75.0
minEnergy 0.25
reproduceEnergy
maxEnergy
reproductionRadius 1
mutationRate 0.01
lifeSpanSD 5.0
lifeSpan 50.0
gridWidth 100
gridHeight 100
randomReproduction false
aggregate true
unboundedSpace true
searchRadius 2
synchronous false
offspringPlay true
randomShuffle false
fixedReproduction true
startingResource 50.0
cost 18.0
benefit 24.0
payoff_multiplier_x 0.0
hh 3.0
hhcw 24.0
hhcl
dd 12.0
ddcw 24.0
ddcl 0.0
hd 24.0
dh 0.0
PD_TYPE 0
SH_TYPE 1
HD_TYPE 2
CHD_TYPE 3
FDHD_TYPE 4
FHHD_TYPE 5
SD_TYPE 6

31. Prisoner’s Dilemma
Next consider the Prisoner’s dilemma with a similar agent-based model
The well-known solution is to defect
We allow agents to play with each other in space

32. What happens if we make the Sucker’s payoff (S) worse?

33. The Worse it is, the Better it is for Cooperation
Smaldino PE, Schank JC, McElreath R (2013) Increased costs of cooperation help cooperators in the long run. The American Naturalist, 181(4), 451–463.

34. The Worse it is, the Better it is for Cooperation
Smaldino PE, Schank JC, McElreath R (2013) Increased costs of cooperation help cooperators in the long run. The American Naturalist, 181(4), 451–463.

35. What if we allow agents to move in space?

36. Smaldino PE, Schank JC (2012) Movement patterns, social dynamics, and the evolution of cooperation. Theoretical Population Biology, 82, 48–58

37. Smaldino PE, Schank JC (2012) Movement patterns, social dynamics, and the evolution of cooperation. Theoretical Population Biology, 82, 48–58

38. Hypothetical Interpretation
Behavioral Syndromes (similar to personality): Behavioral correlations across contexts
For example, an animal may be bold when foraging, predators are presence, and in mating contexts
Or an animal may be shy when foraging, predators are presence, and in mating contexts
Random movement strategies can represent behavioral syndromes
For example, SP and ZZ could be interpreted as bold behavior strategies
Whereas, CY, CH, and TC are shy behavioral strategies

39. Smaldino PE, Schank JC (2012) Movement patterns, social dynamics, and the evolution of cooperation. Theoretical Population Biology, 82, 48–58

40. Smaldino PE, Schank JC (2012) Movement patterns, social dynamics, and the evolution of cooperation. Theoretical Population Biology, 82, 48–58

41. Averaging/Equal Sharing Assumption
Recall that in cooperative games, cooperators share the cooperative benefit
In the Hawk-Dove game, we saw this assumption matters when agents are embodied
Does the sharing assumption matter in general?
Is it rational to share?

42. The Dictator Game
The first player (dictator) decides how to divide a resource with a second player (recipient)
Self-interested Solution: Keep it all
Hundreds of experiments by economists and anthropologists find that across cultures, people often split a resource evenly
On average dictators give 28% to recipients

43. What if we allow agents to move, aggregate and play the DG repeatedly for resources to reproduce?

44. Theoretical example
Consider two groups of agents that can share a resource
One group shares and the other does not
Each agent in a group, chosen at random, is either a sharer or recipient on each round
What is the expected payoff for individuals in each group?

45. The Problem
Different degrees of sharing generate different levels of resource variance among individuals in structured populations (e.g., groups, association networks)
Do these different variances translate into fitness differences?

46. A Simple Model
Consider the same two groups as before with exactly the same properties (e.g., age, etc.)
On each round, they play for RG resources
Assume that each agent needs kRG resources to reproduce
Sharing agents must play 2k rounds to reproduce
However, selfish agents reproduce in a variable number of rounds (x = k + r rounds) described by a negative binomial equation

47. Timing of Reproduction: Sharing vs. Not Sharing

48. An Agent-Based Model 2-Dimensional Space
Mobility (important for engaging other agents in space)
Aggregation (a basic condition for social behavior)
Proximate Spatial Reproduction
Population Structure emerges

49. An Agent-Based Model continued …
Lifespan
Resources required for reproduction
Resources acquired by playing the dictator game
Reproduction, heritability, and mutation
Parental investment

50. Types of Simulation Experiments
Multilevel selection
Individual-only selection (agent swaps)
Group-level-only selection

51. Example

52. Multilevel and Individual Only

53. Group Selection Example

54. Group-only Selection Simulation

55. Conclusions
Game theory is a powerful tool good strategies in games involving one or more opponents
Game theory is also a powerful tool for understanding cooperation and the conditions under which it can occur
Game theory, however, makes two assumptions, in the context of cooperation, that can limit its application
Disembodied agents
Averaged or shared payoffs, particularly for cooperators
By using agent-based models, we can investigate embodied agents and discover that in many cases, stable game-theoretic solutions depend on embodiment and context