1. Social Norm, Costly Punishment and the Evolution to Cooperation : Theory, Experiment and Simulation Tongkui Yu 1, 2, Shu-Heng Chen 2, Honggang Li 1* 1 Department of Systems Science, Beijing Normal University 2 AI-ECON Research Center, Taiwan Chengchi University

2. Kung Fu Tzu: Confucianism (ethics) Han Fei Tzu: Legalism (law and punishment ） Punish or not?

3. (Legalism prevailed in turbulent society ) Han Fei Tzu: Legalism (law and punishment ） Punish or not?

4. Kung Fu Tzu: Confucianism (ethics) (Confucianism prevailed in stable society ） Punish or not?

5. Outline Model of donation game Evolutionary Game Theoretical Analysis Live Interactive Experiments Agent-based Computer Simulation Motivation Conclusion

6. Outline Model of donation game Evolutionary Game Theoretical Analysis Live Interactive Experiments Agent-based Computer Simulation Motivation Conclusion

7. Cooperation is very important not only for a society of human beings Motivation  Model  Theory  Experiment  Simulation  Conclusions

8. Cooperation is very important but also for many other biological systems Motivation  Model  Theory  Experiment  Simulation  Conclusions

9. Why people cooperate? Natural selection (Biology)  Competition Rationality (Economics)  Self-interest Motivation  Model  Theory  Experiment  Simulation  Conclusions Cooperation

10. Costly punishment as a mechanism to promote cooperation People will voluntarily incur costs to punish violations of social norms. E. Fehr, S. Gächter, Nature 415, 137 (2002) E. Fehr, U. Fischbacher, Nature 425, 785 (2003) D. J. Quervain, et al Science 305, 1254 -1258 (2004) C. F. Camerer, E. Fehr, Science 311, 47 (2006) J. Henrich et al., Science 312, 1767 (2006). Motivation  Model  Theory  Experiment  Simulation  Conclusions

11. The role of costly punishment in promoting cooperation is ambiguous Positive  Fehr E. & Gachter S. (2000) Am. Econ. Rev. 90, 980-994  Fehr E. & Gachter S. (2002) Nature 415, 137–140  Gurerk, O., Irlenbusch, B. & Rockenbach, B. (2006). Science 312,108–111 Negative  Dreber, A. etc. (2008) Nature, 452, 348-351.  Egas, M. & Riedl, A. (2008) PNAS, 275, 871-878.  Ohtsuki, H. etc. (2009) Nature 457, 79-82. Costly punishment can hardly lead to an efficient equilibrium The best choice to defectors is withholding help rather than punishing them Motivation  Model  Theory  Experiment  Simulation  Conclusions

12. The role of costly punishment in promoting cooperation is ambiguous Positive  Fehr E. & Gachter S. (2000) Am. Econ. Rev. 90, 980-994  Fehr E. & Gachter S. (2002) Nature 415, 137–140  Gurerk, O., Irlenbusch, B. & Rockenbach, B. (2006). Science 312,108–111 Negative  Dreber, A. etc. (2008) Nature, 452, 348-351.  Egas, M. & Riedl, A. (2008) PNAS, 275, 871-878.  Ohtsuki, H. etc. (2009) Nature 457, 79-82. Costly punishment can hardly lead to an efficient equilibrium The best choice to defectors is withholding help rather than punishing them Motivation  Model  Theory  Experiment  Simulation  Conclusions

13. The role of costly punishment in promoting cooperation is ambiguous Positive  Fehr E. & Gachter S. (2000) Am. Econ. Rev. 90, 980-994  Fehr E. & Gachter S. (2002) Nature 415, 137–140  Gurerk, O., Irlenbusch, B. & Rockenbach, B. (2006). Science 312,108–111. Negative  Dreber, A. etc. (2008) Nature, 452, 348-351.  Egas, M. & Riedl, A. (2008) PNAS, 275, 871-878.  Ohtsuki, H. etc. (2009) Nature 457, 79-82. Costly punishment usually decreases the average payoff of a society, so The best choice to defectors is withholding help rather than punishing them Motivation  Model  Theory  Experiment  Simulation  Conclusions

14. Costly punishment as a mechanism of promoting cooperation is ambiguous In real world, costly punishment is found to be widely existing Motivation  Model  Theory  Experiment  Simulation  Conclusions Fehr and Simon, 2000 Fehr and Gachter, 2002 Gurerk et al., 2006 Rockenbach & Milinski, 2006 Henrich et al., 2010 Falk et al., 2005 Oosterbeek et al., 2004 Henrich et al., 2005, 2006 Marlowe et al., 2008 Gachter & Herrmann, 2009

15. Controversy Costly punishment is less efficient but exists Question  What does costly punishment exist for ?  “ Costly punishment remains one of the most thorny puzzles in human social dilemmas ” (Nature 452, 297-298)  “ Costly punishment requires a mechanism for its evolution ” (Nature 452, 348-351) Motivation  Model  Theory  Experiment  Simulation  Conclusions

16. Our argument Ohtsuki’s analysis only focuses on the equilibrium (i.e. Cooperative Evolutionary Stable State, CESS). Although in equilibrium, punishment is not the most efficient. But if the society starts from an initial state far away from equilibrium, costly punishment may play a different role. Motivation  Model  Theory  Experiment  Simulation  Conclusions

17. Our work Extends Ohtsuki’s model by explicitly modeling the evolution process of individuals’ strategies Investigates the role of punishment in the route to cooperation Motivation  Model  Theory  Experiment  Simulation  Conclusions

18. Our results Costly punishment works in the route to cooperation  (1) Enlarge the attraction basin of cooperative evolutionary stable state (CESS).  (2) Increase the evolution speed to cooperative evolutionary stable state (CESS). Motivation  Model  Theory  Experiment  Simulation  Conclusions

19. Outline Model Evolutionary Game Theoretical Analysis Live interactive Experiments Agent-based Computer Simulation Motivation Conclusion

20. Donation Game A society with a very large population. Each individual is endowed with a binary reputation: good (G) or bad (B). Motivation  Model  Theory  Experiment  Simulation  Conclusions

21. Donation Game At each time, two players are sampled randomly. One as donor and the other as recipient. Donor has 3 choices: cooperation (C), defection (D), and punishment (P). Recipient does nothing. If C, donor spends a cost c (c=2) to give recipient a benefit b (b=3); if D, no gain no loss; If P, donor spends a cost α (α=1) to give recipient a loss β (β=4). Recipient Donor C (b,-c) (0,0) D P (-β,-α) D R Motivation  Model  Theory  Experiment  Simulation  Conclusions

22. Donation Game Each player has a strategy, which determines his action (C, D, or P) when as a donor according to the reputation (G or B) of his recipient. Motivation  Model  Theory  Experiment  Simulation  Conclusions G B C C C D C P Recipient’s reputation D C D D D P P C P D P P Strategy 1 Strategy 2 Strategy 3 Strategy 4

23. Donation Game After each interaction, the reputation of the donor will be updated according to a ‘social norm’ The reputation update process is susceptible to errors.  With probability μ (0<μ<1/2), an incorrect reputation is assigned. Motivation  Model  Theory  Experiment  Simulation  Conclusions

24. Social norm Assigns a new reputation to the donor, according to both the donor’s action (X) and the recipient’s reputation (J). G B C D P G G B G B G (Donor’s action) X J (Recipient’s reputation) Punishment-optional norm Motivation  Model  Theory  Experiment  Simulation  Conclusions G B C D G G B G (Donor’s action) X J (Recipient’s reputation) (Donor’s new reputation) Non-punishment norm G B C D P G G B B B G (Donor’s action) X J (Recipient’s reputation) Punishment-provoking norm

25. 3-level of evolution Reputation  is the instantaneous result of agents’ actions Strategy  is updated by individuals according to their payoff in a period of time Social norm  may evolve slowly according to the average benefit of all social members in a longer horizon

26. Focus of this work Within such a framework, we will  Firstly, keep the social norm fixed and model the evolution dynamics of strategy proportions (i.e. the ratios of individuals taking CC, CD, CP and DD strategy) for each of these 3 social norms (non- punishment, punishment-optional and punishment- provoking)  Then, compare the cooperation ratio and average payoff along the evolution route for these 3 social norms to get the driving force of the social norm evolution  At the same time, we can get a insight of the role of punishment in promoting cooperation. Motivation  Model  Theory  Experiment  Simulation  Conclusions

27. Outline Model Evolutionary Game Theoretical Analysis Live interactive Experiments Agent-based Computer Simulation Motivation Conclusion

28. Strategy dynamics The driving force of individual’s strategy evolution is the expected revenue of each strategy. We calculate the expected revenue of each strategy Insert them into “ Replicator Dynamics” HofbauerJ. & Sigmund K. (2003) Evolutionary game dynamics. Bull. Am. Math. Soc. 40, 479-519. Motivation  Model  Theory  Experiment  Simulation  Conclusions

29. Strategy Dynamics Non-punishment norm Motivation  Model  Theory  Experiment  Simulation  Conclusions

30. Punishment-optional norm Motivation  Model  Theory  Experiment  Simulation  Conclusions Strategy Dynamics

31. Punishment-provoking norm Motivation  Model  Theory  Experiment  Simulation  Conclusions Strategy Dynamics

32. Evolutionary Stable State Evolutionary Stable State (ESS):  Attractive state Given all other individuals take some strategy, the best choice for one individual is to take that strategy.  Cooperative (or Non-cooperative) Evolutionary Stable state (CESS or NESS) Most agents cooperate (defect). Attraction basin of a CESS:  All the initial states that will converge to the CESS.  The larger attraction basin a CESS has, the more probable the society will converge to this CESS. Motivation  Model  Theory  Experiment  Simulation  Conclusions

33. Phase portraits for 3 social norms Punishment-optional norm Punishment-provoking norm Non-punishment norm

34. Phase portraits of 3 social norms NESS Non-punishment norm Punishment-optional norm Punishment-provoking norm

35. Phase portraits of 3 social norms NESS CESS NESS CESS Non-punishment norm Punishment-optional norm Punishment-provoking norm

36. Phase portraits of 3 social norms NESS CESS NESS CESS Non-punishment norm Punishment-optional norm Punishment-provoking norm

37. Phase portraits of 3 social norms 15% NESS CESS NESS CESS Non-punishment norm Punishment-optional norm Punishment-provoking norm

38. Phase portraits of 3 social norms 15% 60% NESS CESS NESS CESS Non-punishment norm Punishment-optional norm Punishment-provoking norm

39. Phase portraits of 3 social norms 15% 60% 81% NESS CESS NESS CESS Non-punishment norm Punishment-optional norm Punishment-provoking norm

40. Intuitive view of attraction basin Punishment-optional norm Punishment-provoking norm Motivation  Model  Theory  Experiment  Simulation  Conclusions

41. Intuitive view of attraction basin Punishment-optional norm Punishment-provoking norm Motivation  Model  Theory  Experiment  Simulation  Conclusions

42. Intuitive view of attraction basin NESS CESS Blue: Punishment- optional norm Red: Punishment- provoking norm Motivation  Model  Theory  Experiment  Simulation  Conclusions

43. Converge speed from the same initial point in different social norms Blue: Punishment- optional norm Red: Punishment- provoking norm Motivation  Model  Theory  Experiment  Simulation  Conclusions

44. Outline Model Evolutionary Game Theoretical Analysis Live interactive Experiments Agent-based Computer Simulation Motivation Conclusion

45. Live interactive Experiments ( Please choose your strategy ) Strategy 1 ( “CC” ) Strategy 2 ( “CD” ) Strategy 3 ( “CP” ) Strategy 4 ( “DD” ) ( 1 ) Subjects are requested to choose a strategy Students of Taipei, Beijing and Chongqing Motivation  Model  Theory  Experiment  Simulation  Conclusions Fischbacher, U. (2007)

46. Live interactive Experiments  Randomly match the subjects to form pairs, one as donor, the other as recipient;  Calculate the payoffs of the subjects according to the donor’s strategy and the recipient’s reputation  Update the donor’s reputation according to the applying social norm ( 2 ) System background processing Motivation  Model  Theory  Experiment  Simulation  Conclusions

47. Live interactive Experiments You are : donor or recipient Your opponent’s reputation Your opponent’s strategy Your payoff in this period Your reputation after this interaction Your strategies and payoffs in last 15 periods The strategies and payoffs in last 15 periods of a randomly selected subjects ( 3 ) Interaction results display Motivation  Model  Theory  Experiment  Simulation  Conclusions ( 4 ) A new period starts and subjects are requested to choose strategy again

48. Live interactive Experiments Starting from state with very few defectors (DD) DD CP The experiment results are consistent with the theoretical analysis qualitatively. Non-Discriminable normPunishment-optional normPunishment-provoking norm Starting from state with many defectors (DD) Motivation  Model  Theory  Experiment  Simulation  Conclusions

49. Initial strategy choice  The strategy chosed by a subject at the beginning of a experiment  Subjects make initial strategy choice without any idea about the experiment and this may reflect the culture of real society Taipei BeijingChongqing Live interactive Experiments Motivation  Model  Theory  Experiment  Simulation  Conclusions

50. Outline Model Evolutionary Game Theoretical Analysis Live interactive Experiments Agent-based Computer Simulation Motivation Conclusion

51. Agent-based Computer Simulation Artificial society system which realizes the individuals’ strategy update process in detail  Each period Two agents are sampled randomly to play the donation game With a probability μ(learning rate), one individual will be sampled to be the learner  He will choose another individual randomly  Compare the average revenues in the last L (memory length) periods  If the learner’s revenue is smaller than that of the learned, the learner will take the strategy of the learned. Motivation  Model  Theory  Experiment  Simulation  Conclusions

52. Agent-based Computer Simulation Artificial society system which realizes the individuals’ strategy update process in detail  Each period Two agents are sampled randomly to play the donation game With a probability μ(learning rate), one individual will be sampled to be the learner  He will choose another individual randomly  Compare the total revenues in the last L (memory length) periods  If the learner’s revenue is smaller than that of the learned, the learner will take the strategy of the learned. Motivation  Model  Theory  Experiment  Simulation  Conclusions

53. Agent-based Computer Simulation Artificial society system which realizes the individuals’ strategy update process in detail  Each period Two agents are sampled randomly to play the donation game With a probability μ(learning rate), one individual will be sampled to be the learner  He will choose another individual randomly  Compare the total revenues in the last L (memory length) periods  If the learner’s revenue is smaller than that of the learned, the learner will take the strategy of the learned. Motivation  Model  Theory  Experiment  Simulation  Conclusions

54. Agent-based Computer Simulation Results  Replicate the theoretical results (long memory length, L; small learning rate, μ) Theory Simulation Motivation  Model  Theory  Experiment  Simulation  Conclusions

55. Agent-based Computer Simulation Results  State with co-existence of CC, CD and CP strategy (short memory length, L; quick learning rate, μ) State with co-existence of CC, CD and CP strategy Motivation  Model  Theory  Experiment  Simulation  Conclusions

56. Agent-based Computer Simulation Results  State with co-existence of CC, CD and CP strategy (short memory length, L; quick learning rate, μ) State with co-existence of CC, CD and CP strategy Initial strategy choice in experiment reflecting the culture in real society Motivation  Model  Theory  Experiment  Simulation  Conclusions

57. Outline Model Evolutionary Game Theoretical Analysis Live interactive Experiments Agent-based Computer Simulation Motivation Conclusion

58. We study the role of punishment in promoting cooperation, on the social norm level, within a framework of 3-level evolution  Reputation, Strategy and Social norm By three methods  Evolutionary game theory  Live interactive expriment  Agent based simulation Motivation  Model  Theory  Experiment  Simulation  Conclusions

59. Conclusion Costly punishment works in promoting cooperation : Motivation  Model  Theory  Experiment  Simulation  Conclusions  (1) It can enlarge the attraction basin of CESS When the society has few people cooperating, it can only struggle out of social dilemma by punishment

60. Conclusion Costly punishment works in promoting cooperation : Motivation  Model  Theory  Experiment  Simulation  Conclusions  (2) It can increase the convergence speed to CESS If the society are less patient, it can only speed up to cooperative state by punishment.

61. (Legalism in turbulent society ) (Confuci anism in stable society ） Motivation  Model  Theory  Experiment  Simulation  Conclusions

62. Thank you!