1. Indirect Reciprocity in the Selective Play Environment Nobuyuki Takahashi and Rie Mashima Department of Behavioral Science Hokkaido University 08/07/2003

2. How can we account for altruism? Kin selection  altruism among genetically related individuals Reciprocal altruism (TFT strategy)  altruism among 2 players in repeated interactions How can we account for altruism among unrelated individuals without repeated interaction? Altruism = Giving resources (helping others, unilateral cooperation) at a cost of oneself Previously proposed answers

3. Indirect reciprocity is the answer! If A gives to B, B does not give back to A. Instead, C, a third party, may give to A. Equivalent to pure-generalized exchange in sociology (Takahashi, 2000) Can we think of a strategy that is equivalent to TFT in direct reciprocity?

4. Review of previous research Giving game and the Image Scoring Strategy (Nowak and Sigmund, 1998a, b) Framework of giving game A pair of donor and recipient is randomly chosen from a population. A donor decides whether to give his resources to his recipient with a cost of c (recipient receives the benefit b: b>c).  In this game, of course a rational donor should never give his resources to a recipient.

5. Framework of Giving Game (continued) Adding a score 1)Errors in strategy execution (occurs with a probability έ) performing an action different from the one prescribed by the strategy. 2) Errors in perception (occurs with a probability δ) misperceiving an action performed by another individual Errors Each player has a reputation score s which has two values: “good” or “bad”. A donor gives if he considers the recipient’s score “good”. A donor doesn’t give if he considers the recipient’s score “bad”. Score How to assign a score is regulated by a strategy.

6. Solution 1: Image Scoring Strategy (Nowak and Sigmund, 1998a, b) Evolutionary computer simulation and mathematical analysis Image Scoring Strategy makes indirect reciprocity possible. Image Scoring Strategy is a TFT-like strategy: It assigns “good” to previous givers, whereas it assigns “bad” to previous non-givers.  It gives only to givers.

7. Problem of Image Scoring (Leimar & Hammerstein, 2001; Panchanathan & Boyd, 2003) When matched with a “bad” recipient who did not give… ◇ Image scoring doesn’t give to the recipient. →becomes being perceived as “bad” by another image scoring. →loses chances to be given by others in the future. ◇ All-C always gives to a recipient. →keeps his “good” score and chances to be given by others. The expected payoff: All-C > Image Scoring ⇒ All-C increases, and All-D can invade.

8. Solution 2: The Standing Strategy (L & H, 2001; P & B, 2003) Standing strategy uses the 2nd order information. ◇ Standing assigns “good” to previous givers. ◇ If the recipient did not give to the previous recipient, standing checks whether or not it was justified as a punishment toward a “bad” recipient by using the information of the score of the recipient’s previous recipient. ◇ If it is a justifiable not giving, the standing assigns “good”, whereas if it is an unjustifiable not giving, it assigns “bad”.

9. 1st and 2nd order information good or bad (2) did not give gave or (1) Current recipient’s previous recipient Current donor Current recipient (1)The recipient’s previous behavior (2)The score of the recipient’s previous recipient

10. Representation of the Standing Strategy Standing distinguishes between justified and unjustified not-giving, so that standing can punish non-givers without losing his “good” score. Current recipient’s previous behavior GoodBad GaveGood Did not giveBadGood Current recipient’s previous recipient’s score Standing = GGBG unjustified defection justified defection as a punishment

11. Problem of the Standing Strategy (Mashima and Takahashi, 2003) If there are no errors in perception, the standing strategy can make indirect reciprocity possible. The expected payoff: Standing > All-C Standing saves the cost of giving when matched with a “bad” recipient.  All-D cannot invade.

12. Problem of the Standing Strategy (Mashima and Takahashi, 2003) If there are errors in perception, it is not the case, however. Suppose the current recipient didn’t give to a “good” person.  The current donor considers the current recipient “bad” and does not give. good bad didn’t give Current donor Current recipient Next donor doesn’t give good will give Previous recipient

13. Problem of the Standing Strategy (Mashima and Takahashi, 2003) However, suppose that actually the current donor misperceived the current recipient’s behavior: the current recipient actually gave to a “good” person. good  The next donor considers the current recipient “good.” good gave Current donor Current recipient Next donor Previous recipient

14. Problem of the Standing Strategy (Mashima and Takahashi, 2003) Nevertheless, the current donor does not give to the current recipient. good gave Current donor Current recipient Next donor doesn’t give The next donor will consider the current donor “bad” since he does not give to a “good” person.  bad Previous recipient

15. Problem of the Standing Strategy (Mashima and Takahashi, 2003) The next donor will not give to the current donor. When there are errors in perception, the expected payoff: All-C > Standing  All-D can invade.  good gave Current donor Current recipient Next donor doesn’t give bad will not give Previous recipient

16. Solution 3: Strict Discriminator (GBBB) (Mashima and Takahashi, 2003) Only an individual who was matched with and gave to a “good” recipient is considered “good” by Strict Discriminator (GBBB). It sometimes considers All-C “bad” since All-C gives even to All-D.  It drives out not only All-D but also All-C. GoodBad Gave ① Good ② Bad Did not give ③ Bad ④ Bad Image Scoring and Standing always consider All-C “good.”

17. Solution 3 (continued) The results of evolutionary computer simulation showed that…… Strict discriminator could always dominate the population and make indirect reciprocity possible.

18. Remaining issue of Mashima and Takahashi (2003) ---- (1) The strict discriminator strategy (GBBB) could maintain indirect reciprocity when there can be only 3 strategies: ALL-C, ALL-D, and strict discriminator  What if other strategies are also present?

19. Remaining issue (2) Too strict! The strict discriminator strategy gives only to the recipient who was matched with a “good” recipient and gave to her on the previous round.  He does not give even to his own kind (i.e., the strict discriminator itself) who did not give on the previous round because she was unfortunately matched with a “bad” recipient. Isn’t this realistic as a behavioral pattern of real people? GoodBad Gave  Good  Bad Didn’t give  Bad  Bad

20. Remaining issue (2) (continued) This unrealistic interpretation of the result is required because the above simulation used the random matching paradigm (i.e., on each round a pair of a donor and a recipient is chosen randomly from a population). However, if a donor knows every other player’s reputation score, why not choose a “good” player as a recipient and avoid being paired with a “bad” player? GoodBad Gave  Good  Bad Didn’t give  Bad  Bad

21. Selective Play Paradigm It was proposed in the 90s in social psychology (e.g., Yamagishi and Hayashi 1996, Takahashi 2000). Forced play --Each player has to interact with the designated partner. (e.g., repeated PD, random matching giving game) VS. Selective play --Each player has a choice of selecting a desirable partner.

22. Potential advantage of Selective Play Remaining issue (2)  The unrealistic nature of GBBB may be resolved since the situation in which the 4 th gene matters occurs rarely. GoodBad Gave  Good  Bad Didn’t give  Bad  Bad

23. Potential advantage of Selective Play It is easier for conditionally altruistic strategies (e.g., image scoring, standing, strict discriminator) to target their giving to givers in the selective play environment.  ”Punishing” non-givers that is sometimes costly since players might lose their “good” score is unnecessary. The emergence of indirect reciprocity may be easier in the selective play environment. 

24. New series of simulations (1) 16 strategies can be present simultaneously. (2) Comparing forced play situation (random matching) with selective play situation (a donor randomly chooses one of the “good” players as his recipient) Purpose: To examine the remaining 2 issues shown above.

25. Strategies in the forced play environment 16 strategies (2  2  2  2=16) are available. GGGG=All-C BBBB=All-D GGBB=image scoring GGBG=standing GBBB=strict discriminator GoodBad Gave  Good or Bad  Good or Bad Didn’t give  Good or Bad  Good or Bad Current recipient’s previous behavior Current recipient’s previous recipient’s score

26. Strategies in the selective play environment When a potential recipient gave to someone on the previous round, a donor decides her score as he would do in the random matching environment. When a potential recipient did not give, if a donor believes that there were “good” players, the recipient is considered to have not given to a “good” recipient. If a donor believes that there was no “good” player, the recipient is considered to have not given to a “bad” recipient. GoodBad Gave  Good or Bad  Good or Bad Didn’t give  Good or Bad  Good or Bad

27. Parameters Evolutionary computer simulation On each round, Random matching condition – a pair of donor and recipient is chosen randomly Selective play condition – a donor is chosen randomly and selects a desirable recipient 1500 rounds per generation (m=1500) At the end of each generation, Selection and Mutation occur (mutation rate:μ=0.0001). Group size (n)=300, Errors in perception (έ ) = 0.025, Errors in behavior (δ) =0.025, Benefit/cost ratio = 2, 4, 6, 8, 10.

28. Questions to be asked 1.Does indirect reciprocity emerge when 16 strategies can be present simultaneously? 2.Is the emergence of indirect reciprocity easier in the selective play environment? 3.What is the composition of a population when indirect reciprocity emerges?

29. Result (1) Figure 1. Frequency of maintained indirect reciprocity (mutual cooperation) after 10000 generations

30. Result (2) Figure 2. An example of the history of giving rate

31. Indirect reciprocity is more attainable in the selective play environment When 16 strategies are possible, although indirect reciprocity failed in most cases in the random matching environment, it was maintained in most cases in the selective play environment.  Which strategy maintained indirect reciprocity in the selective play environment?

32. Result (3) Figure 3. An example of the history of evolution when indirect reciprocity was maintained

33. Result (4) Composition of population When indirect reciprocity was maintained in the selective play environment, two strategies dominated the population in most cases. They are GBBB (strict discriminator) and GBBG (extra standing). RatioGBBB dominated GBBG dominated Other strategies dominated Indirect reciprocity failed 29308 481002 68813 85617 7634 Table 1. Frequency of domination by each strategy

34. What makes the extra standing strategy win in the selective play environment? Extra standing (GBBG) gives to a recipient who gave to a “good” player or who did not give to a “bad” player. GoodBad Gave  Good or Bad  Good or Bad Didn’t give  Good or Bad  Good or Bad

35. What makes the extra standing strategy win in the selective play environment? Extra standing (GBBG) gives to a recipient who gave to a “good” player or who did not give to a “bad” player. In the random matching environment, it cannot maintain indirect reciprocity because of the 4 th gene since it sometimes gives even to All-D. In the selective play environment, the 4 th gene rarely matters. Almost always there are some “good” players in a population. GoodBad Gave  Good or Bad  Good or Bad Didn’t give  Good or Bad  Good or Bad

36. Characteristics of the strategies that maintain indirect reciprocity in the selective play environment The 4 th gene matters little  1.Give to a player who gave to a “good” player 2.Do not give to a player who did not give to a “good” player or who gave to a “bad” player  GBBG (extra standing) and GBBB (strict discriminator) GoodBad Gave  Good  Bad Didn’t give  Bad  Good or Bad

37. Discussion (1) The emergence of indirect reciprocity is more attainable in the selective play environment. (2) Two strategies are promising for the emergence of indirect reciprocity in the selective play environment: strict discriminator (GBBB) and extra standing (GBBG).  Their behaviors are almost identical. GBBB We do not have to adopt the unnatural interpretation of GBBB in the random matching environment.  Selective play is a promising alternative environment in order to investigate indirect reciprocity.

38. Discussion (continued) (3) The strategies that have been proposed to account for indirect reciprocity (e.g., image scoring, standing, strict discriminator, extra standing) are all similar in the sense that only discriminating altruism can hold (Hardin, 1981).  In order for altruism (indirect reciprocity) to emerge, it must take the form of discriminating altruism that have a mechanism that targets its altruistic acts predominantly toward other discriminating altruists.

40. Still remaining issues In order for indirect reciprocity to emerge, is the 2 nd order information really necessary? As long as we use this type of “score,” we have a problem of infinite regress, since we cannot define the scores on the first round.  How can we avoid this problem?