Robert Axelrod’s Tournaments Robert Axelrod’s Tournaments, as reported in Axelrod, Robert. 1980a. “Effective Choice in the Prisoner’s Dilemma.” Journal of Conflict Resolution 24: Axelrod, Robert. 1980b. “More Effective Choice in the Prisoner’s Dilemma.” Journal of Conflict Resolution 24 (3): Axelrod, Robert Evolution of Cooperation. Robert Axelrod’s Tournaments Robert Axelrod’s Tournaments, as reported in Axelrod, Robert. 1980a. “Effective Choice in the Prisoner’s Dilemma.” Journal of Conflict Resolution 24: Axelrod, Robert. 1980b. “More Effective Choice in the Prisoner’s Dilemma.” Journal of Conflict Resolution 24 (3): Axelrod, Robert Evolution of Cooperation.

Tournament Num. 1 Tournament Num. 1 (1980) -non-zero sum setting, given payoff matrix (R=3, T=5, S=0, P=1) -round robin tournament (play all other entrants, twin, and RANDOM) -each entrant told to write a program to select C or D choice every move, can use history of the game so far in this decision making -sent copies of preliminary tournament in which TFT scored second, so known to be powerful competitor, also told RANDOM was somewhere in the competition  tried to improve on TFT principle -known number of moves per game: 200 -entire round robin run 5 times  total 120,000 moves and 240,000 choices Tournament Num. 1 Tournament Num. 1 (1980) -non-zero sum setting, given payoff matrix (R=3, T=5, S=0, P=1) -round robin tournament (play all other entrants, twin, and RANDOM) -each entrant told to write a program to select C or D choice every move, can use history of the game so far in this decision making -sent copies of preliminary tournament in which TFT scored second, so known to be powerful competitor, also told RANDOM was somewhere in the competition  tried to improve on TFT principle -known number of moves per game: 200 -entire round robin run 5 times  total 120,000 moves and 240,000 choices

14 Entrants -3 countries, 5 disciplines (psychology, math, economics, sociology, political sciences) -scores range from 0 to 1000, but “useful benchmark for very good performance is 600,” attained if both always cooperate together -“very poor performance [benchmark] is 200 points” (if both always D) -winner Tit for Tat (TFT) scored 504 (but if change P=2, does not win) -top 8 entries were nice (defined as not first to defect), rest were not -nice entries’ scores scored from 472 to 504, while best of mean entries only scored 401 points (huge disparity!) -logically, because nice ones cooperate together, this is how TFT wins! (though it cannot get a score higher than its opponent’s) 14 Entrants -3 countries, 5 disciplines (psychology, math, economics, sociology, political sciences) -scores range from 0 to 1000, but “useful benchmark for very good performance is 600,” attained if both always cooperate together -“very poor performance [benchmark] is 200 points” (if both always D) -winner Tit for Tat (TFT) scored 504 (but if change P=2, does not win) -top 8 entries were nice (defined as not first to defect), rest were not -nice entries’ scores scored from 472 to 504, while best of mean entries only scored 401 points (huge disparity!) -logically, because nice ones cooperate together, this is how TFT wins! (though it cannot get a score higher than its opponent’s)

14 Entrants -important to be nice and forgiving -2 kingmakers (defined as players who do not do well themselves but “LARGELY determine the rankings among the top contenders”): GRAASKAMP and DOWNING -DOWNING most important kingmaker since it had the largest range of scores achieved with the nice rules, important to note DOWNING was not based on TFT principle -now to look at the actual results!, then to examen the strategies, since strategies aside from TFT are just denoted by name of creator 14 Entrants -important to be nice and forgiving -2 kingmakers (defined as players who do not do well themselves but “LARGELY determine the rankings among the top contenders”): GRAASKAMP and DOWNING -DOWNING most important kingmaker since it had the largest range of scores achieved with the nice rules, important to note DOWNING was not based on TFT principle -now to look at the actual results!, then to examen the strategies, since strategies aside from TFT are just denoted by name of creator

STRATEGIES! 1. Tit for Tat (TFT)- winner with points, from Toronto (psychology), as we all know- cooperates on first move, then does what opponent did last move, “eye for eye” style, 4 lines FORTRAN 2. TIDEMAN and CHIERUZZI points, from US (Economics), begins with cooperation/ TFT, but after opponent finishes second run of D, institutes extra punishment  increases number of punishments (D) by 1 with each run of opponent’s defections, then decides whether to give opponent a fresh start and begin with TFT again based on- if it has 10+ points more than opponent, opponent has not started another run of D’s, been 20+ moves since last fresh start, are 10+ moves left, number of opponent’s D’s “differs from generator by at least 3 standard deviations,” 41 lines of code STRATEGIES! 1. Tit for Tat (TFT)- winner with points, from Toronto (psychology), as we all know- cooperates on first move, then does what opponent did last move, “eye for eye” style, 4 lines FORTRAN 2. TIDEMAN and CHIERUZZI points, from US (Economics), begins with cooperation/ TFT, but after opponent finishes second run of D, institutes extra punishment  increases number of punishments (D) by 1 with each run of opponent’s defections, then decides whether to give opponent a fresh start and begin with TFT again based on- if it has 10+ points more than opponent, opponent has not started another run of D’s, been 20+ moves since last fresh start, are 10+ moves left, number of opponent’s D’s “differs from generator by at least 3 standard deviations,” 41 lines of code

STRATEGIES! 3. NYDEGGER points, starts with TFT for first 3 moves unless it was only one to C on first move and only one to D on second move, then it will D on third move, after third move- it chooses based on a complex weighted sum (2 points for opponent’s D, 1 point for own D, then weight this sum for past three terms- 16 for last term, then 4, then 1; if sum = 63, i.e. three turns of mutual defection  it will C) 4. GROFMAN points, always cooperates unless players did not do the same thing on the last move, then cooperates with prob 2/7 5. SHUBIK pts, cooperates until opponent plays D, then it defects once, if other defects again- it begins again with cooperation, in general- “length of retaliation is increased by one for each departure from mutual cooperation” STRATEGIES! 3. NYDEGGER points, starts with TFT for first 3 moves unless it was only one to C on first move and only one to D on second move, then it will D on third move, after third move- it chooses based on a complex weighted sum (2 points for opponent’s D, 1 point for own D, then weight this sum for past three terms- 16 for last term, then 4, then 1; if sum = 63, i.e. three turns of mutual defection  it will C) 4. GROFMAN points, always cooperates unless players did not do the same thing on the last move, then cooperates with prob 2/7 5. SHUBIK pts, cooperates until opponent plays D, then it defects once, if other defects again- it begins again with cooperation, in general- “length of retaliation is increased by one for each departure from mutual cooperation”

STRATEGIES! 6. STEIN pts, TFT except it cooperates always first four moves and defects on last 2 moves (move 199 and 200 of game), every 15 moves checks to see if opponent is RANDOM with chi-squared test of opponent’s transition probabilities and alternating CD/DC moves 7. FRIEDMAN pts, cooperates until opponent defects, then it defects forever 8. DAVIS pts, last of the nice guys, cooperates first 10 moves, then if there is a defection, it will defect forever 9. GRAASKAMP pts, one of kingmakers, TFT for 50 moves, defects on move 51, then plays 5 more TFT, check to see if opponent is RANDOM, if so- D from then on (also checks for TFT, ANALOGY, CLONE), otherwise- randomly defects every 5-15 moves, enough trust STRATEGIES! 6. STEIN pts, TFT except it cooperates always first four moves and defects on last 2 moves (move 199 and 200 of game), every 15 moves checks to see if opponent is RANDOM with chi-squared test of opponent’s transition probabilities and alternating CD/DC moves 7. FRIEDMAN pts, cooperates until opponent defects, then it defects forever 8. DAVIS pts, last of the nice guys, cooperates first 10 moves, then if there is a defection, it will defect forever 9. GRAASKAMP pts, one of kingmakers, TFT for 50 moves, defects on move 51, then plays 5 more TFT, check to see if opponent is RANDOM, if so- D from then on (also checks for TFT, ANALOGY, CLONE), otherwise- randomly defects every 5-15 moves, enough trust

STRATEGIES! 10. DOWNING , main kingmaker, starts with D since assumes opponent is unresponsive (i.e. initially assumes 1/2 for conditional probabilities, its downfall!), from then on- assesses and updates probabilities (that opponent cooperates if DOWNING defects, etc) to calculate choice to maximize its long-term expected payoff, if the 2 conditional probabilities have similar values- DOWNING determines pays to D, conversely- if opponent is responsive (much more likely to play C after DOWNING plays C than after D), then it will cooperate 11. FELD pts, starts with TFT, gradually lowers probability of C following the other plays C to 1/2 by the 200th move 12. JOSS , cooperates 90% after opponent’s C, always D after D 13. TULLOCK , cooperates first 11 moves, then cooperates 10% less than opponent has on preceding 10 moves STRATEGIES! 10. DOWNING , main kingmaker, starts with D since assumes opponent is unresponsive (i.e. initially assumes 1/2 for conditional probabilities, its downfall!), from then on- assesses and updates probabilities (that opponent cooperates if DOWNING defects, etc) to calculate choice to maximize its long-term expected payoff, if the 2 conditional probabilities have similar values- DOWNING determines pays to D, conversely- if opponent is responsive (much more likely to play C after DOWNING plays C than after D), then it will cooperate 11. FELD pts, starts with TFT, gradually lowers probability of C following the other plays C to 1/2 by the 200th move 12. JOSS , cooperates 90% after opponent’s C, always D after D 13. TULLOCK , cooperates first 11 moves, then cooperates 10% less than opponent has on preceding 10 moves

Last of STRATEGIES! 14. GRADUATE STUDENT NAME WITHHELD pts, starts with probability of C of 30%, which is updated every 10 moves if opponent seems very cooperative, very uncooperative, or random, after 130 moves if losing- probability is adjusted, this complex process kept P between 30% and 70%, making it seem random to most opponents 15. RANDOM pts, C with probability 1/2 and D with probability 1/2 (C and D with equal probabilities) Last of STRATEGIES! 14. GRADUATE STUDENT NAME WITHHELD pts, starts with probability of C of 30%, which is updated every 10 moves if opponent seems very cooperative, very uncooperative, or random, after 130 moves if losing- probability is adjusted, this complex process kept P between 30% and 70%, making it seem random to most opponents 15. RANDOM pts, C with probability 1/2 and D with probability 1/2 (C and D with equal probabilities)

Tournament Num. 2 Tournament Num. 2 (1980) -same non-zero sum setting, again round robin tournament (play all) -each entrant was sent report of first tournament, given same task -instead of known number of moves per game, “length of the game was determined probabilistically with chance of ending with each given move” (one way to include w), w chosen so expected median length = 200 moves (w = in second tournament) -average length turned out to be shorter: closer to 150 moves -endgame effects successfully avoided this time -features of entries do not relate to success (length of program, type, nationality, type of program, etc) Tournament Num. 2 Tournament Num. 2 (1980) -same non-zero sum setting, again round robin tournament (play all) -each entrant was sent report of first tournament, given same task -instead of known number of moves per game, “length of the game was determined probabilistically with chance of ending with each given move” (one way to include w), w chosen so expected median length = 200 moves (w = in second tournament) -average length turned out to be shorter: closer to 150 moves -endgame effects successfully avoided this time -features of entries do not relate to success (length of program, type, nationality, type of program, etc)

63 Entrants -6 countries, contests largely recruited via journals, etc -everyone from first tournament re-invited, entrants ranged from 11 year-old Steve Newman to professors from many disciplines, including computer science and evolutionary biology this time -more than half of entries were nice, Tit for Tat (TFT) won again -Tit for Two Tats- too forgiving, suggested post-Tourney 1, submitted Tourney 2 by evolutionary biologist, ended up in bottom half of group -5 representative rules can predict how a given rule did with the 63 rules- GRAASKAMP & KATZEN (S 6 ), PINKLEY (S 30 ), ADAMS (S 35 ), GLADSTEIN (S 46 ), and FEATHERS (S 27 )  predicted tournament score T = (.202) S 6 + (.198) S 30 + (.110) S 35 + (.072) S 46 + (.086) S Entrants -6 countries, contests largely recruited via journals, etc -everyone from first tournament re-invited, entrants ranged from 11 year-old Steve Newman to professors from many disciplines, including computer science and evolutionary biology this time -more than half of entries were nice, Tit for Tat (TFT) won again -Tit for Two Tats- too forgiving, suggested post-Tourney 1, submitted Tourney 2 by evolutionary biologist, ended up in bottom half of group -5 representative rules can predict how a given rule did with the 63 rules- GRAASKAMP & KATZEN (S 6 ), PINKLEY (S 30 ), ADAMS (S 35 ), GLADSTEIN (S 46 ), and FEATHERS (S 27 )  predicted tournament score T = (.202) S 6 + (.198) S 30 + (.110) S 35 + (.072) S 46 + (.086) S 27