1. Introduction to Cognition and Gaming 9/8/02: Iterated Prisoner’s Dilemma

2. The Problem of Common Pastures Village has a common pasture Ten farmers, each with one cow Enough grass so that the 10 cows get fat Soon one farmer can afford an additional cow – little change is noticed The second and third farmers each buy another cow - all cows become visibly thinner The seventh farmer buys a second cow, the seventeen cows are now so thin they’re less valuable than the original ten The tenth farmer buys a second cow – all twenty cows starve to death

3. The Problem of Common Pastures Majority buys another cow Majority does not buy another cow You buy another cow 2, 2 I have two thin cows 4, 1 I have two fat cows You do not buy another cow 1, 4 I have one super-thin cow 3, 3 I have one super-fat cow

4. Iterated Prisoner’s Dilemmas In a one-shot prisoner’s dilemma, strategies don’t need to consider retaliation If I confess and my accomplice doesn’t, then I don’t need to worry about revenge for ten years If we both confess, my accomplice will have nothing to avenge when we’re set free in 5 years Now, we need to consider trust to a greater degree, for your accomplice can retaliate much sooner Complex long-term strategies are now available

5. The Flood- Dresher Experiment An experiment performed at RAND to test iterated prisoner’s dilemmas Two friends, Armen Alchain of UCLA, and John D. Williams of RAND were recruited as guinea pigs

6. The Payoffs AA cooperatesAA competes JW cooperates 3, 30, 5 JW competes 5, 01, 1

7. Initial comments: AA: JW will defect – sure win. Hence if I cooperate, I lose JW: Hope he’s bright Armen Alchain vs. John D. Williams

8. GameAAJW 1DefectCooperate What is he doing?!! He isn’t but maybe he’ll wise up

9. Armen Alchain vs. John D. Williams GameAAJW 2DefectCooperate Trying mixed?Okay, dope

10. Armen Alchain vs. John D. Williams GameAAJW 3Defect Has he settled on Defect? Okay, dope

11. Armen Alchain vs. John D. Williams GameAAJW 4Defect Perverse!It isn’t the best of all possible worlds

12. Armen Alchain vs. John D. Williams GameAAJW 5CooperateDefect I’m sticking to Defect since he will mix for at least 4 more times Oh ho! Guess I’ll have to give him another chance

13. Armen Alchain vs. John D. Williams GameAAJW 6DefectCooperate Cagey, ain’t he? Well…

14. Armen Alchain vs. John D. Williams GameAAJW 7DefectCooperate In time he could learn, but not in 10 moves, so…

15. Armen Alchain vs. John D. Williams GameAAJW 8Defect If I mix occasionally, he will switch, but why will he ever switch from Defect?

16. Armen Alchain vs. John D. Williams GameAAJW 9Defect Prediction. He will stick with Defect until I change from Defect. I feel like DuPont I have control of the game to a large extent, so AA had better appreciate this and get on the bandwagon

17. Armen Alchain vs. John D. Williams GameAAJW 10Defect Probably learned by now

18. Armen Alchain vs. John D. Williams GameAAJW 11DefectCooperate I’ll be damned! But I’ll try again

19. Armen Alchain vs. John D. Williams GameAAJW 12Cooperate That’s better

20. Armen Alchain vs. John D. Williams GameAAJW 13Cooperate Ha!

21. Armen Alchain vs. John D. Williams GameAAJW 14Cooperate (bliss)

22. Armen Alchain vs. John D. Williams GameAAJW 15Cooperate

23. Armen Alchain vs. John D. Williams GameAAJW 16DefectCooperate The stinker

24. Armen Alchain vs. John D. Williams GameAAJW 17CooperateDefect He’s crazy. I’ll teach him the hard way.

25. Armen Alchain vs. John D. Williams GameAAJW 18CooperateDefect I’m completely confused. Is he trying to convey information to me? Let him suffer.

26. Armen Alchain vs. John D. Williams GameAAJW 19Defect

27. Armen Alchain vs. John D. Williams GameAAJW 20Defect Maybe he’ll be a good boy now

28. Armen Alchain vs. John D. Williams GameAAJW 21DefectCooperate Always takes time to learn

29. Armen Alchain vs. John D. Williams GameAAJW 22Cooperate Time

30. Armen Alchain vs. John D. Williams GameAAJW 23Cooperate

31. Armen Alchain vs. John D. Williams GameAAJW 24Cooperate

32. Armen Alchain vs. John D. Williams GameAAJW 25Cooperate

33. Armen Alchain vs. John D. Williams GameAAJW 26DefectCooperate Same old story

34. Armen Alchain vs. John D. Williams GameAAJW 27CooperateDefect To hell with him

35. Armen Alchain vs. John D. Williams GameAAJW 28Defect

36. Armen Alchain vs. John D. Williams GameAAJW 29Defect

37. Armen Alchain vs. John D. Williams GameAAJW 30Defect Some startOnce again

38. Armen Alchain vs. John D. Williams GameAAJW 31DefectCooperate JW is bent on sticking with Defect. He will not share at all as a price of getting me to stick to Cooperate #$&% he learns slow!

39. Armen Alchain vs. John D. Williams GameAAJW 32Cooperate On the beam again

40. Armen Alchain vs. John D. Williams GameAAJW 33Cooperate

41. Armen Alchain vs. John D. Williams GameAAJW 34Cooperate

42. Armen Alchain vs. John D. Williams GameAAJW 35Cooperate

43. Armen Alchain vs. John D. Williams GameAAJW 36Cooperate

44. Armen Alchain vs. John D. Williams GameAAJW 37Cooperate

45. Armen Alchain vs. John D. Williams GameAAJW 38DefectCooperate The #@$%er!

46. Armen Alchain vs. John D. Williams GameAAJW 39CooperateDefect

47. Armen Alchain vs. John D. Williams GameAAJW 40Defect Always try to be virtuous

48. Armen Alchain vs. John D. Williams GameAAJW 41DefectCooperate Old stuff

49. Armen Alchain vs. John D. Williams GameAAJW 42Cooperate

50. Armen Alchain vs. John D. Williams GameAAJW 43Cooperate

51. Armen Alchain vs. John D. Williams GameAAJW 44Cooperate

52. Armen Alchain vs. John D. Williams GameAAJW 45Cooperate

53. Armen Alchain vs. John D. Williams GameAAJW 46Cooperate

54. Armen Alchain vs. John D. Williams GameAAJW 47Cooperate

55. Armen Alchain vs. John D. Williams GameAAJW 48Cooperate He will not share

56. Armen Alchain vs. John D. Williams GameAAJW 49DefectCooperate He’s a shady character and doesn’t realize we’re playing a 3 rd party, not each other

57. Armen Alchain vs. John D. Williams GameAAJW 50CooperateDefect

58. Armen Alchain vs. John D. Williams GameAAJW 51DefectCooperate He requires great virtue but he doesn’t have it himself

59. Armen Alchain vs. John D. Williams GameAAJW 52Cooperate

60. Armen Alchain vs. John D. Williams GameAAJW 53Cooperate

61. Armen Alchain vs. John D. Williams GameAAJW 54Cooperate

62. Armen Alchain vs. John D. Williams GameAAJW 55Cooperate

63. Armen Alchain vs. John D. Williams GameAAJW 56Cooperate

64. Armen Alchain vs. John D. Williams GameAAJW 57Cooperate He will not share

65. Armen Alchain vs. John D. Williams GameAAJW 58Cooperate He does not want to trick me. He is satisfied. I must teach him to share

66. Armen Alchain vs. John D. Williams GameAAJW 59Cooperate

67. Armen Alchain vs. John D. Williams GameAAJW 60DefectCooperate A shiftless individual - opportunist, knave

68. Armen Alchain vs. John D. Williams GameAAJW 61Cooperate Goodness me! Friendly!

69. Armen Alchain vs. John D. Williams GameAAJW 62Cooperate

70. Armen Alchain vs. John D. Williams GameAAJW 63Cooperate

71. Armen Alchain vs. John D. Williams GameAAJW 64Cooperate

72. Armen Alchain vs. John D. Williams GameAAJW 65Cooperate

73. Armen Alchain vs. John D. Williams GameAAJW 66Cooperate He won’t share

74. Armen Alchain vs. John D. Williams GameAAJW 67DefectCooperate He’ll punish for trying! He can’t stand success

75. Armen Alchain vs. John D. Williams GameAAJW 68CooperateDefect

76. Armen Alchain vs. John D. Williams GameAAJW 69Defect I’ll try once more to share - by taking

77. Armen Alchain vs. John D. Williams GameAAJW 70Defect This is like toilet training a child - you have to be very patient

78. Armen Alchain vs. John D. Williams GameAAJW 71DefectCooperate

79. Armen Alchain vs. John D. Williams GameAAJW 72Cooperate

80. Armen Alchain vs. John D. Williams GameAAJW 73Cooperate

81. Armen Alchain vs. John D. Williams GameAAJW 74Cooperate

82. Armen Alchain vs. John D. Williams GameAAJW 75Cooperate

83. Armen Alchain vs. John D. Williams GameAAJW 76Cooperate

84. Armen Alchain vs. John D. Williams GameAAJW 77Cooperate

85. Armen Alchain vs. John D. Williams GameAAJW 78Cooperate

86. Armen Alchain vs. John D. Williams GameAAJW 79Cooperate Well.

87. Armen Alchain vs. John D. Williams GameAAJW 80Cooperate

88. Armen Alchain vs. John D. Williams GameAAJW 81DefectCooperate He needs to be taught about that

89. Armen Alchain vs. John D. Williams GameAAJW 82CooperateDefect

90. Armen Alchain vs. John D. Williams GameAAJW 83Cooperate

91. Armen Alchain vs. John D. Williams GameAAJW 84Cooperate

92. Armen Alchain vs. John D. Williams GameAAJW 85Cooperate

93. Armen Alchain vs. John D. Williams GameAAJW 86Cooperate

94. Armen Alchain vs. John D. Williams GameAAJW 87Cooperate

95. Armen Alchain vs. John D. Williams GameAAJW 88Cooperate

96. Armen Alchain vs. John D. Williams GameAAJW 89Cooperate

97. Armen Alchain vs. John D. Williams GameAAJW 90Cooperate When will he make a last minute grad of Defect? Can I beat him to it as late as possible?

98. Armen Alchain vs. John D. Williams GameAAJW 91Cooperate Good

99. Armen Alchain vs. John D. Williams GameAAJW 92Cooperate

100. Armen Alchain vs. John D. Williams GameAAJW 93Cooperate

101. Armen Alchain vs. John D. Williams GameAAJW 94Cooperate

102. Armen Alchain vs. John D. Williams GameAAJW 95Cooperate

103. Armen Alchain vs. John D. Williams GameAAJW 96Cooperate

104. Armen Alchain vs. John D. Williams GameAAJW 97Cooperate

105. Armen Alchain vs. John D. Williams GameAAJW 98Cooperate

106. Armen Alchain vs. John D. Williams GameAAJW 99DefectCooperate

107. Armen Alchain vs. John D. Williams GameAAJW 100Defect

108. Final Score AA: 268 JW: 244 Mutual cooperation: 49 times Comments?

109. The Best Strategy? Is it theoretically possible for cooperation to develop in a world where everybody is governed by their own self interests? In 1979, political scientist Robert Axelrod asked several well-known scientists to participate in a competition Send in a computer program of (what they consider to be) an optimal strategy Programs played against each other in a round-robin style

110. Axelrod’s Competitions Program 2 cooperates Program 2 competes Program 1 cooperates 3, 30, 5 Program 2 competes 5, 01, 1

111. Axelrod’s Competitions Number of rounds per ‘battle’ was not revealed to the programmers Strategy with highest cumulative score would be declared the winner Twelve programs entered, some were highly sophisticated Three additional programs - ALL C, ALL D, RANDOM

112. The Winner? Submitted by famous social psychologist Anatol Rapoport Simplest of the twelve 1) Cooperate in the first round 2) Thereafter, do whatever the opponent did in the previous round Called “Tit-for-Tat”, known as TFT in game theory literature

113. Invincible? Axelrod sought to find a strategy to beat TFT Three were found – one which was highly competitive, another was even more “forgiving” than TFT, so what’s the best choice? Axelrod ran a second competition, having already published the first

114. The Second Competition 62 programs from 6 countries and 8 fields of science entered Axelrod submitted the three programs that beat TFT Rapoport entered TFT a second time Rapoport won again! The program that was more forgiving ranked 21 st, the competitive program ranked in the bottom 50%

115. Traits of Winning Programs Niceness – Program wouldn’t initiate competition Forgiveness – Program wouldn’t hold a grudge Provocability – Willingness to retaliate Responsiveness – Taking the opponent’s strategy into consideration Transparency - Length

116. Is TFT the Perfect Strategy? Rapoport himself warned not to overvalue TFT too much Sometimes it reacts too violently, there are many social situations where TFT is too harsh E.g., an “innocent” misunderstanding can lead to eternal competition, unless the opponent includes an “out of turn” pardon

117. The Moral It’s good to be good

118. End