Dynamic Games of complete information: Backward Induction and Subgame perfection - Repeated Games -

Strategic Behavior in Business and Econ Outline 3.1. What is a Game ? The elements of a Game The Rules of the Game: Example Examples of Game Situations Types of Games 3.2. Solution Concepts Static Games of complete information: Dominant Strategies and Nash Equilibrium in pure and mixed strategies Dynamic Games of complete information: Backward Induction and Subgame perfection

Strategic Behavior in Business and Econ Repeated Games A Repeated Game is a special case of a dynamic (sequential moves) game that consists of a (usually) static game being played several times, one after the other The game that is repeated is called the “stage game” The (stage) game can be played a given number of times (known to all the players) or an indefinite number of times.

Strategic Behavior in Business and Econ Repeated Games Thus, we can have a: Finitely Repeated Game. When the stage game is player a number T of rounds (1, 2, 3,..., T). T is known to all the players Infinitely Repeated Game When either After each round the game continues to the next round with probability p and ends with probability (1 – p) The game is played forever but at each round the value of the payoffs decreases by a factor of “p”

Strategic Behavior in Business and Econ Finitely Repeated Games Recall the Prisoners' Dilemma A “generic” version of the game is represented in the table Below Where, C stands for “cooperate” D stands for “defect”

Strategic Behavior in Business and Econ Finitely Repeated Games We saw that both players defecting is the unique equilibrium of the game. In fact, D is a Dominant Strategy for each player The apparently paradoxical behavior is that, although both Players would mutually benefit from Cooperation, self Interests leads to the worse outcome by Defecting

Strategic Behavior in Business and Econ Finitely Repeated Games We saw that both players defecting is the unique equilibrium of the game. In fact, D is a Dominant Strategy for each player Repeating the game opens interesting possibilities To “punish” egoistic (defect) behaviors To “reward” the right (cooperative) behavior

Strategic Behavior in Business and Econ Finitely Repeated Games We saw that both players defecting is the unique equilibrium of the game. In fact, D is a Dominant Strategy for each player Examples: “Stick and Carrot Strategies” (Trigger Strategies) 1. I will start with cooperation, and will mimic your behavior afterwards 2. I will start with cooperation and will keep doing so unless you defect. In such case I will defect forever

Strategic Behavior in Business and Econ The 2-times Repeated Prisoners' Dilemma The tree representation of the 2-times Repeated Prisoners' Dilemma is shown in the next slide: Notice: The “dotted” lines representing the simultaneous choice in each stage of the game The payoffs at the end of the game correspond to the sum of the payoffs in each stage Try to imagine the tree in a 3-times Repeated Prisoners' Dilemma

Strategic Behavior in Business and Econ C C C C C C C C C C C C C C C D D D D D D D D D D D D D D D 6, 6 3, 8 8, 3 4, 4 3, 8 0, 10 5, 5 1, 6 8, 3 5, 5 10, 0 6, 1 4, 4 1, 6 6, 1 2, 2 Stage 1Stage 2

Strategic Behavior in Business and Econ The 2-times Repeated Prisoners' Dilemma The game must be solved by Backward Induction using the Subgame Perfection technique (since there are “linked” nodes that indicate that the game is of Imperfect Information) Notice that his is always the case when we repeat a static game We must, therefore, “solve” each of the 4 “subgames” in the second stage of the game and then move backwards

Strategic Behavior in Business and Econ C C C C C C C C C C C C C C C D D D D D D D D D D D D D D D 6, 6 3, 8 8, 3 4, 4 3, 8 0, 10 5, 5 1, 6 8, 3 5, 5 10, 0 6, 1 4, 4 1, 6 6, 1 2, 2 Stage 1Stage 2 Subgame 1 Subgame 2 Subgame 3 Subgame 4

Strategic Behavior in Business and Econ Subgame 1Subgame 2 Subgame 4Subgame 3

Strategic Behavior in Business and Econ The 2-times Repeated Prisoners' Dilemma Notice that the solution in each subgame is always the same: Player 1: Defect Player 2:Defect And, again, Defect is a Dominant Strategy for each player in each subgame This is not a coincidence (as we will see shortly) Thus, proceeding backwards in the tree we get...

Strategic Behavior in Business and Econ C C C C C C C C C C C C C C C D D D D D D D D D D D D D D D 6, 6 3, 8 8, 3 4, 4 3, 8 0, 10 5, 5 1, 6 8, 3 5, 5 10, 0 6, 1 4, 4 1, 6 6, 1 2, 2 Stage 1Stage 2 Subgame 1 Subgame 2 Subgame 3 Subgame 4

Strategic Behavior in Business and Econ C C C D D D 4, 4 1, 6 6, 1 2, 2 Stage 1Stage 2 Again, what remains after we move backwards in the three is another “simultaneous move” game, the one that corresponds to the first stage of the game. We must “solve” in looking at the table representation

Strategic Behavior in Business and Econ C C C D D D 4, 4 1, 6 6, 1 2, 2 Stage 1Stage 2

Strategic Behavior in Business and Econ C C C D D D 4, 4 1, 6 6, 1 2, 2 Stage 1Stage 2 Thus, knowing what will be the outcome in the second stage of the game...

Strategic Behavior in Business and Econ C C C D D D 4, 4 1, 6 6, 1 2, 2 Stage 1Stage 2 Both players will also defect in the first round. (It's again a Dominant Strategy !)

Strategic Behavior in Business and Econ The T-times Repeated Prisoners' Dilemma Stage 1 (D, D) T=1 Stage 1 (D, D) T=2 Stage 2 (D, D) Stage 1 (D, D) T=3 Stage 2 (D, D) Stage 3 (D, D) Stage 1 (D, D) Any T Stage 2 (D, D) Stage 3 (D, D) Stage T (D, D) · ·

Strategic Behavior in Business and Econ The T-times Repeated Prisoners' Dilemma No matter how many times the Prisoners' Dilemma is repeated, the equilibrium is always the same: Defect in every round. Why don't punishments (rewards) work ? Intuition: At the last repetition, since the players know that there will not be a “new chance” (no punishment-reward is possible), the best thing to do is to Defect Knowing that, in the next-to-last round players know that in the next round the opponent will not cooperate. Then, why should I cooperate today if tomorrow my opponent is going to defect ? Again, the best thing to do is to Defect We can apply this argument “backwards” to conclude that the best thing to do is to Defect all the time.

Strategic Behavior in Business and Econ Finitely Repeated Games: General Facts Any Finitely Repeated Game that consists of the repetition of a stage game that is a static game produces a “Dynamic Game of Imperfect Information The tree representing such game is (usually) very large It should be solved by Backward Induction The following statements are always true in such games: If the stage game has a unique Nash Equilibrium, then the Finitely Repeated Game has unique Subgame Perfect Equilibrium consisting of the repetition of that Nash Equilibrium in every round of the game If the stage game has more than one Nash Equilibria, then any “reasonable” combination of those equilibria is a Subgame Perfect Equilibrium of the Finitely Repeated Game

Strategic Behavior in Business and Econ Infinitely Repeated Games We have seen that both players defecting is the unique subgame perfect equilibrium of the Finitely Repeated Prisoners' Dilemma Trigger Strategies do not lead to cooperation 1. I will start with cooperation, and will mimic your behavior afterwards (Tit-for-Tat) 2. I will start with cooperation and will keep doing so unless you defect. In such case I will defect forever (Grimm Trigger)

Strategic Behavior in Business and Econ There are two possible interpretations of games that are repeated but not a fixed number of rounds After each round the game continues to the next round with probability p and ends with probability (1 – p) Example: Two firms compete day after day, but there is certain probability that one of them goes bankrupt and then the game is over The game is played forever (an indefinite number of times) but at each round the value of the payoffs decreases by “p” Example: Two people negotiate with offers and counteroffers over an item. As time goes by, the item loses value. The game is over when they reach an agreement Infinitely Repeated Games

Strategic Behavior in Business and Econ The two different interpretations of a Infinitely Repeated Game are technically equivalent. Since there is no “last round”, there is no possibility of thinking backwards. This opens real opportunities to achieve cooperation in the Prisoners' Dilemma ! In general, Infinitely Repeated Games are very complex Infinitely Repeated Games

Strategic Behavior in Business and Econ Let x be any positive number (for instance, money) and p any positive number smaller than 1 (for instance, a probability). Then, x·p + x·p 2 + x·p 3 + x·p 4 + · · · = x x·p 2 + x·p 3 + x·p 4 + x·p 5 · · · = x and so on... Mathematical aside (infinite sums) p (1 - p) p2p2

Strategic Behavior in Business and Econ The Infinitely Repeated Prisoners' Dilemma How cooperation can be sustained in equilibrium ? Payoff Computation: Imagine that Player 2 plays Grimm Trigger What is the (expected) payoff for Player 1 if after each round the game continues with probability p (and ends with probability (1-p)) If Player 1 plays “cooperate” all the time Stage 1Stage 2Stage 3 Stage 4 · · · 33·p + 0·(1-p)3·p 2 + 0·(1-p 2 ) · · · ·= 3 +3p + 3p 2 + 3p 3 + · · · If Player 1 plays “defect” all the time Stage 1Stage 2Stage 3 Stage 4 · · · 51·p + 0·(1-p)1·p 2 + 0·(1-p 2 ) · · · ·= 5 + 1p + 1p 2 + 1p 3 + · · ·

Strategic Behavior in Business and Econ The Infinitely Repeated Prisoners' Dilemma How cooperation can be sustained in equilibrium ? Payoff Computation: Imagine that Player 2 plays Grimm Trigger What is the payoff for Player 1 if after each round the value of the money decreases by a factor of p (for instance, if the money decreases a 10% then p=0.9) Playing “cooperate” all the time Stage 1Stage 2Stage 3 Stage 4 · · · 33·p 3·p 2 · · · ·= 3 + 3p + 3p 2 + · · · 3 3·(0.9)3·(0.9)·(0.9)= 3 + 3·(0.9) + 3·(0.9) 2 + · ·.= * · · · Playing “defect” all the time Stage 1Stage 2Stage 3 Stage 4 · · · 51·p 1·p 2 · · · ·= 5 + 1p + 1p 2 + · · · 5 1·(0.9)1·(0.9)·(0.9)= 5 + 1·(0.9) + 1·(0.9) 2 + · · = · · ··

Strategic Behavior in Business and Econ The Infinitely Repeated Prisoners' Dilemma How cooperation can be sustained in equilibrium ? Imagine that Player 2 plays Grimm Trigger What is the best for Player 1, Cooperate or Defect ? Expected Payoff from “cooperate” all the time Stage 1Stage 2Stage 3 Stage 4 · · · 33·p + 0·(1-p)3·p 2 + 0·(1-p 2 ) · · · ·= 3 + 3p + 3p 2 + · · · Expected Payoff from “defect” all the time Stage 1Stage 2Stage 3 Stage 4 · · · 51·p + 0·(1-p)1·p 2 + 0·(1-p 2 ) · · · ·= 5 + 1p + 1p 2 + · · ·

Strategic Behavior in Business and Econ The Infinitely Repeated Prisoners' Dilemma How cooperation can be sustained in equilibrium ? Imagine that Player 2 plays Grimm Trigger What is the best for Player 1, Cooperate or Defect ? Expected Payoff from “cooperate” all the time E(Cooperate) = 3 + 3p + 3p 2 + 3p 3 + · · · = Expected Payoff from “defect” all the time E(Defect) = 5 + 1p + 1p 2 + 1p 3 + · · · = p (1 - p) p

Strategic Behavior in Business and Econ The Infinitely Repeated Prisoners' Dilemma How cooperation can be sustained in equilibrium ? Imagine that Player 2 plays Grimm Trigger What is the best for Player 1, Cooperate or Defect ? E(Cooperate) = E(Defect) = Cooperate will be better if E(Cooperate) > E(Defect), that is, if > p > ½ p (1 - p) p p p

Strategic Behavior in Business and Econ The Infinitely Repeated Prisoners' Dilemma How cooperation can be sustained in equilibrium ? Thus, cooperation can be sustained in equilibrium in the Infinitely Repeated Prisoners' Dilemma thanks to Trigger Strategies Depends on “p” With Tit-for-Tat it is also possible to sustain cooperation, but then p > 2/3 But, “cooperation” is not the unique equilibrium. There are equilibria with “defection” as well

Strategic Behavior in Business and Econ Infinitely Repeated Games: General Facts Any Infinitely Repeated Game that consists of the repetition of a stage game that is a static game produces a “Dynamic Game of Imperfect Information” The tree representing such game is (usually) very large It can not be solved by Backward Induction The following statement is always true in such games: No matter how many equilibria the stage game has, any “reasonable” combination of strategies can be a Subgame Perfect Equilibrium in such games (Folk Theorem)

Strategic Behavior in Business and Econ Infinitely Repeated Games: General Facts What does “reasonable” mean ? Notice that by playing Defect all the time any player can guarantee himself a payoff of at least 1 per each round. Thus, any “reasonable” outcome of the game should pay each player at least 1 per round

Strategic Behavior in Business and Econ Summary Any Repeated Game that consists of the repetition of a stage game that is a static game produces a “Dynamic Game of Imperfect Information” The tree representing such game is very large If the game is Finitely Repeated, it must be solved by Backward Induction If the game is Infinitely Repeated, it can not be solved by Backward Induction

Strategic Behavior in Business and Econ The following statements are always true in Finitely Repeated Games:: If the stage game has a unique Nash Equilibrium, then the Finitely Repeated Game has unique Subgame Perfect Equilibrium consisting of the repetition of that Nash Equilibrium in every round of the game If the stage game has more than one Nash Equilibria, then any “reasonable” combination of those equilibria is a Subgame Perfect Equilibrium of the Finitely Repeated Game The following statement is always true in Infinitely Repeated Games: No matter how many equilibria the stage game has, any “reasonable” combination of strategies can be a Subgame Perfect Equilibrium in such games (Folk Theorem) Summary

Strategic Behavior in Business and Econ Axelrod's Simulation R. Axelrod, The Evolution of Cooperation Prisoner’s Dilemma repeated 200 times Economists submitted strategies Pairs of strategies competed Winner: Tit-for-Tat Reasons: Forgiving, Nice, Clear

Strategic Behavior in Business and Econ Not necessarily tit-for-tat Doesn’t always work Don’t be envious Don’t be the first to cheat Reciprocate opponent’s behavior Cooperation and defection Don’t be too clever To be credible, incorporate a clear policy of punishment Lessons from Axelrod’s Simulation