Download presentation
Presentation is loading. Please wait.
Published byDoddy Sasmita Modified over 6 years ago
1
Multiagent Systems Repeated Games © Manfred Huber 2018
2
Repeated Games Repeated games are an extension to normal form games where the same single-stage game is played multiple times sequentially What information is available to the agents ? Utility of a repeated game ? Strategies in repeated games ? There are two main types of repeated games Finitely-repeated games Infinitely-repeated games © Manfred Huber 2018
3
Finitely-Repeated Games
A fixed single-stage game is repeated a finite number of time After each game agents are aware what the other agents did in the previous stages Payoff for the repeated game is the sum of the payoffs in the stage games Finitely-repeated games can be translated into imperfect information games in extensive form © Manfred Huber 2018
4
Finitely-Repeated Games
Twice repeated Prisoners’ Dilemma C S -5, -5 -1, -10 -10, -1 -3, -3 C S -5, -5 -1, -10 -10, -1 -3, -3 1 2 C S (-5,-5) (-1,-10) (-10,-1) (-3,-3) 1 2 C S 1 2 C S 1 2 C S 1 2 C S 1 2 C S (-10, -10) (-6, -15) (-15, -6) (-8, -8) (-6, -15) (-2, -20) (-11, -11) (-4, -13) (-15, -6) (-11, -11) (-20, -2) (-13, -4) (-8, -8) (-4, -13) (-13, -4) (-6, -6) © Manfred Huber 2018
5
Strategies in Finitely-Repeated Games
Strategy space is richer than the strategy space for the single stage normal form game Strategies can depend on actions played in previous iterations of the game Stationary strategies are a special case where the strategy reduces to the same strategy for each single-stage game © Manfred Huber 2018
6
Nash Equilibria in Finitely-Repeated Games
Nash equilibria can be computed using the corresponding extensive form game Exponential increase in complexity with the number of repetitions A stationary strategy repeating a Nash equilibrium for the stage game is a Nash equilibrium for the repeated game In cases where the stage game has a dominant strategy, backward induction can be used to determine a Nash equilibrium © Manfred Huber 2018
7
Infinitely-Repeated Games
A fixed single-stage game is repeated an infinite number of times After each game agents are aware what the other agents did in the previous stages Can not be converted into an extensive form game Conversion would require an infinite tree Payoff for the repeated game can not be calculated as the sum of payoffs of the stage games Sum of payoffs is in general infinite (not a usable utility) © Manfred Huber 2018
8
Utility/Payoff in Infinitely-Repeated Games
Average reward: game payoff is measured as the average payoff of each stage game Future discounted reward: game payoff is measured as the discounted sum of payoff of the stage games Two interpretations of discounted reward: The agent cares less about future returns than current ones The game ends with probability 1-beta in each round © Manfred Huber 2018
9
Strategies and Equilibria in Infinitely-Repeated Games
A pure strategy defines an action choice for every stage game (decision point) Strategies are infinite There is an infinite number of pure strategies Famous strategies: Tit-for-tat Trigger strategy © Manfred Huber 2018
10
Nash Equilibria in Infinitely-Repeated Games
A stationary strategy repeating a Nash equilibrium for the stage game is again a Nash equilibrium for the repeated game Other equilibria can depend on the choice of payoff function and the discount parameter (in the case of future discounted reward) There can be an infinite number of pure strategy equilibria There is no way to construct the induced normal form or the sequence form for infinitely repeated games © Manfred Huber 2018
11
Computing Nash Equilibria in Infinitely-Repeated Games
Nash equilibria for infinitely repeating games can not be calculated by reducing the problem to a normal (or extensive) form game The payoff received in a Nash equilibrium of an infinitely repeating game can be characterized independently of the strategy by looking only at the stage game The average payoffs in an equilibrium have to be achievable under a mixed strategy in the stage game The average payoff achieved for player i in an equilibrium has to be at least as big as the one received if the other players adopt a minmax strategy against it © Manfred Huber 2018
12
Computing Nash Equilibria The Folk Theorem
Consider an n-player game G=(N,A,u) and a payoff vector r=(r1, …, rn) The minmax value of an agent is the value it can achieve if all other agents are antagonistic (i.e. attempt to minimize the agent’s payoff) minmax value: minmax represents a conservative value below which other agents can not push agent i ’s payoff © Manfred Huber 2018
13
Enforceability A payoff profile is enforceable if for every agent the payoff is at least as high as its minmax value © Manfred Huber 2018
14
Feasibility A payoff profile is feasible if it is a convex, rational (non-negative) combination of the outcomes of the stage game © Manfred Huber 2018
15
The Folk Theorem In any n-player infinitely repeated game with stage game G, the following holds: If r is the payoff profile for a Nash equilibrium in the infinitely repeated game with average rewards, then ri is enforceable If ri were not enforceable then agent i would receive a better payoff when pursuing the minmax strategy If r is feasible and enforceable then r is the payoff profile for some Nash equilibrium of the infinitely repeated game with average rewards If r is feasible and enforceable then we can construct a Nash equilibrium strategy in the form of a trigger strategy that follows the mixed strategy using the rational weights for enforcability and the minmax strategy after an agent diverts from the policy. © Manfred Huber 2018
16
The Folk Theorem Constructing Equilibrium Strategies
Find a payoff profile that is enforceable and feasible Find the rational coefficients for feasibility Construct a trigger strategy that performs the following: As long as no agent deviates from it, follow a pure strategy that repeatedly cycles through all pure strategies a of the stage game G, executing them βa times If in any cycle one of the agents deviates from this, the other agents will follow that agent’s minmax strategy for the rest of the game © Manfred Huber 2018
17
Infinitely-Repeated Games and Bounded Rationality
Solutions to infinitely repeated games assume: Arbitrarily deep reasoning Arbitrarily deep mutual modeling In real problems this is rarely feasible Discounted reward can limit the reliance on perfect future information Bounded rationality can address this problem Approximate ε-Nash equilibria Machine games: Games played by automata (limiting memory and strategy complexity) © Manfred Huber 2018
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.