Multiagent Systems Game Theory © Manfred Huber 2018

Non-Cooperative Game Theory Game Theory is the study of the interaction and decision making of rational, self-interested agents Each agent has its own interests While these are mostly distinct, non-cooperative does not preclude that agents have common interests Agents are rational and only pursue their interests The individual agent is the basic unit Collaborative or coalitional game theory model teams as basic units © Manfred Huber 2018

Game Theory TCP backoff example Agents can use a correct or an incorrect TCP backoff mechanism Correct mechanism backs off after package collisions Incorrect mechanism does not increase backup over time Collisions of packets from multiple agents can lead to delays due to retries Correct implementation leads to an expected delay of 1 Incorrect implementation immediately retries, causing a correct implementation to back off further, leading to a delay of 4 Two incorrect implementations clash multiple times, leading to an expected delay of 3 TCP backoff game -1/-1 , -4/0, 0/-4, -3/-3 © Manfred Huber 2018

Game Theory The interaction and decision process of the agents is modeled as a game with separate utility functions for each agent What action should each player take ? Will both agents behave the same ? Do the precise numbers influence behavior ? Would communication change behavior ? Should actions change when a game is played repeatedly ? Is it important that agents are assumed to be rational ? © Manfred Huber 2018

Single Shot Games Single shot (or single stage) games are games where multiple agents have to make a single action choice. Past experiences have no influence on the current game and the agents’ decisions Utility is equal to the expected reward Finite n-person game N is a finite set of n agents (players) is the joint action set of the agents is the joint utility function for the agent © Manfred Huber 2018

Matrix Games A finite single stage game can be written in the form of an n-dimensional matrix (called “normal form”) Each dimension enumerates the actions of one agent Entries in the matrix are n-tuples of utility values for each agent for outcome of the given action choices The TCP backoff example Correct Incorrect -1, -1 -4, 0 0, -4 -3, -3 © Manfred Huber 2018

Matrix Games The prisoners’ dilemma Cooperate Stay silent -5, -5 -1, -10 -10, -1 -3, -3 © Manfred Huber 2018

Canonical Games Pure competition games are 2-player constant sum games Every utility gain for one agent is an equal utility loss for the other agents Only one utility function is needed Example: Matching Pennies Heads Tails 1, -1 -1, 1 © Manfred Huber 2018

Pure competition Games Example: Rock, Paper, Scissors Rock Paper Scissors 0, 0 -1, 1 1, -1 Compute utility under different assumptions © Manfred Huber 2018

Canonical Games Coordination games are games where players have exactly the same interests All players obtain the same utilities Only one utility function is needed Note: this is still non-cooperative since each agent makes its own decision Example: Driving on a side of the road Left Right 1, 1 -1, -1 © Manfred Huber 2018

Canonical Games General games contain elements of cooperation and competition Example: Battle of the sexes What happens if they feel differently strongly about different options ? Opera Football 3, 1 0, 0 1, 3 © Manfred Huber 2018

Decision-Making: Analyzing Games “Better” outcomes/strategies have to be defined Utilities for different agents can not be compared All agents are equally important Pareto dominance An outcome o Pareto-dominates an outcome o’ if for every agent outcome o is at least as good as outcome o’ © Manfred Huber 2018

Pareto Optimality An outcome/strategy is Pareto optimal if it is not Pareto dominated by any other outcome Every game has at least one Pareto optimal strategy Every game has at least one Pareto optimal pure strategy Games can have multiple Pareto optimal strategy © Manfred Huber 2018

Pareto Optimality Pareto Optimality allows to identify better strategies from an outside point of view. Examples: Heads Tails 1, -1 -1, 1 Left Right 1, 1 -1, -1 Cooperate Stay silent -5, -5 -1, -10 -10, -1 -3, -3 Opera Football 3, 1 0, 0 1, 3 © Manfred Huber 2018

Decision-Making: Best Response If an agent knows the strategies of all other agents it can easily pick its own action Best response: When the strategies of other agents are not known we have to look for “stable” strategies A strategy is a (pure strategy) Nash equilibrium if for every agent its action is a best response to the actions of the other agents © Manfred Huber 2018

Nash Equilibrium A Nash Equilibrium is a stable strategy Any agent deviating from it will not increase its utility An agent that is aware of the other agents’ strategies can not increase its utility using this knowledge (the equilibrium is its best strategy) In an equilibrium it is possible to determine the other agents’ strategies based on the assumption of rationality © Manfred Huber 2018

Nash Equilibrium Nash equilibria allow agents to identify stable strategies that lead to “best” outcomes. Examples: Heads Tails 1, -1 -1, 1 Left Right 1, 1 -1, -1 Cooperate Stay silent -5, -5 -1, -10 -10, -1 -3, -3 Opera Football 3, 1 0, 0 1, 3 © Manfred Huber 2018

Nash Equilibrium Many games do not have pure strategy equilibria In competitive settings it is often a bad idea to always do the same If other agents know your actions they can use this knowledge to utilize your weaknesses © Manfred Huber 2018

Mixed Strategies Agents can follow different strategies Pure strategies are strategies where agents select actions deterministically Mixed strategies are strategies where actions are taken according to a probability distribution Strategy profiles are the joint strategies of all the agents © Manfred Huber 2018

Utility of Mixed Strategies The utility of a mixed strategy profile can be computed as the expected value of the outcome lottery © Manfred Huber 2018

Best Response and Nash Equilibrium If an agent knows the strategies of all other agents it can pick its own (mixed) strategy Best response: When the strategies of other agents are not known we have to look for “stable” strategies A strategy profile is a Nash equilibrium if for every agent its strategy is a best response to the strategies of the other agents © Manfred Huber 2018

Nash Equilibrium A Strict Nash equilibrium is a Nash equilibrium where for each agent the strategy is a strictly better (higher utility) response to the other agents’ strategies than any other strategy A Weak Nash equilibrium is any Nash equilibrium that is not a Strict Nash equilibrium All mixed strategy equilibria are Weak © Manfred Huber 2018

Nash Equilibrium Nash (1951): Every finite game has at least one Nash equilibrium Computing Nash equilibria can be complex It becomes easier if the support of the strategy (i.e. the set of actions with probabilities greater than 0) is known If support is known then there is no regret for the pure actions involved – they have to have the same response value Heads Tails 1, -1 -1, 1 Opera Football 3, 1 0, 0 1, 3 © Manfred Huber 2018

Solution Approaches To address decision making in matrix games we need algorithms that can determine Nash equilibria Maxmin and Minmax for constant sum games Linear programming for (zero-sum) matrix games Linear Complimentarity Problem (LPC) for general 2-player games Lemke-Howson algorithm © Manfred Huber 2018