Game Dynamics Out of Sync Michael Schapira (Yale University and UC Berkeley) Joint work with Aaron D. Jaggard and Rebecca N. Wright

Motivation: Internet Routing Establish routes between Autonomous Systems (ASes). Currently handled by the Border Gateway Protocol (BGP). AT&T Qwest Comcast Sprint

Internet Routing as a Game [Levin-S-Zohar] Internet routing is a game! –players = ASes –players’ types = preferences over routes –strategies = outgoing edges BGP = Best-Response Dynamics –each AS constantly selects its best available route to each destination –… until a “stable state” (= PNE) is reached.

But… Challenge I: No synchronization of players’ actions –players can best-reply simultaneously. –players can best-reply based on outdated information. Challenge II: Are players incentivized to follow best-response dynamics? –Can a player benefit from not best-replying? this talk [Nisan-S-Valiant-Zohar]

Game Dynamics and Asynchrony Dynamic environments –Internet protocols –large-scale markets –social networks –multi-processor computer architectures Game theory provides useful tools to analyze these interactions, but…. … has so far primarily concentrated on synchronous environments (simultaneous, sequential).

2,12,1 0,00,0 1,21,2 0,00,0 Row Player Column Player Illustration

2,12,1 0,00,0 1,21,2 0,00,0 Row Player Column Player

But… 2,12,1 0,00,0 1,21,2 0,00,0 Row Player Column Player

Model for asynchronous game dynamics Impossibility result Circumventing our impossibility result Complexity of asynchronous game dynamics Directions for future research Agenda

n nodes 1,…,n Node i has action space A i –A=A 1 … A n –A -i =A 1 … A i-1 A i+1 … A n Node i has reaction function f i :A → A i –f=(f 1,…,f n ) Simple Model: Nodes Interacting

Infinite sequence of discrete time steps t=1,… Initial state a 0, Schedule  :{1,…} → 2 [n] –fair schedule The (a 0,  )-dynamics –Start at the initial state a 0 –In each time step t let the nodes in  (t) react. Simple Model: Dynamics

Defn: an action profile a=(a 1,…,a n ) is a stable state if f i (a)=a i for all i. –that is, a is a fixed point of f. Defn: The system is convergent if the (a 0,  )-dynamics converges to a stable state for all choices of a 0 and (fair) . Simple Model: Convergence

Defn: f is node independent if, for each node i, f i :A -i → A i Thm: If f is node independent, and there exist multiple stable states, then the system is not convergent. Can be generalized to reaction functions that are –randomized –bounded-recall –non-stationary Guaranteed Convergence?

Internet protocols –Internet routing [Sami-S-Zohar] –congestion control [Godfrey-S-Zohar-Shenker] Best-response dynamics –with consistent tie-breaking –orthogonal to the results of Hart and Mas-Colell Diffusion of technologies in social networks –2 technologies {A,B}. Each node wants to be consistent with the majority of its neighbours. Circuit design Applications

Example 1: (node-dependent reactions) Each f i is such that for every a=(a 1,…,a n ) it holds that f i (a)=a i. “Tightness” of Our Result

Example 1: (node dependent reactions) Each f i is such that for every a=(a 1,…,a n ) it holds that f i (a)=a i. Example 2: (unbounded recall) –2 nodes, 1 and 2, each with action space {a,b}. –Node 2 wants to match node 1’s action. –Node 1 selects b if node 2 changed its action from a to b in the past, and a otherwise. –What happens at the initial state (b,a)? “Tightness” of Our Result

Thm: If f is node independent, and there exist multiple stable states, then the system is not convergent. Interesting connections to fundamental results in distributed computing theory. –the Fischer-Lynch-Patterson impossibility result for consensus protocols (1983) But, neither result is a special case of the other. Proving Our Result

The Dynamics Graph action vector a S =(a S 1,… a S n ) knowledge vector b S =(b S 1,… b S n ) State R knowledge transition i-transition State T State S 1.a T :=a S 2.b T :=a S 1.a R :=a S except a R i :=f i (b S ) 2.b R :=b S

The dynamics graph captures all dynamics. The scenario where –the initial state is a 0. –nodes 1 and 3 react simultaneously. –then nodes 2 and 3 react simultaneously. is captured as follows: Visualising Dynamics

The dynamics graph captures all dynamics. The scenario where –the initial state is a 0. –nodes 1 and 3 react simultaneously. –then nodes 2 and 3 react simultaneously. is captured as follows: Visualising Dynamics State S a S =b S =a 0

The dynamics graph captures all dynamics. The scenario where –the initial state is a 0. –nodes 1 and 3 react simultaneously. –then nodes 2 and 3 react simultaneously. is captured as follows: Visualising Dynamics State S a S =b S =a 0 1-transition 3-transitionk-transition

The dynamics graph captures all dynamics. The scenario where –the initial state is a 0. –nodes 1 and 3 react simultaneously. –then nodes 2 and 3 react simultaneously. is captured as follows: Visualising Dynamics State S a S =b S =a 0 1-transition 3-transitionk-transition 2-transition3-transitionk-transition

Defn: A state S in the dynamics graph is stable if every outgoing edge from S leads to S. Defn: A fair path in the dynamics graph is a path that (1) for each i, contains an i-transition; and (2) also contains a knowledge transition. Stability and Fairness

Defn: The attractor region of a stable state S are all states from which any (long enough) fair path reaches S. Attractor Regions

Claim: A fair cycle in the dynamics graph implies an oscillation in our model. Proposition: If there are multiple stable states then there are states in the dynamics graph that are not in any attractor region (“neutral states”). Proof Sketch (Cont.)

Colour each attractor region in a different colour – red, blue, etc. Colour the neutral states in purple. Colouring States

Key idea: We show that from every purple state there is a fair path that leads to another purple state. The number of purple states is finite and so this implies a fair cycle. Creating Oscillations

Lemma: There cannot be two edges leading from a purple state to two non-purple states that do not have the same colour. Intuition: We can swap the order of activations without affecting the outcome. Proof Sketch (Cont.)    ?  : different transitions

Fix a purple state p. Let R be a “maximal” fair path from p to another purple state. Proof Sketch (Cont.) p … … q R

Let  be a transition that is not on R. Observe that  at q takes us to a non- purple state. p … … q R  Proof Sketch (Cont.)

Because q is purple it must have a fair path to a non-purple non-red state. p … … q R  … … u Proof Sketch (Cont.)

Now, we prove that  at u must take us to a red state --- a contradiction! p … … q R  … … u  Proof Sketch (Cont.)

Our result holds for randomized reaction functions. –adversarially-chosen schedule What if the schedule is randomized? –our impossibility result breaks … –… but no general possibility result either Circumventing Our Impossibility Result: Randomness

Defn: A schedule  is r-fair if each node is activated at least once within every r consecutive time steps. Can we prove our impossibility result for schedules that are r-fair? If so, for what values of r? We present positive and negative results. Circumventing Our Impossibility Result: r-Fair Schedules

Thm: Determining if a system with n nodes, each with two actions, is convergent requires exponential communication (in n). The proof requires reaction functions to be of exponential size. Combinatorial proof: a “Snake in the Box” construction Complexity Results

What if the reaction functions can be succinctly described? Thm: Determining if a system with n nodes is convergent is PSPACE- Complete. Hence, there is no “short” characterization of asynchronous convergence! Complexity Results

Other notions of asynchrony Other reaction functions –fictitious play, regret minimization –Observation: regret minimization is much more resilient to asynchrony (different framework…). Other restrictions on schedules –random schedules –r-fair schedules –more Directions for Future Research

THANK YOU