Nash Equilibria In Graphical Games On Trees Edith Elkind Leslie Ann Goldberg Paul Goldberg

Games and Strategies Games: strategic interactions between rational entities Solution concepts: what’s going to happen? –dominant strategies –Nash equilibrium –…. Can it be computed? –if your computer cannot find it, the market probably cannot either

Matrix (normal form) Games Row player: Column player: finite set of players {1, …, n} each player has k actions (pure strategies): 1, …, k payoffs of the i th player: P i : {1, …, k} n → R

Nash Equilibrium Row player: Column player: Nash equilibrium: a strategy profile such that noone wants to deviate given other players’ strategies, i.e., each player’s strategy is a best response to others’ strategies: –(0, 0) and (1, 1) are both NE

Pure vs. Mixed Strategies Row player: Column player: H T HTHT H T NE in pure strategies may not exist! –“matching pennies” Mixed strategy: a probability distribution over actions –50% tail, 50% head

Existence of NE Theorem (Nash 1951): any n-player k-action game in normal form has an equilibrium in mixed strategies can we find one in poly-time?

2 players, n actions Representation: two n x n matrices Computation: –all known methods are exptime –can it be NP-hard? no: NE always exists –PPAD-hardness: notion of hardness for total search problems –DGP’06: finding NE in 4-player games is PPAD-hard –CD’06: finding NE in 2-player games is PPAD-hard –DGP reduction uses graphical games (the topic of this talk!)

n-player 2-action games representation: payoffs to each player for every action profile (vector in {0, 1} n ): n2 n numbers graphical games: –players are vertices of a graph –V’s payoff depends on actions of W in N (V) U V –n players, max degree d => n2 d+1 numbers T U V W t=0, u=0, v=0, w=0: 12 t=1, u=0, v=0, w=0: 31 …. t=1, u=1, v=1, w=1: -6 W’s payoffs (16 cases):

Complexity: what is known Bounded-degree trees: –Exp-time algorithm/poly-time approximation algorithm to find all NE (Kearns, Littman, Singh, UAI 2001) –??? poly-time algorithm to find a single NE (Kearns, Littman, Singh, NIPS’2001) Heuristics for graphs with cycles General graphs: –PPAD-complete (DGP’06) even if max deg=3

Our Results (1) Algorithm in NIPS’01 paper is incorrect (does not always output a NE) We fix the NIPS’01 algorithm, but… –our algorithm runs in poly-time on paths –with a trick, also on cycles –can be used to find (a representation for) all NE in n 3 time, or a single NE in n 2 time

Our Results (2) There is a graph of pathwidth 2 on which our algorithm runs in exp time –true for all algorithms that use the basic approach of the UAI’01 paper The problem is PPAD-complete for bounded pathwidth graphs Open question: what if pathwidth = 1? –generalizes a cool geometry problem (talk to me if you like those, or see the paper)

Warm-up: 2-player 2-action games Row player: Column player: Suppose R plays 1 w.p. r EP(C) from playing 0: (1-r)*1 EP(C) from playing 1: r*3 1-r > 3r iff r < ¼ Suppose C plays 1 w.p. c EP(R) from playing 0: (1-c)*2 EP(R) from playing 1: c*1 (1-c)*2 > c iff c < 2/3 1/4 1 r BR(C) c 1 2/3 BR(R) mixed NE: r=1/4, c=2/3

Potential best response: v is a PBR to w iff when W plays w, there is a NE for T V in which V plays v. upstream pass: construct PBR V (w) from PBR U1 (v), PBR U2 (v) and PBR U3 (v) downstream pass: root selects its strategy based on the children’s PBR’s; propagates to leaves Algorithm for Trees (KLS’01) TVTV W V U1U1 U2U2 U3U3 v w

KLS algorithm: running time For bounded-degree trees, constructs all PBR (and then find a NE) in exp time FPTAS for an  -NE: –superimpose PBR with a  -grid –there exists a grid point  -close to PBR –  -NE (  = poly(  ) ): no one can gain more than  by deviating

Computing PBR: Example Payoffs to V: –P 000 = 1, P 001 = -9, P 100 = 9, P 101 = -1, P u1w = 0 for u, w =0, 1 E0 = EP(V) from playing 0: (1-u)(1-w)*1+(1-u)w*(-9)+u(1-w)*9+uw*(-1) = 1+8u-10w E1 = EP(V) from playing 1: 0 E0 = E1 iff w = (8u+1)/10 = f(u) UVW.51 1 u v v w (v, u) → (f(u), v) PBR U (v) PBR V (w)

Trees: too many segments vvw u t v v1v1 v2v2 v1v1 v2v2 v1v1 v2v2 KLS (NIPS’01): can “trim” PBR Incorrect! W V TU (v,t), (v,u) → (f(u,t), v) u2u2 u1u1 t2t2 t1t1

Solutions? Solution 1 (for paths): algorithm of UAI’01 paper, careful analysis –the number of segments/rectangles in each PBR is O(n 2 ) –running time O(n 3 ) Solution 2 (for paths): can pick a subset of each PBR consisting of O(n) segments –O(n 2 ) running time

O(n 3 ) algorithm f(u) =(au+b)/(cu+d) u*: cu*+d = 0 [v 1, v 2 ] x {u} => {f(u)} x [v 1, v 2 ] {v} x [u 1, u 2 ] => [f(u 1 ), f(u 2 )] x {v} if u* not in [u 1, u 2 ] [0, f(u 2 )] U [f(u 1 ), 1] x {v} if u 1 ≤ u*≤ u 2 PBR V (w) vs PBR U (v): new segments at v=1 and v=0, some segments break into two --- double in size? no: count the event points u v v w (v, u) → (f(u), v) PBR U (v) PBR V (w) u*

Extension to trees? V0V0 V1V1 V2V2 V n-1 VnVn U1U1 T1T1 U n-1 T2T2 U2U2 T n-1 TnTn UnUn

Hardness results pathwidth 2: our algorithm is not poly-time –and neither is any two-pass algorithm that stores subsets of PBR pathwidth > k: (probably) all algorithms are not poly-time –finding NE in this case is PPAD-hard – idea: modify the construction in DGP’06

Good Nash Equilibria Row player: Column player: Nash equilibria: (0, 0): total payoff is 3 (1, 1): total payoff is 4 (1/4, 2/3): total payoff is 17/12 not all NE are created equal…

What is a good NE? maximize sum of player’s payoffs guarantee to each player a payoff of at least t i (almost) equal payoffs any combination of those…. Can we use PBR data structure to compute those?

Can we represent it? any GG with integer payoffs on a tree has a rational NE Any PBR consists of segments and rectangles with rational coordinates Yet, total payoff-maximizing NE may be irrational Our result (EGG’07): for any algebraic , deg(  ) = n, there is a GG with int payoffs on a path of length O(n) in which in the best NE player 1 plays 

Approximation Can we use the FPTAS of KLS’01? –superimpose PBR with a  -grid Observation: there is a grid point  -close to best NE –look for best point on the grid close to PBR dynamic programming –  -NE (  = poly(  ) ): no one can gain more than  by deviating

True Nash  -NE is not always appropriate –what if players are not willing to lose  ? Can we find a (true) NE that is  -close to the best (true) NE? Idea: –add borders of rectangles in PBR to the grid –only consider grid points in PBR

Bounded Payoff Nash Similar algorithm works --- FPTAS –Also for other kinds of “good” NE If all payment bounds are rational, there is a BP NE that is “almost” rational (deg ≤ 2) Open question: can we compactly represent all bounded payoff NE? –perhaps by incorporating payoff bounds into PBR?

Conclusions Nash equilibria in graphical games on trees complexity still unknown…