COLOR TEST COLOR TEST

Dueling Algorithms N ICOLE I MMORLICA, N ORTHWESTERN U NIVERSITY WITH A. T AUMAN K ALAI, B. L UCIER, A. M OITRA, A. P OSTLEWAITE, AND M. T ENNENHOLTZ

Social Contexts Normal-form games: Players choose strategies to maximize expected von Neumann-Morgenstern utility. Social context games [AKT’08]: Players choose strategies to achieve particular social status among peers.

Social Contexts Ranking games [BFHS’08]: Players choose strategies to achieve particular payoff rank among peers.

Two-Player Ranking Games G Alice Bob RG payoff of Alice: 1Alice beats Bob in G Alice and Bob play game: ½Alice ties Bob in G 0Alice loses to Bob in G

Implicit Representations Succinct games [FIKU’08]: Payoff matrix represented by boolean circuit. NE hard to solve or approximate. Blotto games [B’21, GW’50, R’06, H’08]: Distribute armies to battlefields.

Implicit Representations Optimization duels [this work]: Underlying game is optimization problem. Goal is to optimize better than opponent.

Ranking Duel A search engine is an algorithm that inputs set Ω = {1, 2, …, n} of items probabilities p 1 + … + p n = 1 of each and outputs a permutation π of Ω. Monopolist objective: minimize E i ~ p [π(i)].

Ranking Duel Competitive objective: Let the expected score of a ranking π versus a ranking π’ be Pr i ~ p [ π(i) < π’(i) ] + (½) Pr i ~ p [ π(i) = π’(i) ]. Then objective is to output a π that maximizes expected score given algorithm of opponent.

Optimizing a Search Engine User searches for object drawn according to known probability dist.

Search: pretty shape 1. (27%) 2. (22%) 3. (19%) 4. (16%) 5. (09%) 6. (07%)

Choosing a Search Engine 1.Search for “pretty shape”. 2.See which search engine ranks my favorite shape higher. 3.Thereafter, use that one.

Search: pretty shape 1. (27%) 2. (22%) 3. (19%) 4. (16%) 5. (09%) 6. (07%) Search: pretty shape 6. (27%) 1. (22%) 2. (19%) 3. (16%) 4. (09%) 5. (07%)

Questions Can we efficiently compute an equilibrium of a ranking duel? How poorly does greedy perform in a competitive setting? What consequences does the duel have for the searcher?

Optimization Problems as Duels RankingBinary SearchRouting Parking CompressionHiring Start Finish ? ? ? ? ? ? ?

Duel Framework Finite feasible set X of strategies. Prob. distribution p over states of nature Ω. Objective cost c: Ω × X R. Monopolist: choose x to minimize E ω ~ p [c ω (x)].

Duel Framework 1.Players select strategies x, x’ from X. 2.Nature selects state ω from Ω according to p. 3.Payoffs v(x,x’), (1-v(x,x’)) are realized. 1 if c ω (x) < c ω (x’) 0 if c ω (x) > c ω (x’) ½ if c ω (x) = c ω (x’) v(x,x’) = E ω ~ p

Results: Computation An LP-based technique to compute exact equilibria, A low-regret learning technique to compute approximate equilibria, … and a demonstration of these techniques in our sample settings

Computing Exact Equilibria Formulate game as bilinear duel: 1.Efficiently map strategies to points X in R n. 2.Define constraints describing K=convex-hull(X). 3.Define payoff matrix M that computes values. 4.Maps points in K back to strategies in original setting.

Bilinear Duels If feasible strategies X are points in R n, and payoff v(x, x’) is x t Mx’ for some M in R nxn, then max v,x v s.t. x t Mx’ ≥ v for all x’ in X x is in K (=convex-hull(X)) Exponential, but equivalent poly-sized LP.

Ranking Duel Formulate game as bilinear duel: 1.Efficiently map strategies to points X in R n. X = set of permutation matrices (entry x ij indicates item i placed in position j) 2.Define constraints describing K=convex-hull(X). K = set of doubly stochastic matrices (entry y ij = prob. item i placed in position j)

Ranking Duel Formulate game as bilinear duel: 4.Design “rounding alg.” that maps points in K back to strategies in original setting. Birkhoff–von Neumann Theorem: Can efficiently construct permutation basis for doubly stochastic matrix (e.g., via matching).

Ranking Duel Formulate game as bilinear duel: 3.Define payoff matrix M that computes values. E p,y,y’ [v(x,x’)] = ∑ i p(i) ( ½ Pr y,y’ [x i = x’ i ] + Pr y,y’ [x i > x’ i ]) = ∑ i p(i) (∑ i y ij ( ½ y’ ij + ∑ k>j y’ ik )) which is bilinear in y,y’ and so can be written y t My’.

Ranking Duel Result: Can reduce computation time to poly(n) versus poly(n!) with standard LP approach. Technique also applies to hiring duel and binary search duel.

Compression Duel data Goal: smaller compression (i.e., lower depth in tree). (each with prob. p(.))

Classical Algorithm

Compression Duel Formulate game as bilinear duel: 1.Efficiently map strategies to points X in R n. X = subset of zero-one matrices* (entry x ij indicates item i placed at depth j) 2.Define constraints describing K=convex-hull(X). K = subset of row-stochastic matrices* (entry y ij = prob. item i placed at depth j) * Must correspond to depth profile of some binary tree!

Compression Duel Formulate game as bilinear duel: 3.Define payoff matrix M that computes values. E p,y,y’ [v(x,x’)] = ∑ i p(i) (∑ i y ij ( ½ y’ ij + ∑ k>j y’ ik )) which is bilinear in y,y’ and so can be written y t My’.

Compression Duel Bilinear Form: max v,x v s.t. x t Mx’ ≥ v for all x’ in X x is in K (=convex-hull(X)) Problems: 1. How to round points in K back to a random binary tree with right depth profile? 2. How to succinctly express constraints describing K?

Approximate Minimax Defn. For any ε > 0, an approximate minimax strategy guarantees payoff not worse than best possible value minus ε. Defn. For any ε > 0, an approximate best response has payoff not worse than payoff of best response minus ε.

Best-Response Oracle Idea. Use approximate best-response oracle to get approximate minimax strategies. 1. Low-regret learning: if x 1,…,x T and x’ 1,…,x’ T have low regret, then ave. is approx minimax. 2. Follow expected leader: on round t+1, play best-response to x 1,…,x t to get low-regret.

Compression Best-Response Given lists of items with values and weights, pick one from each list with max total value and total weight at most one. Multiple-choice Knapsack:

Compression Best-Response Depth: 1234

Compression Best-Response (each with prob. p(.)) x’ in K For j from 1..n, list of depth j: v( ) = Pr[win at depth j | x’ ] w( ) = 2 -j … Kraft inequality

Other Duels 1.Hiring duel: constraints defining Euclidean subspace correspond to hiring probabilities. 2.Binary search duel: similar to hiring duel, but constraints defining Euclidean subspace more complex (must correspond to search trees). 3.Racing duel: seems computationally hard, even though single-player problem easy.

Conclusion Every optimization problem has a duel. Classic solutions (and all deterministic algorithms) can usually be badly beaten. Duel can be easier or harder to solve, and can lead to inefficiencies. OPEN QUESTION: effect of duel on the solution to the optimization problem?