Fast Convergence of Selfish Re-Routing Eyal Even-Dar, Tel-Aviv University Yishay Mansour, Tel-Aviv University

Overview Routing on Parallel links –Model –Coordination Ratio –Migration Distributed model –Convergence results Few Types of Equilibrium: –termination, migration, overall.

Routing on parallel links Job scheduling Classic setting: –Centralize control –Optimize a global objective function minimize MAX load –Full cooperation Game theory setting: –Each user optimizes its objective function Load of the machine it selects.

Model: Users and Links n users m links

Model: Users and Links n users m links job weights

Model: Users and Links n users m links

Model: Links and Users Routing: –m links & n users Link Model: –Link M i has speed S i User Model: –Weighted: User U has a weight w (U) –Unrelated: user U has a weight w k (U) on M k Load on link M i at time t: –B i (t) = Users routing on M i at time t –L i (t) = [Σ j in Bi(t) w i (j) ] / S i

Nash Equilibrium n users m links

Model: Nash Equilibrium No user can move and lower its load. For a user U on link M i –For any link M j –If U moves to link M j –Then L i  L j + w j (U)/S j The load after any move is not lower than before!

Coordination Ratio A global optimization function –minimize MAX load Coordination Ratio Compares: –Optimal value –Worse Nash Value Results for job scheduling [KP,MS,CV,AAR] –Identical: 2 or O(log n / log log n) –Related: O(log n / log log n) –Unrelated: unbounded

Convergence to Nash How (fast) users reach the Nash Eq. Main concern: –Duration Non-issue: –Quality of Nash Eq. Migration models –Elementary Step Size (ESS) –Distributed

ESS: Migration n users m links Scheduler

ESS: Migration n users m links Scheduler

ESS: Migration n users m links Scheduler

ESS Migration model [ORS] Introduced to study routing User’s aim: minimize its observed load Elementary step system: –Only one user moves at a time. –Scheduler: arbitrary; Specific: random; FIFO; Max Weight; Max Load –User’s move improvement/best reply

Potential Games [M+S] Global Potential function Relates: –user utility change –global potential change Potential functions: –Perfect/Weighted/Ordinal Deterministic Nash Eq. Equivalent to congestion games. –Exponential reduction

Potential games and routing [EKM, ICALP 2003] Potential type UsersLinks perfectidentical weighted related ordinalunrelated

Example of Perfect Potential Identical users and links Potential: User moving from link i to link j:

Upper Bound : Identical machines Max Weight + Best response Theorem: Max Weight + Best response: stabilizes in at most n moves Claim: Best Response & identical machine, after job J migrates, it will move only after a larger job reached its machine.

Upper Bound : Identical machines Max Weight + Best response Consider user U that moves to M i –At time of move its stable User U’ moves from M j to M k U’ U

Upper Bound : Identical machines Max Weight + Best response Consider user U which moved to M i –At time of move its stable User U’ moves from M j to M k U’ U

Upper Bound : Identical machines Max Weight + Best response U’ U User U’ moves to M i and w(U’) < w(U) –This is the best response of U’

Upper Bound : Identical machines Max Weight + Best response User U’ moves to M i and w(U’) < w(U) –This is also U best response U’ U

Other results [EKM] Identical links: –Max weight user scheduler No user moves twice. –Min weight user scheduler Exponential lower bound Related & Unrelated links: –Various schedulers

ESS model Orda Asych ICALP

This work: Distributed model Concurrent migration –Randomized policies –no scheduler Major difference: –User might be worse off after migration Convergence time –Identical users: O(log log n)

Distributed Model Users: –Identical and Anonymous Termination Nash Equilibrium: –Balanced load on links Policy –Sets a prob. for migration between links. Convergence time –Number of steps until Termination

Two Links: Balance Policy Assume n is even Migration: –From Overloaded to Underloaded with p= d(t)/L 1 (t) Expected load: E[L i (t+1)]=n/2 Theorem: converges in expected O(loglog n) L 1 (t)L 2 (t) 2 d(t)

Two Links: Balance Policy Sketch of Proof: Two phases: –Switch phases when d(t)  3 ln 1/  First phase: –simple Chernoff bound –Completes after O(loglog n ) steps Second phase: –Each step terminates with prob. Setting  =1/T.

Two Links: Nash ReRouting Balance: p=1/4 Single user: –Load on 1: 300 – ¾ –Load on 2: 300 – ¼ –Best response: STAY! Nash ReRouting: –Every migration step is Nash Equilibrium –Myopic users

Two Links: Nash ReRouting Loads (n=2K): –L 1 = K+d –L 2 = K-d Nash ReRouting: Migration prob: Diff. Exp. Loads! Similar Convergence bound L1L1 L2L2 2d

Two Links: Sub-game Perfect Cost accumulate –discounted over time User optimizes its discounted return. Existence: Similar to Stochastic games Convergence: –Number of steps O(log log n) –Constants depend on the discount factor!

Two Links: Sub-game Perfect Proof ideas: –Let A=1/(1-  ) –Can “guarantee” 0.5 from any state. –Bound the value of a state |v d | < 0.5 A –Migration prob. p d = d/(n+d) +/- O(A/n) –Low probabilities: Can not be too small O(1/An) –Termination in one step in low prob.

Multiple Links: Balance policy Loads (n=mK): –L i = K+d i –Over = {i:d i > 0} –d =  i in OVER d i Migration prob: –Migrate: d i /L i –Destination: |d k |/d Exp. Load: E[L i ]=K Theorem: Õ(loglog n + log m) L2L2 L3L3 L4L4 L1L1

Multiple Links: Balance Theorem: O(loglog n+ log 1.5 m) Proof Sketch: First phase as before O(loglog n) –Phase ends: –Problem: many links Second phase: –Unbalanced(t) > log 1.5 1/  –  = 1/(T m)

Multiple Links: Balance Goal: – Unbalanced(t+1) < 0.48 *q*Unbalanced(t) –q = Prob. of Link to be balanced in one step. After O(log m /q) steps –Unbalanced(t) = O(log 1.5 1/  ) –Small number of links Analysis of Unbalanced(t) –Separate Over and Under –Negative association

Multiple Links: Nash ReRouting Always exists: –Similar to Symmetric Players Computation: –Independent of n (num. of users) –Exponential in m (num. links) –Algorithm: For each link guess support. Linear set of Eq. Convergence: Similar to Balance

Other results Link Speeds: –Results and analysis carry over. Weighted Users –lower bound: –Two links:  (  n) Exponential weights High Probability results.

Future work Nash Computation –Nash ReRouting many links –Sub-Game Perfect Eq. Two Links Weighted users: –Algorithms Two links O(log W max ) ?

Concurrent job migrations Model –Identical machines and Unit jobs Two machines –O(log log n) Multiple machines –O(log log n + log c m) Nash Eq. at move –Existence –convergence

What’s next Nashification [FGMLR] Consider paths in graphs –General Load and Additive cost: No DET Nash [LO] –Max Cost: Always converges. General Congestion/Potential games –Personal Preferences and weights [M] Beyond DET Nash