Convergence Time to Nash Equilibria in Load Balancing Eyal Even-Dar, Tel-Aviv University Alex Kesselman, Tel-Aviv University Yishay Mansour, Tel-Aviv University

Game Theory and CS AI: –Machine Learning –Reinforcement learning –Multiple agents Communication Networks: –Huge networks with little control –Futuristic control mechanisms CS Theory

Motivation: Internet Diverse set of users Very large scale system –Extremely hard to optimize Selfish goals and behavior –Relative anonymity Consider game theory –Non-cooperative players

Routing: Motivating example Each source can select a route Source Goal: Minimize latency –Solution concept: Nash Eq. Global Goal: Maximize utilization Coordination Ratio –How bad can Nash Eq. be?

Job Scheduling Classic setting: –Centralize control –Optimize an objective function minimize MAX load –Full cooperation Game theory setting: –Congestion and Potential games –Each job optimizes its objective Load of the machine it selects.

What are we after? Convergence TIME to reach a Nash Eq. Major issue –For implementation –Understanding the constraints –Theoretical interest. Non Issue (here) –The quality of the resulting Nash.

Model: Jobs and Machines Job Scheduling: –m machines –n jobs Machine Model: –Machine M i has speed S i –m machines ands speeds in [S min, S max ] Jobs Model: –Unrelated: job J a weight w k (J) on M k –Otherwise: job J a weight w (J) –Restricted: job J can be assign to M in R(J)

Model: Weights and Load Weights: –Total sum of weights W –Maximum weight w max –integer versus arbitrary weights. –Discrete weights: K different weights Load Model: –Machine M i at time t: –B i (t) = jobs running on M i –L i (t) = Σ j in Bi(t) w i (j) ; L max = MAX L i –T i (t) = L i (t) / S i ; T max = MAX T i

Machines M4M2M3 M1 L 4 B 2

Model: Nash Equilibrium No job can move and lower its load. For a job J at M i –For any M j –If J moves to M j –Then T i  T j + w j (J)/S j The load after the move is not lower than before!

Model: Migration Elementary step system (ESS): –Only one job moves at a time. –Job’s aim: minimize its observed load –A(t) = jobs wanting to move at time t –Job’s move improvement best reply –Scheduler: arbitrary; Specific: random; FIFO; Max Weight; Max Load

Our motivation Study the time it takes the system to converge. Arbitrary schedule: –Universal guarantee. Specific Schedule: –Optimize convergence

Results overview Unrelated Machines –Always converges Identical Machines: –Arbitrary [Min Weight]: Lower bound (Exponential in m) –Max Weight: n –FIFO:  (n 2 ) –Random: O(n 2 )

Results Overview Related Machines [ignore speeds]: –Arbitrary: O(W 2 ) –Max Load: O(W√m + n (w max ) 2 ) –Restricted and unit weight Jobs: strategy with O( m n ) [ideas similar to Milchtaich ’96] Discrete weights: –w max = O( K n 4K )

Max Job First M4M1M2M3

Min Job First M4M1M2M3

Upper bound: Unrelated Claim: No global system state occurs twice. –Lexicographic sorted order of states 90, 100, 20, 1, 3, 100, 5, 2, 90, 90, , 100, 90, 90, 90,90, 20, 5, 3, 2, 1 –improvement step -> lower order move from load 90 to 1 90 lowers to 85 1 increases to , 100, 85, 90, 90,90, 20, 5, 3, 2, , 100, 90, 90,90, 89, 85, 20, 5, 3, 2

Upper bound: Unrelated Convergence bounds: –Number of state General: m n K Weights:

Upper Bound: Unrelated Integer Weights: –Potential Function: –Each move the Potential reduces: Consider a move from M i to M k

Upper Bound: Unrelated Convergence bound: Arbitrary: –Let W=  J max j w j (J) –Initially 4 W  max{P(t)}, –Each move drops by at least 2 –Bound O(4 W ) Max Load Machine: –Initially, each move drops by P(t)/2m –O(mW + m 4 W/m+ w max )

Lower Bound: Two identical Machines m 1 2 m 1 2 m 1 2 Theorem: Min Job First requires at least  (n 2 ) steps to converge.

Lower bound: Identical Machines K distinct weights –Each weight has n/K=r jobs –weight w i+1 >> w i m=K+1 Machines numbered 0 to K Initial configuration –Machine M i has all jobs of weight w i Scheduler: Min job First –Each move of job w i creates an avalanche

Lower Bound: Identical Machines

Identical Machines Lower Bound –duration of phase i is r i /(2 i!) –lower bound (n/k) k /(2 k!) Upper bound –arbitrary schedule –upper bound (n/k +1) k

Upper Bound : Identical machines Max Weight + Best response Theorem: Max Weight + Best response: stabilizes in at most n moves Claim: Using Best Response in identical machine, after job J stabilizes it will move only after a larger job reached its machine.

Upper Bound : Identical machines Max Weight + Best response Consider job J which moved to M i –At time of move its stable Job J’ moves to M k –J’ improved –J did not want to move to M k Job J’ moves to M i and w(J’) < w(J) –This was the best response of J’

Upper Bound: Related Machines Potential function: –S min =1 and w min =1 Lemma –When a job of size w move from M u to M v P r (t+1)-P r (t) = 2 w (T v (t+1)-T u (t)) < 0

Upper Bound: Related Theorem: –Related & Restricted assignments –  -Nash: O(W 2 /  ) Initial potential =O(W 2 ) Each move improves by  –Nash [integer weights]: O(W 2 (S max ) 2 ) Smallest  = O(1/ (S max ) 2 )

Upper Bound: Related Theorem: –Max Load [Related & unrestricted] –  -Nash

Discrete Integer Weights Bound the maximum weight A priori, unbounded Two weights: w max  n Beyond two: much more tricky! Define equivalence of weights –Same “relative size” for two assignments View it as an integer program Bound solution size: K (c S max n) 4K

What’s next Consider paths in graphs –General Load and Additive cost: No DET Nash [LO] –Max Cost: Always converges. General Congestion games –Personal Preferences and weights [M] Beyond DET Nash.