Stability of decentralised control mechanisms Laurent Massoulié Thomson Research, Paris

Congestion control Point-to-point data flows Data rates regulated by TCP at end-points Multipath versions: “Overlay” TCP

Peer-to-peer-broadcasting Pplive, Sopcast,… Hosts exchange data with “overlay” neighbors Aim: real-time playback at all hosts

Outline Proportional fairness for congestion control  New characterisation  Implications on stability and insensitivity “Random useful” packet forwarding for p2p broadcasting  Optimality properties  Open questions

Network Bandwidth Allocation problem Flows of distinct types, s  S N s such flows Which rate  s to type s flows? Vector (  s ) s  S : must lie in set C C: captures physical network constraints  Convex  Non-increasing

Network capacity set C, single path flows Fixed routes & link capacity constraints (  i )  C   0 +  1 ≤ C 1,  0 +  2 ≤ C 2 Polyhedral, convex non-increasing capacity set C N0N0 C1C1 N2N2 C2C2 N1N1

Network capacity set C, multi-path flows Type s flows can use network paths from set P(s) Bandwidth:  s =  p  P (s)  p Network capacity (path var.): {  p }  C’ path variables:  ab +  cba +  bac ≤ c,… c a bc 2c a b c Capacity set C (class variables):  b +  c ≤ 2c,  a +  c ≤ 2c,  b +  a ≤ 2c.

Utility-maximising allocations Maximise  s  S N s U s (  s /N s ) over C Distributed control mechanisms (single- and multi-path) known A special case: U s (x)= w s [x 1-  -1]/(1-  ) if   1, w s log(x) if  =1  (w,  )-fair allocations In terms of Kuhn-Tucker multipliers: TCP square root formula:  “TCP-fairness” corresponds to  =2, w s =1/RTT 2 s

The dynamic set-up Type s file transfers: start at instants of Poisson process, rate s File sizes: Exponential distribution (  s ) [or general i.i.d.] Markov process: N s ++ at rate s, N s -- at rate  s  s where  s : result of congestion control (time scale separation assumption)

Objectives of congestion control  Maximise schedulable region, defined as R = Set of vectors of loads  s = s /  s such that Markov process ergodic  Make performance insensitive to assumption of exponential service times

Previous results Optimal schedulable region R=int(C)  Exponential service times Max-min fairness [Konstantopoulos et al. 99] (w,a)-fairness [Bonald-M. 01] General utility-maximisation schemes [Ye 03]  General i.i.d. service times Balanced fairness [Bonald-Proutiere 02-04] exactly insensitive; no known distributed control to achieve it Max-min fairness [Bramson 05]

Proportional fairness Definition: Alternative characterisation: where J: Fenchel-Legendre transform of (log of) capacity set C:

Main application Theorem Proportional Fairness achieves maximal schedulable region R=int(C) for arbitrary phase- type service time distributions (more generally, for original dynamics augmented by Markovian user routing)

Proof insights PF “almost” reversible: Suggests proof outline: The “right” Lyapunov function is given by Apply suitable Lyapunov function criteria for ergodicity (Foster, Rybko-Stolyar, Dai, Robert)

Reversible allocations Markov process reversible iff for some F, in which case, stationary distribution:

Reversible allocations (ctd) “Rate function” of equilibrium distribution decreases along “fluid dynamics” of system (by decrease of Kullback-Leibler divergence between current and stationary distributions)

Congestion control – summary Characterisation of proportional fairness  Yields new stability result  Explains previously observed reversibility on particular topologies (hypergrids)  could yield finer results, e.g. characterisation of rate function at equilibrium

Based on joint work with Andy Twigg, Christos Gkantsidis & Pablo Rodriguez P2P broadcasting

Broadcast problem Transmit data from source to all nodes  Unstructured (overlay) network  Nodes have no global knowledge Models many p2p applications  Content distribution  Video-on-Demand  Live video streaming

Broadcast problem Goal: Efficient decentralized schemes  Metrics: broadcast rate & playback delay Constraints:  Edge capacities (well studied, centralized) [distributed]  Node capacities (less explored) Models different nodes in P2P networks: ADSL, cable, …

Outline Rate-optimal scheme for edge-capacitated networks Node-capacitated networks Application: video streaming Summary

Edge-capacitated case: background λ* = min number of edges to disconnect some node from s Can be achieved by packing edge-disjoint spanning trees [Edmonds,Lovasz, Gabow,…] centralized algorithms broadcast rate, λ* = min [ mincut(s,i): i  V ] [Edmonds, 1972] a s b c a s b c a s b c +

Challenges Aim for decentralised schemes No explicit tree construction  simplifies management with node churn Manage tension between timeliness and diversity  in-order delivery from s to a & b reduces potential rate from 2 to a s b a b c

Random Useful packet forwarding Let P(u) = packets received by u for each edge (u,v) send a random packet from P(u) \ P(v) New packets injected at rate λ λ a s b c

Assumptions: G: arbitrary edge-capacitated graph Min(mincut(G)): λ * Poisson packet arrivals at source at rate λ Pkt transfer time along edge (u,v): Exponential random variable with mean 1/c(u,v) Theorem With RU packet forwarding, Nb of pkts present at source not yet broadcast: A stable, ergodic process. RU packet forwarding: Main result

a s b s,a s s,b s,a,b c s,a,cs,b,c s,a,b,c s,a,c Correct description of state space: Number of packets X A present exactly at nodes u  A, for any set of nodes A (plus state of packets in flight on edges) Optimality of RU – proof

Optimality proof s,a s s,b s,a,bs,a,cs,b,c s,a,b,c s,a,c Identify fluid dynamics: λ ?? λ Random Block Choice These capture the original system’s dynamics after some space/time rescaling; Prove that solution of fluid dynamics converges to zero when λ < λ* by exhibiting suitable Lyapunov function:

Outline Rate-optimal scheme for edge-capacitated networks Node-capacitated networks Application: video streaming Summary

Node-capacitated case P2P networks constrained by node upload capacity:  Cable, ADSL

Node-capacitated case P2P networks constrained by node upload capacity:  Cable, ADSL How to allocate upload capacity to neighbours?  By Edmonds thm, optimum can be achieved by assigning node capacities to edges and packing spanning trees a s b c a s b c 2 a s b c a s b c

Most-deprived neighbour selection for each node u  choose a neighbour v maximizing |P(u)\P(v)|  If u=source, and has fresh pkt, send random fresh pkt to v  Otherwise send random pkt from P(u)\P(v) to v Distributed: uses only local information Can estimate |P(u) \ P(v)| efficiently

Optimality properties Let λ* be the optimal rate that can be achieved by a feasible allocation of edge capacities {c* ij }. Theorem: For the complete graph and injection rate λ < λ*, system ergodic under fresh/RU pkt forwarding to most deprived neighbour. More general networks?

Outline Optimal & decentralized packet forwarding in edge-capacitated networks Node-capacitated networks Application: video streaming Summary

Video streaming Model  Assume feasible injection rate λ  Source begins sending at time 0  At time D, users start playing back at rate λ Packets not yet received are skipped  p = fraction of skipped packets How much delay to achieve target p?

Grid networks 40x40 grid Add shortcut edges with Pr=0.01 Place source in centre of grid

Grid networks

Delay/loss trade-off for RU policy Expected fraction of skipped packets is (1-1/k) D ~ e -D/k s v network A toy model: Let k=expected Nb of packets s has and v doesn’t Approximate the network by the following: Source begins with k packets 1..k Source receives new packets at rate λ Source gives randomly useful packets to v at rate λ k reflects connectivity between s and v Fraction of skipped packets decreases exponentially with delay D Can be used to determine suitable playback delay at receiver v.

Simulation Fraction of nodes Uplink capacity Random graph (n=500,p=0.05) Distribution of node capacities as observed in Gnutella [Bharambe et al] Optimal rate, λ* ≤ 1180 Delay < 1000 inter-pkt send times (<1min)

Conclusions Edge-capacitated networks  Random Useful pkt forwarding achieves optimal broadcast rate  Future: Understand topology impact on delays Extend to dynamic networks Node-capacitated networks  “Most deprived” neighbour selection appears to perform well Proven rate-optimal for complete graphs Future: optimal for other networks?

BACKUP