Cloud Control with Distributed Rate Limiting Barath Raghavan, Kashi Vishwanath, Sriram Ramabhadran, Kenneth Yocum, and Alex C. Snoeren University of California, San Diego

Centralized network services Hosting with a single physical presence –However, clients are across the Internet

Running on a cloud Resources and clients are across the world Services combine these distributed resources 1 Gbps

Key challenge We want to control distributed resources as if they were centralized

Ideal: Emulate a single limiter Make distributed feel centralized –Packets should experience same limiter behavior S S S D D D 0 ms Limiters

Distributed Rate Limiting (DRL) Achieve functionally equivalent behavior to a central limiter Global Random Drop Flow Proportional Share Packet-level (general) Flow-level (TCP specific) Global Token Bucket 123

Distributed Rate Limiting tradeoffs Accuracy (how close to K Mbps is delivered, flow rate fairness) + Responsiveness (how quickly demand shifts are accommodated) Vs. Communication Efficiency (how much and often rate limiters must communicate)

Limiter 1 DRL Architecture Limiter 2 Limiter 3 Limiter 4 Gossip Estimate local demand Estimate interval timer Set allocation Global demand Enforce limit Packet arrival

Token Buckets Token bucket, fill rate K Mbps Packe t

Demand info (bytes/sec) Building a Global Token Bucket Limiter 1Limiter 2

Baseline experiment Limiter 1 3 TCP flows S D Limiter 2 7 TCP flows S D Single token bucket 10 TCP flows S D

Global Token Bucket (GTB) Single token bucket Global token bucket 7 TCP flows 3 TCP flows 10 TCP flows Problem: GTB requires near-instantaneous arrival info (50ms estimate interval)

Global Random Drop (GRD) 5 Mbps (limit) 4 Mbps (global arrival rate) Case 1: Below global limit, forward packet Limiters send, collect global rate info from others

Global Random Drop (GRD) 5 Mbps (limit) 6 Mbps (global arrival rate) Case 2: Above global limit, drop with probability: Excess Global arrival rate Same at all limiters 1616 =

GRD in baseline experiment Single token bucket Global random drop 7 TCP flows 3 TCP flows 10 TCP flows (50ms estimate interval) Delivers flow behavior similar to a central limiter

GRD with flow join (50ms estimate interval) Flow 1 joins at limiter 1 Flow 2 joins at limiter 2Flow 3 joins at limiter 3

Flow Proportional Share (FPS) Limiter 1 3 TCP flows S D Limiter 2 7 TCP flows S D

Flow Proportional Share (FPS) Limiter 1Limiter 2 “3 flows” “7 flows” Goal: Provide inter-flow fairness for TCP flows Local token-bucket enforcement

Estimating TCP demand Limiter 1 D Limiter 2 3 TCP flows S D 1 TCP flow S S

Estimating TCP demand Local token rate (limit) = 10 Mbps Flow A = 5 Mbps Flow B = 5 Mbps Flow count = 2 flows

Estimating TCP demand Limiter 1 1 TCP flow S D Limiter 2 3 TCP flows S D S 1 TCP flow

Key insight: Use a TCP flow’s rate to infer demand Estimating skewed TCP demand Local token rate (limit) = 10 Mbps Flow A = 2 Mbps Flow B = 8 Mbps Flow count ≠ demand Bottlenecked elsewhere

Estimating skewed TCP demand Local token rate (limit) = 10 Mbps Flow A = 2 Mbps Flow B = 8 Mbps Local Limit Largest Flow’s Rate 10 8 = Bottlenecked elsewhere = 1.25 flows

10 Mbps x Flow Proportional Share (FPS) Global limit = 10 Mbps Set local token rate = = 3.33 Mbps Global limit x local flow count Total flow count =

Under-utilized limiters Limiter 1 D S 1 TCP flow S Set local limit equal to actual usage Wasted rate (limiter returns to full utilization)

Flow Proportional Share (FPS) (500ms estimate interval)

Additional issues What if a limiter has no flows and one arrives? What about bottlenecked traffic? What about varied RTT flows? What about short-lived vs. long-lived flows? Experimental evaluation in the paper –Evaluated on a testbed and over Planetlab

Cloud control on Planetlab Apache Web servers on 10 Planetlab nodes 5 Mbps aggregate limit Shift load over time from 10 nodes to 4 nodes 5 Mbps

Static rate limiting Demands at 10 apache servers on Planetlab Demand shifts to just 4 nodes Wasted capacity

FPS (top) vs. Static limiting (bottom)

Conclusions Protocol agnostic limiting (extra cost) –Requires shorter estimate intervals Fine-grained packet arrival info not required –For TCP, flow-level granularity is sufficient Many avenues left to explore –Inter-service limits, other resources (e.g. CPU)

Questions!

FPS state diagram Case 2: Underutilized Set local limit to actual usage Case 1: Fully utilized Flows start/end, Network congestion, Bottlenecked flows

DRL cheat-proofness Conservation of rate among limiters for FPS EWMA compensates for past cheating with higher drop rates in the future To cheat GRD, forever increase the demand Authenticated inter-limiter communication assumed Difficult to quickly move traffic demands

DRL applications Cloud-based services (e.g. Amazon’s EC2/S3) Content-distribution networks (e.g. Akamai) Distributed VPN limiting Internet testbeds (e.g. Planetlab) Overlay network service limiting

DRL ≠ QoS DRL provides: Fixed, aggregate rate limit No service classes No bandwidth guarantees No reservations No explicit fairness; implicit (TCP) fairness

DRL provisioning Providers could ensure that the limit K Mbps is available at all locations; wasteful We expect it to be like current practice: statistical multiplexing Even today, can’t guarantee bandwidth to all destinations with a single pipe

Experimental setup Modelnet network emulation on testbed 40ms inter-limiter RTT Emulated 100 Mbps links No explicit inter-limiter loss Ran limiters across 7 testbed machines On Linux 2.6; packet capture via iptables ipq

Short flows with FPS 2 limiters, 10 bulk TCP vs. Web, 10Mbps limit Web traffic based on CAIDA OC-48 trace Loaded via httperf, Poisson arrivals (μ = 15) CTBGRDFPS Bulk rate Fairness (Jain’s) Web rate [0, 5K) [5K, 50K) [50K, 500K) [500K, ∞)

Bottlenecked flows with FPS Baseline experiment: 3-7 flow split, 10 Mbps At 15s, 7 flow aggregate limited to 2Mbps upstream At 31s, 1 unbottlenecked flow arrives joins 7 flows

RTT heterogeneity with FPS FPS doesn’t reproduce RTT unfairness 7 flows (10 ms RTT) vs. 3 flows (100 ms RTT) CTBGRDFPS Short RTT (Mbps) (stddev) Long RTT (Mbps) (stddev)

Scaling discussion How do various parameters affect scaling? Number of flows present: per-limiter smoothing Number of limiters: ~linear overhead increase Estimate interval: limits responsiveness Inter-limiter latency: limits responsiveness Size of rate limit delivered: orthogonal

Comm. Fabric requirements Up-to-date information about global rate Many designs are possible: Full-mesh (all-pairs) Gossip (contact k peers per round) Tree-based …

Gossip protocol Based on protocol by Kempe et al. (FOCS 2003) Send packets to k peers per estimate interval Each update contains 2 floats: a value and a weight To send an update to k peers, divide value and weight each by (k+1), store result locally, send k updates out To merge an update, add the update’s value and weight to the locally known value and weight