Mohammad Alizadeh Adel Javanmard and Balaji Prabhakar Stanford University Analysis of DCTCP:Analysis of DCTCP: Stability, Convergence, and FairnessStability, Convergence, and Fairness

Data Center Packet TransportData Center Packet Transport Transport inside the DC – TCP rules (99.9% of traffic in some DCs) But, TCP: – Needs large buffers for high throughput – Induces large queuing delays – Does not handle bursty traffic well (Incast) DCTCP was proposed to address these shortcomings (SIGCOMM’10). 2

TCP Buffer RequirementTCP Buffer Requirement Bandwidth-delay product rule of thumb: – A single flow needs C×RTT buffers for 100% Throughput. B Buffer Size B = C×RTT B B < C×RTT Buffer Size Throughput loss! B Buffer Size B > C×RTT More latency! To lower the buffering requirements, we must reduce sending rate variations. 3

DCTCP: Main IdeasDCTCP: Main Ideas 1.React in proportion to the extent of congestion. Reduce window size based on fraction of marked packets. 2.Mark based on instantaneous queue length. Fast feedback to better deal with bursts. Simplifies hardware. ECN MarksTCPDCTCP Cut window by 50%Cut window by 40% Cut window by 50%Cut window by 5% 4

DCTCP: AlgorithmDCTCP: Algorithm Switch side: – Mark packets when Queue Length > K. Sender side: – Maintain running average of fraction of packets marked (α).  Adaptive window decreases: – Note: decrease factor between 1 and 2. B K Mark Don’t Mark 5

DCTCP vs TCPDCTCP vs TCP Setup: Win 7, Broadcom 1Gbps Switch Scenario: 2 long-lived flows, K = 30KB (Kbytes) 6

Analysis of DCTCPAnalysis of DCTCP

Steady State AnalysisSteady State Analysis What is the effect of the various network and algorithm parameters on system throughput and latency? – Network: Capacity, Round-trip Time, Number of flows – Algorithm: Marking threshold (K), Averaging parameter (g) The standard approach is to study control loop behavior via fluid models. – Kelly et al., Low et al., Misra et al., Srikant et al, … 8

DCTCP Fluid ModelDCTCP Fluid Model 9 × N/RTT(t) W(t) p(t) Delay p(t – R * ) C + − 1 0 K q(t) Switch LPF AIMD α(t) Source

Fluid Model vs ns2 simulationsFluid Model vs ns2 simulations Parameters: N = {2, 10, 100}, C = 10Gbps, d = 100μs, K = 65 pkts, g = 1/16. N = 2N = 10N =

We make the following change of variables: The normalized system: The normalized system depends on only two parameters: Normalization of Fluid ModelNormalization of Fluid Model 11

Equilibrium Characterization Case 1: Equilibrium Characterization Case 1: Very large N: system (globally) converges to a unique fixed point: Example: 12

Very large N: system (globally) converges to a unique fixed point: 12 Example: Equilibrium Characterization Case 1: Equilibrium Characterization Case 1:

System has a periodic limit cycle solution. Example: 13 Equilibrium Characterization Case 2: Equilibrium Characterization Case 2:

System has a periodic limit cycle solution. Example: 13 Equilibrium Characterization Case 2: Equilibrium Characterization Case 2:

Stability of Limit CyclesStability of Limit Cycles Let X * = set of points on the limit cycle. A limit cycle is locally asymptotically stable if δ > 0 exists s.t.: 14

Poincaré MapPoincaré Map 15 x1x1 x2x2 x 2 = P(x 1 ) Stability of Poincaré Map ↔ Stability of limit cycle x * α = P(x * α )

Stability CriterionStability Criterion Theorem: The limit cycle of the DCTCP system: is locally asymptotically stable if and only if ρ(Z 1 Z 2 ) < 1. -J F is the Jacobian matrix with respect to x. -T = (1 + h α )+(1 + h β ) is the period of the limit cycle. Proof: Show that P(x * α + δ) = x * α + Z 1 Z 2 δ + O(|δ| 2 ). 16 We have numerically checked this condition for:

Parameter GuidelinesParameter Guidelines How big does the marking threshold K need to be to avoid queue underflow? B K 17

Throughput-Latency TradeoffThroughput-Latency Tradeoff Throughput > 94% as K  0 18 Parameters: C = 10Gbps, d = 480μs, g = For TCP: Throughput → 75% For TCP: Throughput → 75%

Convergence AnalysisConvergence Analysis How long does it take for DCTCP sources to converge to their “fair share” rate (C/N)? – DCTCP is slower to converge than TCP since it cuts its window by smaller factors. The fluid model is not suitable for transient analyses. We use a hybrid (continuous- and discrete-time) model. – The model is inspired by the AIMD models of Baccelli et al. and Shorten et al. 19

The Hybrid ModelThe Hybrid Model 20 Time Window Sizes Time p(t) (Marking Prob.) 1 RTT

Rate of Convergence (Theorem)Rate of Convergence (Theorem) Assume N DCTCP flows with arbitrary W i (0) and α i (0), evolving according to the Hybrid Model, with: Define function, and let 0 < α * ≤ 1 be the unique positive solution to Then: Also: where: 21

Consequences DCTCP converges at most 40% slower than TCP: The parameter g should not be too small: 22

(g = 0.07) (g = 0.025) (g = 0.005) Convergence: ns2 SimulationsConvergence: ns2 Simulations 23

Conclusion Our analysis shows DCTCP: – requires 17% of C×RTT for full throughput – achieves 94% throughput as K → 0. – converges at most 1.4 times slower than TCP. We provide guidelines for setting the DCTCP parameters. The analysis suggests a simple modification that improves the RTT-fairness of DCTCP. – Achieves linear-RTT fairness (Thrput RTT -1 ), like TCP-RED 24

25