Sachin Katti, CS244 Slides courtesy: Nick McKeown Sizing Router Buffers Sachin Katti, CS244 Slides courtesy: Nick McKeown

Routers need Packet Buffers It’s well known that routers need packet buffers It’s less clear why and how much Goal of this work is to answer the question: How much buffering do routers need? Given that queueing delay is the only variable part of packet delay in the Internet, you’d think we’d know the answer already!

How much Buffer does a Router need? Source Router Destination C 2T Universally applied rule-of-thumb: A router needs a buffer size: 2T is the two-way propagation delay (or just 250ms) C is capacity of bottleneck link Context Mandated in backbone and edge routers. Appears in RFPs and IETF architectural guidelines.. Usually referenced to Villamizar and Song: “High Performance TCP in ANSNET”, CCR, 1994. Already known by inventors of TCP [Van Jacobson, 1988] Has major consequences for router design Notes: Widely used rule-of-thumb that router per interface needs one delay bw product or 2T*C Router serves link of capacity C, for this interface needs buffer of 2T*C The RFC is 3439 “Some Internet Architectural Guidelines

Only W=2 packets may be outstanding TCP Only W=2 packets may be outstanding Router Source Dest C’ > C C TCP Congestion Window controls the sending rate Sender sends packets, receiver sends ACKs Sending rate is controlled by Window W, At any time, only W unacknowledged packets may be outstanding The sending rate of TCP is Storyline: TCP controls sending rate using Congestion Window W The rule is that you never have more than W packets outstanding These outstanding packets can be in one of three places - the buffer, on the wire or dropped Here is a practical example Rate of TCP is W/RTT. Send W packets, wait one RTT before can send the next one. Don’t: Mention congestion Outstanding packet argument

For every W ACKs received, Single TCP Flow Router with large enough buffers for full link utilization For every W ACKs received, send W+1 packets B Source Dest C’ > C C t Window size RTT Storyline: With no buffer we can’t get full utilization Now let’s assume we have enough buffer for full utilization (and work ourselves backwards to how much that is) (click) If full utilization, bneck link is always full Clocked acks (in pkts/s) Clocked sends Suprising: TCP sends at constant rate, does not vary! It might stop occasionally (e.g. when window scales down) but will either send at C or not at all Occasionally sender increases W – what happens with those extra packets? Go into the buffer If buffer is full, they are dropped, window size reduced Summary if there is enough buffering All links are filled (allmost) all of the time Sender increases window, where do packets go? Extra packets are absorbed by the buffer Buffer compensates for changes in Window size Also follows sawtooth pattern (so does RTT) As R const, W sawtooth, RTT same sawtooth When no of outstanding packets is reduced (W->W/2), buffer has to compensate for scale down

Required buffer is height of sawtooth t

Buffer = rule of thumb R is const as RTT=Window

Over-buffered Link Story: Too much buffering When window size drops, buffer reduced but not empty Always fully utilized But additional latency More would be more latency

Under-buffered Link Story: Again the buffer empties when window is decreased But not enough packets outstanding to fill link Bottleneck link is underutilized

Origin of rule-of-thumb Before and after reducing window size, the sending rate of the TCP sender is the same Inserting the rate equation we get The RTT is part transmission delay T and part queueing delay B/C . We know that after reducing the window, the queueing delay is zero. Don’t: mention one of several ways Spend too much time 

Rule-of-thumb Rule-of-thumb makes sense for one flow Typical backbone link has > 20,000 flows Does the rule-of-thumb still hold? Answer: If flows are perfectly synchronized, then Yes. If flows are desynchronized then No. Comment: Today RoT is appied everywhere

Outline The Rule of Thumb The buffer requirements for a congested router Synchronized flows Desynchronized flows The 2T×C/sqrt(n) rule Buffer requirements for short flows (slow-start) Experimental Verification Conclusion

If flows are synchronized Story: If we have several flows, role of buffer is to absorb fluctuation in the sum of the window sizes In this case three flows, each flow has 1/3 of W of a single flow saturating the router If they are synchronized they add up to one big sawtooth t Aggregate window has same dynamics Therefore buffer occupancy has same dynamics Rule-of-thumb still holds.

When are Flows Synchronized? Small numbers of flows tend to synchronize Large aggregates of flows are not synchronized For > 200 flows, synchronization disappears Measurements in the core give no indication of synchronization

If flows are not synchronized Probability Distribution B Buffer Size Story What happens if flows are not synchronized What I mean with not synchronized is that the congestion windows W(t) independentl of each other What can we say about the sum of the congestion windows? We would expect the fluctuation to be smaller – statistical multiplexing! There is a simple argument we can make A – Same Distr - The congestion windows will have the same distribution as they see the same loss probability B - Indep - Sum of independent random variables should is according to CLT a gaussian Measure This is actually the case Parameters: 800 Mb/s 2000 Flows Each win size in low 10’s.

Quantitative Model Model congestion window of a flow as random variable model as where For many de-synchronized flows We assume congestion windows are independent All congestion windows have the same probability distribution XXX-Script Now central limit theorem gives us the distribution of the sum of the window sizes

Buffer vs. Number of Flows for a given Bandwidth If for a single flow we have For a given C, the window W scales with 1/n and thus Standard deviation of sum of windows decreases with n Story: How does the required buffer depend on number of flows? Let’s assume we know average window and standard deviation for a single flow If we have twice as many flows, each will have have half the mean and half the standard deviation Both scale with 1/n If we plug this into the formula from the last page, we find that standard deviation of the sum of windows should decrease with 1/sqrt(n) We would thus expect the buffer to decrease with 1/sqrt(n) Thus as n increases, buffer size should decrease

Required buffer size Simulation

Summary Flows in the core are desynchronized For desynchronized flows, routers need only buffers of Notes: - Mention contrary to what was previously assumed

Experimental Evaluation Overview Simulation with ns2 Over 10,000 simulations that cover range of settings Simulation time 30s to 5 minutes Bandwidth 10 Mb/s - 1 Gb/s Latency 20ms -250 ms, Physical router Cisco GSR with OC3 line card In collaboration with University of Wisconsin Experimental results presented here Long Flows - Utilization Mixes of flows - Flow Completion Time (FCT) Mixes of flows - Heavy Tailed Flow Distribution Short Flows – Queue Distribution

Long Flows - Utilization (I) Small Buffers are sufficient - OC3 Line, ~100ms RTT 99.9% 99.5% 2× 98.0%

Impact on Router Design 10Gb/s linecard with 200,000 x 56kb/s flows Rule-of-thumb: Buffer = 2.5Gbits Requires external, slow DRAM Becomes: Buffer = 6Mbits Can use on-chip, fast SRAM Completion time halved for short-flows 40Gb/s linecard with 40,000 x 1Mb/s flows Rule-of-thumb: Buffer = 10Gbits Becomes: Buffer = 50Mbits For more details… “Sizing Router Buffers – Guido Appenzeller, Isaac Keslassy and Nick McKeown, to appear at SIGCOMM 2004 Flavio Bonomi: 10%-40% less cost, power, space on board Allows to build cards at higher line rates If we can move buffer on chip, radically changes line card design