Bridging Router Performance and Queuing Theory Dina Papagiannaki, Intel Research Cambridge with Nicolas Hohn, Darryl Veitch and Christophe Diot

Intel Research 2 Motivation  End-to-end packet delay is an important metric for performance and SLAs  Building block of end-to-end delay is through router delay  We measure the delays incurred by all packets crossing a single router

Intel Research 3 Overview  Full Router Monitoring  Delay Analysis  Modeling  Delay Performance: Understanding and Reporting  Causes of microcongestion

Intel Research 4 Measurement Environment

Intel Research 5 Full Router Monitoring  Gateway router  2 backbone links (OC-48), 2 domestic customer links (OC- 3, OC-12), 2 Asian customer links (OC-3)  13 hours of trace collection on Aug. 14, 2003  7.3 billion packets – 3 TeraBytes of IP traffic  Monitor more than 99.9% of all through traffic  μs timestamp precision

Intel Research 6 Packet matching SetLink Matched pkts % traffic C2-out C4In % C1In % BB1In % BB2In % C2out %

Intel Research 7 Packet matching (cntd)

Intel Research 8 Overview

Intel Research 9 Store & Forward Datapath  Store: storage in input linecard’s memory  Forwarding decision  Storage in dedicated Virtual Output Queue (VOQ)  Decomposition into fixed-size cells  Transmission through switch fabric cell by cell  Packet reconstruction  Forward: Output link scheduler Not part of the system

Intel Research 10 Delays: 1 minute summary

Intel Research 11 Minimum Transit Time Packet size dependent minimum delay Δ(L), specific to router architecture and linecard technology

Intel Research 12 Store & Forward Datapath  Store: storage in input linecard’s memory  Forwarding decision  Storage in dedicated Virtual Output Queue (VOQ)  Decomposition into fixed-size cells  Transmission through switch fabric cell by cell  Packet reconstruction  Forward: Output link scheduler Not part of the system Δ(L) FIFO queue

Intel Research 13 Overview

Intel Research 14 Modeling

Intel Research 15 Modeling

Intel Research 16 Model Validation

Intel Research 17 Model validation

Intel Research 18 Error as a function of time

Intel Research 19 Modeling results  Our crude model performs well  Use effective link bandwidth (account for encapsulation)  The front end Δ only matters when the output queue is empty  The model defines Busy Periods:  The model defines Busy Periods: time between the arrival of a packet to the empty system and the time when the system becomes empty again.

Intel Research 20 Overview

Intel Research 21 Delay Performance  Packet delays cannot be inferred from output link utilization  Source of large delays: queue build-ups in output buffer  Busy Period structures contain all delay information  Busy Period durations and idle duration contain all utilization information

Intel Research 22 Reporting BP Amplitude

Intel Research 23 Reporting BP Duration

Intel Research 24 Report BP joint distribution

Intel Research 25 Busy periods have a common shape

Intel Research 26 Reporting Busy Periods  Answer performance related questions directly  How long will a given level of congestion last?  Method:  Report partial busy period statistics A and D  Use “triangular shape”

Intel Research 27 Understanding Busy Periods

Intel Research 28 Reporting Busy Periods

Intel Research 29 Summary of modeling part  Results  Full router empirical study  Delay modeling  Reporting performance metrics

Intel Research 30 Overview

Intel Research 31 Causes of microcongestion

Intel Research 32 Stretching and merging Queue Buildup!

Intel Research 33 Causes of microcongestion

Intel Research 34 Multiplexing

Intel Research 35 Causes of microcongestion

Intel Research 36 Traffic Burstiness  Duration and amplitude of busy periods depends on the spacing of packets at the input.  Highly clustered packets at the input are more likely to form busy periods.

Intel Research 37 Busy periods Maximum amplitude: 5 ms Maximum duration: 15 ms 120,000 busy periods > 1 ms tsts D A tAtA

Intel Research 38 Methodology  Run semi-experiments  Simulate busy periods and measure their amplitude A(S, μ) under two different traffic scenarios, one that contains the effect studied and one that does not  Define a metric to quantitatively capture the studied effect

Intel Research 39 Reduction in Bandwidth

Intel Research 40 Amplification factor  Reference stream:  S T : traffic from a single OC-48 link  Output link rate: μ i  Test stream:  S s : traffic from a single OC-48 link  Output link rate: μ o

Intel Research 41 Amplification factor (2)

Intel Research 42 Link multiplexing

Intel Research 43 Link multiplexing  Reference stream:  S T : output link traffic  Output link rate: μ o  Test stream:  S i : traffic from a single OC-48 link  Output link rate: μ o

Intel Research 44 Link multiplexing (2)

Intel Research 45 Flow burstiness Non-bursty flow Bursty flow

Intel Research 46 Flow Burstiness  Reference stream:  S T : input traffic stream from a single OC-48 link  Output link rate: μ o  Test stream:  S j : top 5-tuple flow OR the set of ALL bursty flows  Output link rate: μ o

Intel Research 47 Flow burstiness

Intel Research 48 Summary  Methodology (and metrics) to investigate impact of different congestion mechanisms  In today’s access networks:  Reduction in link bandwidth plays a significant role  Multiplexing has a definite impact since individual links would not have led to similar delays  Flow burstiness does NOT significantly impact delay (bottleneck bandwidths too small to dominate the backbone)  Congestion may be the outcome of network design!

Thank you!

Intel Research 50 References  K. Papagiannaki, S. Moon, C. Fraleigh, P.Thiran, F. Tobagi, C. Diot. Analysis of Measured Single-Hop Delay from an Operational Backbone Network. In IEEE Infocom, New York, U.S.A., June,  N. Hohn, D. Veitch, K. Papagiannaki, C. Diot. Bridging router performance and queuing theory. To appear in ACM Sigmetrics, New York, U.S.A., June,  K. Papagiannaki, D. Veitch, and N. Hohn. Origins of Microcongestion in an Access Router. In Passive & Active Measurement Workshop, Antibes, France, April, 2004.

Intel Research 51 Busy Period Construction