1. www.intel.com/research Bridging Router Performance and Queuing Theory Dina Papagiannaki, Intel Research Cambridge with Nicolas Hohn, Darryl Veitch and Christophe Diot

2. www.intel.com/research Intel Research 2 Motivation  End-to-end packet delay is an important metric for performance and SLAs  Building block of end-to-end delay is through router delay  We measure the delays incurred by all packets crossing a single router

3. www.intel.com/research Intel Research 3 Overview  Full Router Monitoring  Delay Analysis  Modeling  Delay Performance: Understanding and Reporting  Causes of microcongestion

4. www.intel.com/research Intel Research 4 Measurement Environment

5. www.intel.com/research Intel Research 5 Full Router Monitoring  Gateway router  2 backbone links (OC-48), 2 domestic customer links (OC- 3, OC-12), 2 Asian customer links (OC-3)  13 hours of trace collection on Aug. 14, 2003  7.3 billion packets – 3 TeraBytes of IP traffic  Monitor more than 99.9% of all through traffic  μs timestamp precision

6. www.intel.com/research Intel Research 6 Packet matching SetLink Matched pkts % traffic C2-out C4In2159870.03% C1In703760.01% BB1In34579662247.00% BB2In38915377252.89% C2out735236757 99.93%

7. www.intel.com/research Intel Research 7 Packet matching (cntd)

8. www.intel.com/research Intel Research 8 Overview

9. www.intel.com/research Intel Research 9 Store & Forward Datapath  Store: storage in input linecard’s memory  Forwarding decision  Storage in dedicated Virtual Output Queue (VOQ)  Decomposition into fixed-size cells  Transmission through switch fabric cell by cell  Packet reconstruction  Forward: Output link scheduler Not part of the system

10. www.intel.com/research Intel Research 10 Delays: 1 minute summary

11. www.intel.com/research Intel Research 11 Minimum Transit Time Packet size dependent minimum delay Δ(L), specific to router architecture and linecard technology

12. www.intel.com/research Intel Research 12 Store & Forward Datapath  Store: storage in input linecard’s memory  Forwarding decision  Storage in dedicated Virtual Output Queue (VOQ)  Decomposition into fixed-size cells  Transmission through switch fabric cell by cell  Packet reconstruction  Forward: Output link scheduler Not part of the system Δ(L) FIFO queue

13. www.intel.com/research Intel Research 13 Overview

14. www.intel.com/research Intel Research 14 Modeling

15. www.intel.com/research Intel Research 15 Modeling

16. www.intel.com/research Intel Research 16 Model Validation

17. www.intel.com/research Intel Research 17 Model validation

18. www.intel.com/research Intel Research 18 Error as a function of time

19. www.intel.com/research Intel Research 19 Modeling results  Our crude model performs well  Use effective link bandwidth (account for encapsulation)  The front end Δ only matters when the output queue is empty  The model defines Busy Periods:  The model defines Busy Periods: time between the arrival of a packet to the empty system and the time when the system becomes empty again.

20. www.intel.com/research Intel Research 20 Overview

21. www.intel.com/research Intel Research 21 Delay Performance  Packet delays cannot be inferred from output link utilization  Source of large delays: queue build-ups in output buffer  Busy Period structures contain all delay information  Busy Period durations and idle duration contain all utilization information

22. www.intel.com/research Intel Research 22 Reporting BP Amplitude

23. www.intel.com/research Intel Research 23 Reporting BP Duration

24. www.intel.com/research Intel Research 24 Report BP joint distribution

25. www.intel.com/research Intel Research 25 Busy periods have a common shape

26. www.intel.com/research Intel Research 26 Reporting Busy Periods  Answer performance related questions directly  How long will a given level of congestion last?  Method:  Report partial busy period statistics A and D  Use “triangular shape”

27. www.intel.com/research Intel Research 27 Understanding Busy Periods

28. www.intel.com/research Intel Research 28 Reporting Busy Periods

29. www.intel.com/research Intel Research 29 Summary of modeling part  Results  Full router empirical study  Delay modeling  Reporting performance metrics

30. www.intel.com/research Intel Research 30 Overview

31. www.intel.com/research Intel Research 31 Causes of microcongestion

32. www.intel.com/research Intel Research 32 Stretching and merging Queue Buildup!

33. www.intel.com/research Intel Research 33 Causes of microcongestion

34. www.intel.com/research Intel Research 34 Multiplexing

35. www.intel.com/research Intel Research 35 Causes of microcongestion

36. www.intel.com/research Intel Research 36 Traffic Burstiness  Duration and amplitude of busy periods depends on the spacing of packets at the input.  Highly clustered packets at the input are more likely to form busy periods.

37. www.intel.com/research Intel Research 37 Busy periods Maximum amplitude: 5 ms Maximum duration: 15 ms 120,000 busy periods > 1 ms tsts D A tAtA

38. www.intel.com/research Intel Research 38 Methodology  Run semi-experiments  Simulate busy periods and measure their amplitude A(S, μ) under two different traffic scenarios, one that contains the effect studied and one that does not  Define a metric to quantitatively capture the studied effect

39. www.intel.com/research Intel Research 39 Reduction in Bandwidth

40. www.intel.com/research Intel Research 40 Amplification factor  Reference stream:  S T : traffic from a single OC-48 link  Output link rate: μ i  Test stream:  S s : traffic from a single OC-48 link  Output link rate: μ o

41. www.intel.com/research Intel Research 41 Amplification factor (2)

42. www.intel.com/research Intel Research 42 Link multiplexing

43. www.intel.com/research Intel Research 43 Link multiplexing  Reference stream:  S T : output link traffic  Output link rate: μ o  Test stream:  S i : traffic from a single OC-48 link  Output link rate: μ o

44. www.intel.com/research Intel Research 44 Link multiplexing (2)

45. www.intel.com/research Intel Research 45 Flow burstiness Non-bursty flow Bursty flow

46. www.intel.com/research Intel Research 46 Flow Burstiness  Reference stream:  S T : input traffic stream from a single OC-48 link  Output link rate: μ o  Test stream:  S j : top 5-tuple flow OR the set of ALL bursty flows  Output link rate: μ o

47. www.intel.com/research Intel Research 47 Flow burstiness

48. www.intel.com/research Intel Research 48 Summary  Methodology (and metrics) to investigate impact of different congestion mechanisms  In today’s access networks:  Reduction in link bandwidth plays a significant role  Multiplexing has a definite impact since individual links would not have led to similar delays  Flow burstiness does NOT significantly impact delay (bottleneck bandwidths too small to dominate the backbone)  Congestion may be the outcome of network design!

49. www.intel.com/research Thank you!

50. www.intel.com/research Intel Research 50 References  K. Papagiannaki, S. Moon, C. Fraleigh, P.Thiran, F. Tobagi, C. Diot. Analysis of Measured Single-Hop Delay from an Operational Backbone Network. In IEEE Infocom, New York, U.S.A., June, 2002.  N. Hohn, D. Veitch, K. Papagiannaki, C. Diot. Bridging router performance and queuing theory. To appear in ACM Sigmetrics, New York, U.S.A., June, 2004.  K. Papagiannaki, D. Veitch, and N. Hohn. Origins of Microcongestion in an Access Router. In Passive & Active Measurement Workshop, Antibes, France, April, 2004.

51. www.intel.com/research Intel Research 51 Busy Period Construction