A Nonstationary Poisson View of Internet Traffic Thomas Karagiannis joint work with Mart Molle, Michalis Faloutsos, Andre Broido

2 What is the nature of Internet traffic? The fundamental question How does Internet traffic look like?How does Internet traffic look like? Two competing models Poisson and independence assumptionPoisson and independence assumption  Kleinrock (1976) Self-similarity, Long-Range Dependence, heavy tailsSelf-similarity, Long-Range Dependence, heavy tails  Revolutionized modeling  Poisson has largely been discredited

3 The Poisson assumption may still be applicable ! We revisit the question: LRD or Poisson? We focus on Internet coreWe focus on Internet core Things may have changed: massive scale and multiplexingThings may have changed: massive scale and multiplexing Our observations: Packet arrivals appear Poisson and independentPacket arrivals appear Poisson and independent We observe nonstationarity at multi-second time scalesWe observe nonstationarity at multi-second time scales Traffic exhibits LRD properties at scales of seconds and aboveTraffic exhibits LRD properties at scales of seconds and above Our conjecture: Traffic as a nonstationary Poisson process? This view appears to reconcile the multifaceted behaviorThis view appears to reconcile the multifaceted behavior

4 Background: Self-similarity and LRD Self-similarity opens new horizons in traffic modeling On the Self-Similar Nature of Ethernet Traffic. (1994)On the Self-Similar Nature of Ethernet Traffic. (1994)  W. E. Leland, M. S. Taqqu, W. Willinger, and D. V. Wilson. Wide Area Traffic: The Failure of Poisson Modeling. (1995)Wide Area Traffic: The Failure of Poisson Modeling. (1995)  V. Paxson and S. Floyd. Self-similarity through high-variability: statistical analysis of ethernet LAN traffic at the source level (1995)Self-similarity through high-variability: statistical analysis of ethernet LAN traffic at the source level (1995)  W. Willinger, M. S. Taqqu, R. Sherman, and D. V. Wilson. Self-Similarity in World Wide Web Traffic: Evidence and Possible Causes. (1997)Self-Similarity in World Wide Web Traffic: Evidence and Possible Causes. (1997)  M. E. Crovella and A. Bestavros. New tools and models Wavelet Analysis of Long-Range Dependence (1998)Wavelet Analysis of Long-Range Dependence (1998)  P. Abry and D. Veitch.

5 Traces Traces taken by CAIDA monitors at a Tier 1 Internet Service Provider (ISP) OC48 link (2.4Gbps)OC48 link (2.4Gbps) State of the art Dag4 monitorsState of the art Dag4 monitors August 2002, January 2003, April 2003August 2002, January 2003, April 2003 Traces from the WIDE backbone Trans-Pacific 100Mbps link (June 2003)Trans-Pacific 100Mbps link (June 2003)

6 Packet arrivals appear Poisson! Backbone: Interarrival times follow the exponential distribution CCDF is a straight line with 99.99% correlation coefficientCCDF is a straight line with 99.99% correlation coefficient Arrivals appear uncorrelated We examine correlations with several toolsWe examine correlations with several tools CCDF of packet interarrival times (OC48) log(P[X>x])log(P[X>x]) interarrival times (microsec) CCDF of packet interarrival times (100Mbps) interarrival times (microsec)

7 LAN 1989 vs. Backbone 2003 LAN - August 1989 Bellcore tracesBellcore traces The trace that started the LRD revolutionThe trace that started the LRD revolution Backbone - January 2003 Current backbone tracesCurrent backbone traces Packet interarrival distribution

8 At the same time, traffic exhibits LRD properties Statistical tools show LRD at large scales Dichotomy in scaling behavior Hurst exponent at larger scalesHurst exponent at larger scales Abry-Veitch Wavelet estimator

9 Backbone traffic appears smooth but nonstationary at multi-second time-scales Rate changes at second scales Canny Edge Detector algorithm from image processing to detect changes

10 Could nonstationarity appear as LRD? LRD properties diminish when global average is replaced by moving average in ACF

11 How can we reconcile the observed behavior? Observed behavior Poisson packet arrivalsPoisson packet arrivals Nonstationary rate variationNonstationary rate variation Long-range dependenceLong-range dependence Our conjecture: A time-dependent Poisson characterization of traffic when viewed across very long time scales, exhibits the observed long-range dependencewhen viewed across very long time scales, exhibits the observed long-range dependence It has been supported by theoretical workIt has been supported by theoretical work  (e.g., Andersen et al. JSAC ’98)

12 Caveats – Why we don’t have a definitive answer Data collection Duration, representative sampleDuration, representative sample Backbone versus access linkBackbone versus access link Estimation not calculation Tools offer approximations and not definite conclusionsTools offer approximations and not definite conclusions Approaching the truth Different theories may explain different facets of the behavior at different scalesDifferent theories may explain different facets of the behavior at different scales