1. 1 Why is the Internet traffic bursty in short time scales? Constantine Dovrolis Hao Jiang College of Computing Georgia Institute of Technology

2. 2 The many faces of traffic burstiness ŸFACT: Internet traffic is bursty in very wide range of time scales (microseconds to hours) Ÿ“Burstiness” can be related to different statistical aspects of the traffic process: ŸVariance of marginal distribution in certain time scale ŸE.g., Gamma renewal process is burstier than Poisson process (with same average rate) ŸStrong correlations in interarrivals or counts ŸE.g., packet-trains or ON-OFF user behavior ŸScaling behavior in a range of time scales ŸE.g., IID process: variance decreases with time scale T -1 ŸE.g., Self-similar process: variance decreases with T -2(1-H), H: Hurst parameter (0.5<H<1)

3. 3 Burstiness in large/medium time scales ŸWell understood cause-effect relationship in large scales (> seconds) ŸLeland et al. (TNet’94): asymptotically self-similar scaling and LRD behavior, assuming stationarity ŸMajor causes: ŸWillinger et al. (Sigcomm’95): heavy-tailed ON-OFF behavior in packet interarrivals ŸCrovella/Bestavros (TNet’99): heavy-tailed ON-OFF behavior in Web transfers ŸRather well understood behavior also in medium scales (in the order of seconds): ŸFigueiredo et al. (CompNets’02): pseudo-self similar behavior in a range of medium time scales ŸFrom a single RTT to tens/hundreds of RTTs ŸCause: strong correlations due to TCP congestion control and timeouts

4. 4 Burstiness in short (sub-RTT) time scales ŸSeveral (rather contradictory) proposed traffic models: ŸMultifractal & non-Gaussian model (Riedi et al. TransIT‘99, Feldmann et al. Sigcomm’98.) ŸMonofractal & Gaussian (Zhang et al. Infocom’03) ŸCluster process with Gamma interarrivals (Hohn et al.TransSP’03) ŸNonstationary Poisson (Karagiannis et al. Infocom’04) ŸMore importantly, however, the open question is: What causes short-scale (sub-RTT) burstiness? ŸTCP ACK compression (Feldmann et al. Sigcomm ’99) Ÿ“Dense” (i.e., highly variable) interarrivals (Zhang et al. Infocom ’02) ŸOur objectives: ŸIdentify major cause(s) for sub-RTT burstiness of aggregate Internet traffic ŸRelate flow characteristics (capacity, RTT, window size, flow size) with resulting burstiness ŸExplain why some of the previous measurements resulted in contradictory models

5. 5 Main questions ŸWhat is the major cause for sub-RTT burstiness in aggregate Internet traffic? ŸIndividual flows or of aggregate traffic? ŸIf individual TCP flows cause sub-RTT burstiness, which component of the TCP protocol is to blame? ŸCongestion control? Self-clocking? ACK compression? ŸWhich TCP flows are mostly responsible for short scale burstiness? ŸLarge? Dense? High-capacity? ŸIs there a practical way to reduce short scale burstiness? ŸTraffic shaping? Change in TCP? ŸWhat is the effect of traffic multiplexing (aggregation) on short scale burstiness? ŸDoes aggregation produce “Poisson-like” traffic? ŸWhat is the impact of short scale burstiness on queueing performance? ŸConventional wisdom: “LRD behavior dominates queueing performance”

6. 6 Overview ŸWavelet-based MRA and burstiness definition ŸSub-RTT ON-OFF behavior due to TCP self- clocking ŸCase study: the burstiness of an OC-48 trace ŸSmoothing effect of TCP pacing

7. 7 Multi-Resolution Analysis (MRA) Analyze variability of traffic process in successive time scales T j = 2 j T 0 :

8. 8 Wavelet-based MRA (Abry-Veitch) Traffic process in time scale T j = 2 j T 0 : where X j k is amount of traffic in Energy of X j (for Haar wavelet): Energy of Poisson process of rate λ: E j = λT 0 (constant) Energy of periodic process in scale T j : E j = 0 (a periodicity causes energy drop) Energy plot: logE j vs j, j=0,1,2,…

9. 9 An unusual definition of traffic burstiness ŸA traffic process X j is bursty at scale j if the energy of X j is higher than the energy of Poisson process with same average rate ŸExample: energy plot of an OC-48 Abilene trace

10. 10 Overview ŸWavelet-based MRA and a burstiness definition ŸSub-RTT ON-OFF behavior due to TCP self-clocking ŸCase study: the burstiness of an OC-48 trace ŸSmoothing effect of TCP pacing

11. 11 TCP self-clocking ŸCan a single TCP flow create bursty traffic in sub-RTT scales, and if so, under which condition? ŸMain TCP parameters: ŸW: send-window size (segs)T: round-trip time ŸL:MSS C: path capacity ŸCT: bandwidth-delay product ŸIdeal TCP sending behavior within a single RTT: ŸDriven by self-clocking and delayed-ACKs ŸBasic operation: each (new) received ACK triggers the back-to- back transmission of at least two new segments ŸInterarrivals of back-to-back segments after bottleneck: L/C ŸInterarrivals of generated ACKs at receiver: 2*L/C ŸWithout ACK compression, ACKs will arrive at sender with same interarrival

12. 12 ŸAlmost all interarrivals of data segments are equal to L/C ŸThe only interarrivals that are larger than L/C are between successive windows

13. 13 Self-clocking model without cross traffic: case 1 WL < CT, one level ON/OFF

14. 14 Self-clocking model without cross traffic: case 2 WL >= CT, periodic interarrivals

15. 15 ŸVantage point: about 50% of interarrivals at L/C p due to delayed ACKs ŸWindow is split in clusters of bursts, due to interfering cross traffic

16. 16 Self-clocking with cross traffic ŸWithin each RTT, send-window is segmented into a number of bursts, with each burst being two or more packets sent back- to-back ŸBasically, a two-level ON-OFF process ŸWindow duration: Δ, K bursts, burst length: B i

17. 17 Energy plot for TCP self-clocking with cross traffic

18. 18 Summary so far.. ŸIf TCP’s send-window is less than the flow’s bandwidth-delay product (W < CT), TCP generates bursty traffic in sub-RTT scales ŸWithout cross traffic, one-level ON-OFF process ŸWith cross traffic, two-level ON-OFF process ŸOtherwise, if W > CT, traffic is almost periodic ŸCan we verify the previous observations based on the analysis of a real Internet trace?

19. 19 Overview ŸWavelet-based MRA and a burstiness definition ŸSub-RTT ON-OFF behavior due to TCP self-clocking ŸCase study: the burstiness of an OC-48 trace ŸSmoothing effect of TCP pacing

20. 20 Case study: burstiness of OC-48 trace ŸGoal: identify minimal set of flows that are responsible for short scale burstiness in aggregate traffic ŸAlso, relate flow characteristics (RTT, capacity, flow size, window size) with the resulting energy plot of the aggregate traffic ŸDefinitions: ŸMice: flow size < 15KB ŸBurstiness ratio = CT/W Subset GByte Flows all4.37458669 TCP4.23458669 T 2.4140885 T, C 2.2510484 elephant 2.223207 mice 0.037277 Large ratio 2.123124 Small ratio 0.1084

21. 21 ŸTraffic is bursty across (almost) all time scales ŸEnergy plots of original traffic and TCP subset are the same ŸNon-TCP traffic does not affect burstiness of aggregate

22. 22 Distribution of per-flow RTTs Estimation technique: Jiang-Dovrolis, ACM CCR’02

23. 23 ŸDistribution of per-flow capacities ŸEstimation technique: see Jiang-Dovrolis PAM’04

24. 24 The dip in the 12-th scale reflects a periodicity around 200ms ŸWeighted average RTT of TCP traffic

25. 25 ŸLarge flows determine the shape of the aggregate energy plot ŸSmall flows are also bursty, but they do not affect the energy plot of the aggregate traffic

26. 26 Distribution of BDP/W ratio for large flows ŸRatio is quite larger than 1.0 for most of the large flows ŸAs little as 5% traffic has ratio close to 1.0

27. 27 ŸBulk flows with large BDP/W ratio create sub-RTT burstiness ŸBulk flows with small BDP/W ratio are smooth in sub-RTT scales

28. 28 Summary so far.. ŸAnalysis of real aggregate Internet traffic confirms that short scale burstiness is due to the following kind of traffic: 1.Individual TCP flows 2.Large size (bulk transfers) 3.Large BDP relative to the average window size ŸAlso, the extent of the short scale burstiness is related to the (effective) RTT of the TCP traffic ŸIdentified in the energy plot as a dip at that time scale

29. 29 Overview ŸWavelet-based MRA and a burstiness definition ŸSub-RTT ON-OFF behavior due to TCP self-clocking ŸCase study: the burstiness of an OC-48 trace ŸSmoothing effect of TCP pacing

30. 30 Smoothing effect of TCP pacing ŸFundamental issue with TCP self-clocking: ŸMay send a window as a burst or a cluster of bursts ŸPacket transmissions are not “spread” during RTT ŸPacing is an alternative to self-clocking: ŸTransmit packets periodically during RTT ŸDriven by OS timer at sender ŸIdeal pacing ŸArbitrarily small granularity ŸBut timer overhead is too high ŸPractical pacing ŸTimer granularity T c is typically 1ms or 10ms ŸSend m packets every nT c time units

31. 31 ŸIdeal pacing of individual TCP flows makes traffic smoother than Poisson in sub-RTT scales

32. 32 ŸThe 1ms timer is effective in reducing short scale burstiness ŸThe 10ms timer may be unable to eliminate short scale burstiness

33. 33 Conclusions ŸWhat is the major cause for short scale burstiness in aggregate Internet traffic? ŸON-OFF packet transmission pattern in large TCP flows ŸIf individual TCP flows cause most of the burstiness, which component of the TCP protocol is to blame? ŸSelf-clocking ŸWhich TCP flows are mostly responsible for short scale burstiness? ŸLarge TCP flows with bandwidth-delay product > average flow’s window ŸIs there a practical way to reduce short scale burstiness? ŸYes, pacing at the sender instead of self-clocking ŸWhat is the effect of traffic multiplexing (aggregation) on short scale burstiness? ŸAggregation reduces the CoV of the marginal distribution, but it does not remove the correlations in the packet interarrivals (i.e., the traffic does not converge to Poisson process) ŸWhat is the impact of short scale burstiness on queueing performance? ŸSub-RTT burstiness is important in moderate load conditions, but also in high-load conditions when the bottleneck buffer is small

34. 34 Thanks

35. 35 more slides

36. 36 ŸBackground and related work ŸWavelet-based multi-resolution analysis ŸSub-RTT ON/OFF behavior due to TCP self-clocking ŸEffects of aggregation ŸCase study: the burstiness of an OC-48 trace ŸSmoothing effect of TCP pacing ŸQueueing performance

37. 37 Effect of aggregation Aggregate if X and Y independent ŸFlows that do not have significant energy relative to the aggregate do not have a major impact on the burstiness of the aggregate

38. 38 ŸEnergy plot of aggregate maintains the shape as that of a single constituent, independent of N ŸCorrelation does not die out with degree of aggregation although statistical multiplexing gain exists

39. 39 ŸPoint process theorem: the aggregation of N independent point processes converges to Poisson process as N increase ŸContradiction ? ŸTheorem assumes the rate of each constituent flow becomes smaller as N increases, i.e., the rate of aggregate is constant independent of N

40. 40 ŸBackground and related work ŸWavelet-based multi-resolution analysis ŸSub-RTT ON/OFF behavior due to TCP self-clocking ŸEffects of aggregation ŸCase study: the burstiness of an OC-48 trace ŸSmoothing effect of TCP pacing ŸQueueing performance

41. 41 Queueing performance ŸLarge buffer vs. small buffer ŸHeavy-load condition vs. moderate-load condition ŸSetup ŸRandomly sample input OC-48 trace to achieve desired utilization

42. 42 95-th percentile of queue size (Buffer=10MB) ŸSub-RTT burstiness matters in moderate loading conditions ŸIn heavy-load conditions, LRD is more important

43. 43 Loss rate vs. buffer size (utilization=0.95) ŸLoss rate with pacing is significantly lower for small buffer size

44. 44 Loss rate vs. offered load (Buffer=50KB) ŸSub-RTT burstiness matters even in heavy-load conditions for underbuffered link

45. 45 ŸTCP flow: C p = C = 100Mbps,L=1500B, 4.5sec, 1000pkts ŸAbout 80% of interarrivals are within a factor of two from L/C

46. 46 ŸMost interarrivals at L/C p and at (2L/C - L/C p )

47. 47 ŸTCP flow: L=1500B, 37sec, 4200pkts ŸC p = 100Mbps, C = 1.3Mbps (notice two interarrival modes)

48. 48

49. 49 ŸTCP flow: L=1500B, 25sec, 5400pkts ŸC p =100Mbps (note 50% of interarrivals at L/C p )