1 Why is the Internet traffic bursty in short time scales? Constantine Dovrolis Hao Jiang College of Computing Georgia Institute of Technology
2 The many faces of traffic burstiness FACT: Internet traffic is bursty in very wide range of time scales (microseconds to hours) “Burstiness” can be related to different statistical aspects of the traffic process: Variance of marginal distribution in certain time scale E.g., Gamma renewal process is burstier than Poisson process (with same average rate) Strong correlations in interarrivals or counts E.g., packet-trains or ON-OFF user behavior Scaling behavior in a range of time scales E.g., IID process: variance decreases with time scale T -1 E.g., Self-similar process: variance decreases with T -2(1-H), H: Hurst parameter (0.5<H<1)
3 Burstiness in large/medium time scales Well understood cause-effect relationship in large scales (> seconds) Leland et al. (TNet’94): asymptotically self-similar scaling and LRD behavior, assuming stationarity Major causes: Willinger et al. (Sigcomm’95): heavy-tailed ON-OFF behavior in packet interarrivals Crovella/Bestavros (TNet’99): heavy-tailed ON-OFF behavior in Web transfers Rather well understood behavior also in medium scales (in the order of seconds): Figueiredo et al. (CompNets’02): pseudo-self similar behavior in a range of medium time scales From a single RTT to tens/hundreds of RTTs Cause: strong correlations due to TCP congestion control and timeouts
4 Burstiness in short (sub-RTT) time scales Several (rather contradictory) proposed traffic models: Multifractal & non-Gaussian model (Riedi et al. TransIT‘99, Feldmann et al. Sigcomm’98.) Monofractal & Gaussian (Zhang et al. Infocom’03) Cluster process with Gamma interarrivals (Hohn et al.TransSP’03) Nonstationary Poisson (Karagiannis et al. Infocom’04) More importantly, however, the open question is: What causes short-scale (sub-RTT) burstiness? TCP ACK compression (Feldmann et al. Sigcomm ’99) “Dense” (i.e., highly variable) interarrivals (Zhang et al. Infocom ’02) Our objectives: Identify major cause(s) for sub-RTT burstiness of aggregate Internet traffic Relate flow characteristics (capacity, RTT, window size, flow size) with resulting burstiness Explain why some of the previous measurements resulted in contradictory models
5 Main questions What is the major cause for sub-RTT burstiness in aggregate Internet traffic? Individual flows or of aggregate traffic? If individual TCP flows cause sub-RTT burstiness, which component of the TCP protocol is to blame? Congestion control? Self-clocking? ACK compression? Which TCP flows are mostly responsible for short scale burstiness? Large? Dense? High-capacity? Is there a practical way to reduce short scale burstiness? Traffic shaping? Change in TCP? What is the effect of traffic multiplexing (aggregation) on short scale burstiness? Does aggregation produce “Poisson-like” traffic? What is the impact of short scale burstiness on queueing performance? Conventional wisdom: “LRD behavior dominates queueing performance”
6 Overview Wavelet-based MRA and burstiness definition Sub-RTT ON-OFF behavior due to TCP self- clocking Case study: the burstiness of an OC-48 trace Smoothing effect of TCP pacing
7 Multi-Resolution Analysis (MRA) Analyze variability of traffic process in successive time scales T j = 2 j T 0 :
8 Wavelet-based MRA (Abry-Veitch) Traffic process in time scale T j = 2 j T 0 : where X j k is amount of traffic in Energy of X j (for Haar wavelet): Energy of Poisson process of rate λ: E j = λT 0 (constant) Energy of periodic process in scale T j : E j = 0 (a periodicity causes energy drop) Energy plot: logE j vs j, j=0,1,2,…
9 An unusual definition of traffic burstiness A traffic process X j is bursty at scale j if the energy of X j is higher than the energy of Poisson process with same average rate Example: energy plot of an OC-48 Abilene trace
10 Overview Wavelet-based MRA and a burstiness definition Sub-RTT ON-OFF behavior due to TCP self-clocking Case study: the burstiness of an OC-48 trace Smoothing effect of TCP pacing
11 TCP self-clocking Can a single TCP flow create bursty traffic in sub-RTT scales, and if so, under which condition? Main TCP parameters: W: send-window size (segs)T: round-trip time L:MSS C: path capacity CT: bandwidth-delay product Ideal TCP sending behavior within a single RTT: Driven by self-clocking and delayed-ACKs Basic operation: each (new) received ACK triggers the back-to- back transmission of at least two new segments Interarrivals of back-to-back segments after bottleneck: L/C Interarrivals of generated ACKs at receiver: 2*L/C Without ACK compression, ACKs will arrive at sender with same interarrival
12 Almost all interarrivals of data segments are equal to L/C The only interarrivals that are larger than L/C are between successive windows
13 Self-clocking model without cross traffic: case 1 WL < CT, one level ON/OFF
14 Self-clocking model without cross traffic: case 2 WL >= CT, periodic interarrivals
15 Vantage point: about 50% of interarrivals at L/C p due to delayed ACKs Window is split in clusters of bursts, due to interfering cross traffic
16 Self-clocking with cross traffic Within each RTT, send-window is segmented into a number of bursts, with each burst being two or more packets sent back- to-back Basically, a two-level ON-OFF process Window duration: Δ, K bursts, burst length: B i
17 Energy plot for TCP self-clocking with cross traffic
18 Summary so far.. If TCP’s send-window is less than the flow’s bandwidth-delay product (W < CT), TCP generates bursty traffic in sub-RTT scales Without cross traffic, one-level ON-OFF process With cross traffic, two-level ON-OFF process Otherwise, if W > CT, traffic is almost periodic Can we verify the previous observations based on the analysis of a real Internet trace?
19 Overview Wavelet-based MRA and a burstiness definition Sub-RTT ON-OFF behavior due to TCP self-clocking Case study: the burstiness of an OC-48 trace Smoothing effect of TCP pacing
20 Case study: burstiness of OC-48 trace Goal: identify minimal set of flows that are responsible for short scale burstiness in aggregate traffic Also, relate flow characteristics (RTT, capacity, flow size, window size) with the resulting energy plot of the aggregate traffic Definitions: Mice: flow size < 15KB Burstiness ratio = CT/W Subset GByte Flows all TCP T T, C elephant mice Large ratio Small ratio
21 Traffic is bursty across (almost) all time scales Energy plots of original traffic and TCP subset are the same Non-TCP traffic does not affect burstiness of aggregate
22 Distribution of per-flow RTTs Estimation technique: Jiang-Dovrolis, ACM CCR’02
23 Distribution of per-flow capacities Estimation technique: see Jiang-Dovrolis PAM’04
24 The dip in the 12-th scale reflects a periodicity around 200ms Weighted average RTT of TCP traffic
25 Large flows determine the shape of the aggregate energy plot Small flows are also bursty, but they do not affect the energy plot of the aggregate traffic
26 Distribution of BDP/W ratio for large flows Ratio is quite larger than 1.0 for most of the large flows As little as 5% traffic has ratio close to 1.0
27 Bulk flows with large BDP/W ratio create sub-RTT burstiness Bulk flows with small BDP/W ratio are smooth in sub-RTT scales
28 Summary so far.. Analysis of real aggregate Internet traffic confirms that short scale burstiness is due to the following kind of traffic: 1.Individual TCP flows 2.Large size (bulk transfers) 3.Large BDP relative to the average window size Also, the extent of the short scale burstiness is related to the (effective) RTT of the TCP traffic Identified in the energy plot as a dip at that time scale
29 Overview Wavelet-based MRA and a burstiness definition Sub-RTT ON-OFF behavior due to TCP self-clocking Case study: the burstiness of an OC-48 trace Smoothing effect of TCP pacing
30 Smoothing effect of TCP pacing Fundamental issue with TCP self-clocking: May send a window as a burst or a cluster of bursts Packet transmissions are not “spread” during RTT Pacing is an alternative to self-clocking: Transmit packets periodically during RTT Driven by OS timer at sender Ideal pacing Arbitrarily small granularity But timer overhead is too high Practical pacing Timer granularity T c is typically 1ms or 10ms Send m packets every nT c time units
31 Ideal pacing of individual TCP flows makes traffic smoother than Poisson in sub-RTT scales
32 The 1ms timer is effective in reducing short scale burstiness The 10ms timer may be unable to eliminate short scale burstiness
33 Conclusions What is the major cause for short scale burstiness in aggregate Internet traffic? ON-OFF packet transmission pattern in large TCP flows If individual TCP flows cause most of the burstiness, which component of the TCP protocol is to blame? Self-clocking Which TCP flows are mostly responsible for short scale burstiness? Large TCP flows with bandwidth-delay product > average flow’s window Is there a practical way to reduce short scale burstiness? Yes, pacing at the sender instead of self-clocking What is the effect of traffic multiplexing (aggregation) on short scale burstiness? Aggregation reduces the CoV of the marginal distribution, but it does not remove the correlations in the packet interarrivals (i.e., the traffic does not converge to Poisson process) What is the impact of short scale burstiness on queueing performance? Sub-RTT burstiness is important in moderate load conditions, but also in high-load conditions when the bottleneck buffer is small
34 Thanks
35 more slides
36 Background and related work Wavelet-based multi-resolution analysis Sub-RTT ON/OFF behavior due to TCP self-clocking Effects of aggregation Case study: the burstiness of an OC-48 trace Smoothing effect of TCP pacing Queueing performance
37 Effect of aggregation Aggregate if X and Y independent Flows that do not have significant energy relative to the aggregate do not have a major impact on the burstiness of the aggregate
38 Energy plot of aggregate maintains the shape as that of a single constituent, independent of N Correlation does not die out with degree of aggregation although statistical multiplexing gain exists
39 Point process theorem: the aggregation of N independent point processes converges to Poisson process as N increase Contradiction ? Theorem assumes the rate of each constituent flow becomes smaller as N increases, i.e., the rate of aggregate is constant independent of N
40 Background and related work Wavelet-based multi-resolution analysis Sub-RTT ON/OFF behavior due to TCP self-clocking Effects of aggregation Case study: the burstiness of an OC-48 trace Smoothing effect of TCP pacing Queueing performance
41 Queueing performance Large buffer vs. small buffer Heavy-load condition vs. moderate-load condition Setup Randomly sample input OC-48 trace to achieve desired utilization
42 95-th percentile of queue size (Buffer=10MB) Sub-RTT burstiness matters in moderate loading conditions In heavy-load conditions, LRD is more important
43 Loss rate vs. buffer size (utilization=0.95) Loss rate with pacing is significantly lower for small buffer size
44 Loss rate vs. offered load (Buffer=50KB) Sub-RTT burstiness matters even in heavy-load conditions for underbuffered link
45 TCP flow: C p = C = 100Mbps,L=1500B, 4.5sec, 1000pkts About 80% of interarrivals are within a factor of two from L/C
46 Most interarrivals at L/C p and at (2L/C - L/C p )
47 TCP flow: L=1500B, 37sec, 4200pkts C p = 100Mbps, C = 1.3Mbps (notice two interarrival modes)
48
49 TCP flow: L=1500B, 25sec, 5400pkts C p =100Mbps (note 50% of interarrivals at L/C p )