The Joint Distribution of Internet Flow Sizes and Durations C HEOLWOO P ARK J. S TEPHEN M ARRON The University of North Carolina at Chapel Hill

The Joint Distribution of Internet Flow Sizes and Durations Motivation of the study Data description, scatter plots and density estimation Correlation plots Conclusions and future plans

The Joint Distribution of Internet Flow Sizes and Durations Started from conflict between two papers Extremal Dependence: Internet Traffic Applications (2002) - Felix Hernandez Campos, J. S. Marron, Sidney I. Resnick and Kevin Jeffay On the Characteristics and Origins of Internet Flow Rates (2002) - Yin Zhang, Lee Breslau, Vern Paxson and Scott Shenker, SIGCOMM’02

Why interested in this topic? Size and rate are naturally considered as independent Users determine sizes of files transferred depending on their available bandwidths? Modeling of Internet traffic The Joint Distribution of Internet Flow Sizes and Durations

Nearly contradictory answers! The Joint Distribution of Internet Flow Sizes and Durations Hernandez-Campos et al. (EDA) Zhang et al. (log-log Correlations) S vs. D Inconclusive (0.50 ~ 0.59) Inconclusive (0.10 ~ 0.30) S vs. R Independent (0.23 ~ 0.31) Dependent (0.84 ~ 0.89) D vs. IR Dependent (0.69 ~ 0.71) Inconclusive (0.18 ~ 0.45) Different earlier analyses of Internet Flow Sizes and Durations: S: Size, D: Duration, R (=S/D): Rate, IR: Inverse Rate

Why? Possibilities: Data from different sources? Different types of data? (HTTP Resp. vs all web traces) Different correlation measure? The Joint Distribution of Internet Flow Sizes and Durations Hernandez-Campos et al. (log-log Correlations) Zhang et al. (log-log Correlations) S vs. D Inconclusive (0.65) Inconclusive (0.10 ~ 0.30) S vs. R Independent (-0.06) Dependent (0.84 ~ 0.89) D vs. IR Dependent (0.80) Inconclusive (0.18 ~ 0.45) Different threshold values?

Threshold values: The Joint Distribution of Internet Flow Sizes and Durations applied thresholding to different variables used different threshold values Hernandez-Campos et al.Zhang et al. Size> 100 Kbytes> 0 bytes Duration> 0 sec> 5 sec

The Joint Distribution of Internet Flow Sizes and Durations Motivation of the study Data description, scatter plots and density estimation Correlation plots Conclusions and future plans

Data : HTTP responses Sunday Morning (8:00 AM – 12:00 PM) In April 2001 From UNC Main Link The Joint Distribution of Internet Flow Sizes and Durations Variables of Interest: S : Size (bytes) D : Duration (time in seconds) R : Rate (throughput, byte/sec) IR : Inverse Rate (sec/byte)

Scatterplot log 10 (Size) vs. log 10 (Duration) The Joint Distribution of Internet Flow Sizes and Durations

Scatterplot log 10 (Size) vs. log 10 (Rate) The Joint Distribution of Internet Flow Sizes and Durations

Scatterplot log 10 (Duration) vs. log 10 (Inv. Rate) The Joint Distribution of Internet Flow Sizes and Durations

Motivation of the Study Data description and scatter plots Log-log correlation plots with global thresholdings Conclusions and future plans

The Joint Distribution of Internet Flow Sizes and Durations log 10 (Size) vs. log 10 (Duration)

The Joint Distribution of Internet Flow Sizes and Durations log 10 (Size) vs. log 10 (Rate)

The Joint Distribution of Internet Flow Sizes and Durations log 10 (Duration) vs. log 10 (Inv. Rate)

The Joint Distribution of Internet Flow Sizes and Durations log 10 (Size) vs. log 10 (Rate) Simulated bivariate normal

The Joint Distribution of Internet Flow Sizes and Durations Motivation of the Study Data description and scatter plots Log-log correlation plots with global thresholdings Conclusions and future plans

Conclusions: The blind men and the elephant Thresholding is CRITICAL The Joint Distribution of Internet Flow Sizes and Durations

Deeper investigation: What values should we use ? On Size ? On Duration ? On Both ? How to handle 0 durations ? Which methods are robust to thresholding? The Joint Distribution of Internet Flow Sizes and Durations