Multiscale Traffic Processing Techniques for Network Inference and Control R. Baraniuk R. Nowak E. Knightly R. Riedi V. Ribeiro S. Sarvotham A. Keshavarz NMS PI meeting San Diego May 2003 SPiN.Rice.edu: Signal Processing in Networking

Rice University, SPiN Group spin.rice.edu Chirp Probing always-on, non-intrusive bandwidth estimation on-line decision anomaly detection Effort 1

Rice University, SPiN Group spin.rice.edu Milestones pathChirp May 2003 milestone: C code testing + distribution [completed] –Allen McIntosh (Telcordia): testing, useful comments, –SPAWAR (hosted Phuong Nguyen at Rice), –CAIDA (kc Claffy, Margaret Murray), –RPI (Shivkumar). –Demo: on-line estimation of bandwidth Towards Nov 2003 milestones: Integration of pathChirp –GaTech pdns (Riley/Fujimoto) [completed] –UIUC, JavaSim (Hou) [in progress] Validation in controllable environment [enabled, to be done] Towards May 2004 milestones: [basis provided, work to be done] Final tool and theory for bursty traffic over multiple hops

Rice University, SPiN Group spin.rice.edu Efficient probing: PathChirp Traditional probing paradigm: –Produce (light) congestion –PacketPair: Sample the traffic –Pathload: flood at variable rate intolerable level of congestion –TOPP: PacketPairs at variable spacing New: –PathChirp: Variable rate within a train of probes More efficient, light

Rice University, SPiN Group spin.rice.edu pathChirp Developments pathChirp …a real world tool …with improved performance –Increased queuing delay correlates with cross traffic on network path –Last excursion in chirp  link capacity –Weighted averaged of onset of excursions  available link resources Departure pattern Queuing against departure Methodology

Rice University, SPiN Group spin.rice.edu pathChirp Developments pathChirp …a real world tool …with improved performance –Queuing delay  cross traffic –Final excursion  link capacity –Averaged excursions  available resources …converges in a handful of RTTs Departure pattern Queuing against departure Methodology Number of chirps  12 chirps Real world experiments Estimation against true x-traffic Internet experiment

Rice University, SPiN Group spin.rice.edu PathChirp Performance pathChirp performs comparably to –PathLoad –PacketPair –TOPP …at smaller probing rate …more robust to bursty traffic Best paper at PAM2003 Ongoing work: –Exploit dispersion information captured in excursions to become robust against multiple hops pathChirp vs TOPP square error PathLoad converged after 6.7 Mb.5 Mb1 Mb

Rice University, SPiN Group spin.rice.edu Connection-level Analysis and Modeling of Network Traffic understanding the cause of bursts control and improve performance detect changes of network state Effort 2

Rice University, SPiN Group spin.rice.edu Milestones Alpha-Beta May 2003 milestone [completed]: C++ version of decomposition and analysis module Towards Nov 2003 milestone: Verification of alpha-beta hypothesis in wider range of topologies, protocols, applications [Analysis module ready; collection and analysis to be done] Collaborations with Telcordia and SLAC [initiated] Collaborations with CAIDA [pursuing] December 2004 milestones [to be done]: –Integration into simulators, verification in large simulation –Applications: alpha-bottleneck aware AQM, Admission control

Rice University, SPiN Group spin.rice.edu Non-Gaussianity and Dominance Connection level separation: –remove packets of the ONE strongest connection –Leaves “Gaussian” residual traffic Traffic components: –Alpha connections: high rate (> ½ bandwidth) –Beta connections: all the rest Overall traffic Residual traffic1 Strongest connection = + Mean 99%

Rice University, SPiN Group spin.rice.edu Simple Connection Taxonomy Bursts arise from large transfers over fast links.

Rice University, SPiN Group spin.rice.edu CWND or RTT? Correlation coefficient=0.68 Short RTT correlates directly with high rate and bursts. peak-rate (Bps) 1/RTT (1/s) Correlation coefficient= peak-rate (Bps) cwnd (B) Colorado State University trace, 300,000 packets BetaAlphaBetaAlpha

Rice University, SPiN Group spin.rice.edu Impact: Performance Beta Traffic rules the small Queues Alpha Traffic causes the large Queue-sizes (despite small Window Size) Alpha connections Queue-size overlapped with Alpha Peaks Total traffic

Rice University, SPiN Group spin.rice.edu Two models for alpha traffic Impact of alpha burst in two scenarios: Flow control at end hosts – TCP advertised window Congestion control at router – TCP congestion window

Modeling Alpha Traffic ON/OFF model revisited: High variability in connection rates (RTTs) Low rate = betaHigh rate = alpha fractional Gaussian noise stable Levy noise + = + + =

Rice University, SPiN Group spin.rice.edu Alpha-Beta Model of Traffic Model assumptions: –Total traffic = Alpha component + Beta component –Alpha and Beta are independent –Beta=fractional Brownian motion Alpha traffic: two scenarios –Flow control through thin or busy end-hosts ON-OFF Burst model –Congestion control allowing large CWND Self-Similar Burst model Methods of analysis –Self-similar traffic –Queue De-multiplexing –Variable service rate

Self-similar Burst Model Alpha component = self-similar stable –(limit of a few ON-OFF sources in the limit of fast time) This models heavy-tailed bursts –(heavy tailed files) TCP control: alpha CWND arbitrarily large –(short RTT, future TCP mutants) Analysis via De-Multiplexing: –Optimal setup of two individual Queues to come closest to aggregate Queue De-Multiplexing: Equal critical time-scales Q-tail Pareto Due to Levy noise Beta (top) + Alpha

Rice University, SPiN Group spin.rice.edu ON-OFF Burst Model Alpha traffic = High rate ON-OFF source (truncated) This models bi-modal bandwidth distribution TCP: bottleneck is at the receiver (flow control through advertised window) Current state of measured traffic Analysis: de-multiplexing and variable rate queue Beta (top) + Alpha Variable Service Rate Queue-tail Weibull (unaffected) unless rate of alpha traffic larger than capacity – average beta arrival and duration of alpha ON period heavy tailed

Rice University, SPiN Group spin.rice.edu Alpha traffic: Influence of TCP All Alpha connections show –Unusually small advertised window –Drastic drop in advertised window (sometimes to zero) –…which correlates with burst arrival  Flow controlled, Weibull Q-tails

Rice University, SPiN Group spin.rice.edu Separation on Connection Level Alpha connections: dominant. Properties: –Definition: Peak rate > mean arrival rate + 1 std dev –Few, light load –Responsible for violent bursts, large queuing delays –Typically short RTT –Typically FLOW-CONTROLLED (limited at receiver) Beta connections: Residual traffic –Main load –Gaussian, LRD –Typically limited at bottleneck link Future of empowered hosts and transfer protocols: –Higher peaks, larger bursts, longer queues

Rice University, SPiN Group spin.rice.edu Future work : Network/user-driven traffic model –Correlations between network and user –Through simulation and measurements assess impact of protocols, applications, clientele, end-host server –Performance parameters from network and user specifications pathChirp –Model based estimation meeting challenges of bursty traffic –Through simulation validate realism (multihop, bursty traffic) –Anomaly detection through chirp-web Current Collaborations & Tech Transfer –IP-tunneling, coordinated measurements (Telcordia) –Integration of PathChirp into network simulators (GaTech, UIUC) –Ready for integration into SPAWAR –Demystify self-similarity ( UC Riverside)

Rice University, SPiN Group spin.rice.edu spin.rice.edu