Multiscale Traffic Processing Techniques for Network Inference and Control R. Baraniuk R. Nowak E. Knightly R. Riedi V. Ribeiro S. Sarvotham A. Keshavarz R. King NMS PI meeting Monterey November 2004 SPiN Signal Processing in Networking

Rice University spin.rice.edu Spatio-Temporal Available Bandwidth Estimation On-line localization of the tight link in a path Effort 1

Rice University spin.rice.edu Key Definitions Available bandwidth: left-over capacity on link Tight link: link with least available bandwidth Goal: locate tight link in space and over time using end-to-end probing

Rice University spin.rice.edu Applications Network monitoring - locating hot spots Network aware applications - Server selection - Route selection Science: where do Internet tight links occur and why?

Rice University spin.rice.edu Methodology Path available bandwidth Sub-path available bandwidth Methodology: For m>tight link, A[1,m] remains constant

Rice University spin.rice.edu Packet Tailgating Packet train contains: –Large packets stressing, with m hops life time –Small packets tailgating, full life time Purpose: –Large packets “measure” bandwidth via their delay –Small packets transport this timing information to the receiver

Rice University spin.rice.edu Lite-probing: pathChirp real world tool –Queuing delay  cross traffic –Averaged excursions  available resources Light weight –Probe at various rates simultaneously …converges in a handful of RTTs Departure pattern Queuing against departure Methodology Number of chirps  12 chirps Real world experiments Estimation against true x-traffic Internet experiment

Rice University spin.rice.edu Bandwidth: a Probabilistic entity Available bandwidth depends on temporary congestion level of potential tight links UIUC – Rice Probability of being tight link Estimates taken 10 mins apart REAL WORLD EXPERIMENTS UIUC (J. Hou)– Rice Available sub-path bandwidth space time avail

Rice University spin.rice.edu STAB: Spatio Temporal available Bandwidth STAB detects new tight link and reduced available bandwidth around 250 secs into simulation ns2 Simulation setting: Double web farm in ns2 (420 clients, 40 servers) Estimate Truth

Rice University spin.rice.edu GUI: ease of configuration Running on windows Instrumental for distribution and transfer

Rice University spin.rice.edu GUI in action See Demo

Connection-level Analysis and Modeling of Network Traffic understanding the cause of bursts detect changes of network state Effort 2

Rice University spin.rice.edu Bursts and Dominance Connection level separation: –Detrimental bursts caused by the ONE strongest connection –….called Alpha connections Origin of Alpha: –High rate  today from small RTT (round trip time) –Not congestion controlled Beta connections: –All the rest –Well controlled Overall traffic Residual traffic Beta 1 Strongest connection Alpha = + Mean 99%

Rice University spin.rice.edu Burst Model Alpha traffic = High rate ON-OFF source –bottleneck at the receiver (TCP advertised window)  Rate determined by RTT –Current state of measured traffic Analysis: Queues will explode with TCP’s ability to achieve large rates (HSTCP, BIC) Beta (top) + Alpha Variable Service Rate Queue-tail Weibull (as for self-similar traffic) unless rate of alpha traffic larger than available bandwidth and duration of alpha ON period heavy tailed

Rice University spin.rice.edu Free parameters TotalAlphaBeta Duration - Rate Duration - Size Size - Rate X Beta users: rate determines file size Alpha users are “free”

Scheme RD: Rate Duration independent Scheme SD: Size Duration independent Scheme SR: Size Rate independent Total Alpha Beta Real Trace SIMULATION

Rice University spin.rice.edu Network-User Driven Traffic model Mixture fit of alpha-beta Alpha: free to choose files  RATE DISTRIBUTIONS Beta: patience factor  FILE SIZE DISTRIBUTIONS RAPID PERFORMANCE ASSESSMENT TCP Control: manages only BETA traffic effectively  Congestion and admission control Original trace (Bellcore) Alpha (SR) + Beta (RD)

Rice University spin.rice.edu Effort 3 Model based Protocols

Rice University spin.rice.edu Prediction and what if scenarios High-performance protocols pushed –HSTCP –STCP –XCP –FAST-TCP –BI-TCP RTT bias –alpha-beta differentiation between flows more pronounced for STCP and HSTCP (which have large RTT bias). Need for RTT-fair high-performance TCP

Rice University spin.rice.edu TCP Africa Hybrid  two modes –Fast mode: (absence of congestion) Rapid, opportunistic increase of window (rate) –Slow mode: (presence of congestion) Linear (slow) increase in congestion avoidance –Congestion inference: Current average RTT – minimal RTT (Vegas-type) Adaptive and Fair Rapid Increase Congestion Avoidance

Rice University spin.rice.edu TCP Africa Hybrid  two modes –Fast mode: (absence of congestion) Rapid, opportunistic increase of window (rate) –Slow mode: (presence of congestion) Linear (slow) increase in congestion avoidance –Congestion inference: Current average RTT – minimal RTT (Vegas-type) Induces LOSSES infrequently (like Reno) Combines aggressiveness of HSTCP with reliability and low loss induction of Reno

Rice University spin.rice.edu RTT Fairness Against peers with different RTT HSTCP: low RTT overwhelms Africa: RTT bias is comparable to Reno

Rice University spin.rice.edu Safety Degrade performance of other flows …as compared to normal conditions ns2 sim: Reno over 100Mbps link …and with 1 Gbps HSTCP poor Africa acceptable HSTCP poor Africa acceptable

Rice University spin.rice.edu Impact and Transfer

Rice University spin.rice.edu Software STAB pathChirp Alpha-Beta decomposition User Interface on Windows (GUI) –80% completed Free, available at spin.rice.edu

Rice University spin.rice.edu Publications Enable transition to DoD contractors –Note and understand –Write own code IEEE Internet Computing Magazine –pathChirp and STAB Computer Networks –Special issue on LRD traffic –Alpha-Beta / Network-User driven traffic model IEEE Signal Processing –special issue on SPiN IEEE SP Magazine –Special issue on Complexity in Networking –Network modeling, MWM, role of multifractal scaling InfoCom 2005 –TCP Africa

Rice University spin.rice.edu Tech Transfer and Integration GUI: Pathchirp Running on Windows Raytheon: Doug Fowler discussing transitions pathChirp Sensor networks: Steve Beck (BAE Austin) Hitachi – David Diep makes pathChirp IPv4 and IPv6 compatible Computer Sciences Corporation (DoD contractor) –Steve Tsang uses MWM for DSN VoIP User Interface GridLab Project (Verstoep) – Deployed pathChirp for Grid computing measurements SPAWAR (consulted Phuong Nguyen) J9 (consulted Jasom Boyer) GaTech (Riley-Fujimoto) – On-line pathChirp inference in integrated demo to detect UDP storms UC Riverside (Faloutsos) – On-line Traffic estimation / demystify LRD UIUC (Hou) and ISI (Heidemann) – Integration of probing schemes into network – simulators JavaSim and ns-2 SLAC (Cottrell) – Large scale monitoring using pathChirp

Rice University spin.rice.edu Ongoing work pathChirp: chirp-web –Tight links on high speed networks –Anomaly detection through chirp-web : Network/user-driven traffic model –Through simulation and measurements assess impact of protocols, applications, clientele, end- host server –Parameters from network and user specifications High-speed protocols and congestion control –continue to integrate advanced modeling/probing techniques into new protocols