Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.

Slides:



Advertisements
Similar presentations
Ch. 12 Routing in Switched Networks
Advertisements

Congestion Control and Fairness Models Nick Feamster CS 4251 Computer Networking II Spring 2008.
Congestion Control and Fairness Models Nick Feamster CS 4251 Computer Networking II Spring 2008.
Merging Logical Topologies Using End-to-end Measurements Michael Rabbat Mark Coates Robert Nowak Internet Measurement Conference 2003 Tuesday October 28,
pathChirp Efficient Available Bandwidth Estimation
1 School of Computing Science Simon Fraser University CMPT 771/471: Internet Architecture & Protocols TCP-Friendly Transport Protocols.
Ch. 12 Routing in Switched Networks Routing in Packet Switched Networks Routing Algorithm Requirements –Correctness –Simplicity –Robustness--the.
Computer Networking Lecture 20 – Queue Management and QoS.
Florin Dinu T. S. Eugene Ng Rice University Inferring a Network Congestion Map with Traffic Overhead 0 zero.
Presentation of Designing Efficient Irregular Networks for Heterogeneous Systems-on-Chip by Christian Neeb and Norbert Wehn and Workload Driven Synthesis.
COS 461 Fall 1997 Routing COS 461 Fall 1997 Typical Structure.
Traffic Shaping Why traffic shaping? Isochronous shaping
XCP: Congestion Control for High Bandwidth-Delay Product Network Dina Katabi, Mark Handley and Charlie Rohrs Presented by Ao-Jan Su.
1 Estimating Shared Congestion Among Internet Paths Weidong Cui, Sridhar Machiraju Randy H. Katz, Ion Stoica Electrical Engineering and Computer Science.
Lo Presti 1 Network Tomography Francesco Lo Presti Dipartimento di Informatica - Università dell’Aquila.
15-441: Computer Networking Lecture 26: Networking Future.
1 Estimating Shared Congestion Among Internet Paths Weidong Cui, Sridhar Machiraju Randy H. Katz, Ion Stoica Electrical Engineering and Computer Science.
 Don Towsley 2000 Network Tomography for the Internet: Open Problems D. Towsley U. Massachusetts.
Internet Traffic Patterns Learning outcomes –Be aware of how information is transmitted on the Internet –Understand the concept of Internet traffic –Identify.
1 A General Introduction to Tomography & Link Delay Inference with EM Algorithm Presented by Joe, Wenjie Jiang 21/02/2004.
Network Tomography CS 552 Richard Martin. What is Network Tomography? Derive internal state of the network from: –external measurements (probes) –Some.
Maximum Likelihood Network Topology Identification Mark Coates McGill University Robert Nowak Rui Castro Rice University DYNAMICS May 5 th,2003.
Diffusion Mechanisms for Active Queue Management Department of Electrical and Computer Engineering University of Delaware May 19th / 2004 Rafael Nunez.
Local Area Network Layer-2 Topology Mapping Local Area Network Layer-2 Topology Mapping Doron Peled Michal Rimmer Supervisor: Zigi Walter Networked Software.
Low Delay Marking for TCP in Wireless Ad Hoc Networks Choong-Soo Lee, Mingzhe Li Emmanuel Agu, Mark Claypool, Robert Kinicki Worcester Polytechnic Institute.
Diffusion Mechanisms for Active Queue Management Department of Electrical and Computer Engineering University of Delaware May 19th / 2004 Rafael Nunez.
Basics of Statistical Estimation. Learning Probabilities: Classical Approach Simplest case: Flipping a thumbtack tails heads True probability  is unknown.
Network Tomography from Multiple Senders Rob Nowak Thursday, January 15, 2004 In collaboration with Mark Coates and Michael Rabbat.
Traffic Matrix Estimation: Existing Techniques and New Directions A. Medina (Sprint Labs, Boston University), N. Taft (Sprint Labs), K. Salamatian (University.
1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003.
Network Tomography through End- End Multicast Measurements D. Towsley U. Massachusetts collaborators: R. Caceres, N. Duffield, F. Lo Presti (AT&T) T. Bu,
Computer Networking Lecture 17 – Queue Management As usual: Thanks to Srini Seshan and Dave Anderson.
FTDCS 2003 Network Tomography based Unresponsive Flow Detection and Control Authors Ahsan Habib, Bharat Bhragava Presenter Mohamed.
Network Tomography (A presentation for STAT 593E) Mingyan Li Radha Sampigethaya.
Rafael C. Nunez - Gonzalo R. Arce Department of Electrical and Computer Engineering University of Delaware May 19 th, 2005 Diffusion Marking Mechanisms.
Diffusion Mechanisms for Active Queue Management Department of Electrical and Computer Engineering University of Delaware Aug 19th / 2004 Rafael Nunez.
Diffusion Mechanisms for Active Queue Management Department of Electrical and Computer Engineering University of Delaware May 19th / 2004 Rafael Nunez.
1 Network Tomography Don Towsley UMass-Amherst. 2 Network Tomography - I Goal: obtain detailed picture of a network/internet from end-to-end views  infer.
Diffusion Early Marking Department of Electrical and Computer Engineering University of Delaware May / 2004 Rafael Nunez Gonzalo Arce.
Traffic Matrix Estimation for Traffic Engineering Mehmet Umut Demircin.
Not All Microseconds are Equal: Fine-Grained Per-Flow Measurements with Reference Latency Interpolation Myungjin Lee †, Nick Duffield‡, Ramana Rao Kompella†
A Machine Learning-based Approach for Estimating Available Bandwidth Ling-Jyh Chen 1, Cheng-Fu Chou 2 and Bo-Chun Wang 2 1 Academia Sinica 2 National Taiwan.
Particle Filtering in Network Tomography
Multiple Source, Multiple Destination Network Tomography Michael Rabbat IEEE Infocom, Hong Kong Wednesday, March 10, 2004 Co-Authors: Mark Coates and Robert.
Multiscale Traffic Processing Techniques for Network Inference and Control Richard Baraniuk Edward Knightly Robert Nowak Rolf Riedi Rice University INCITE.
Internet Performance Measurements and Measurement Techniques Jim Kurose Department of Computer Science University of Massachusetts/Amherst
Queueing and Active Queue Management Aditya Akella 02/26/2007.
Multiplicative Wavelet Traffic Model and pathChirp: Efficient Available Bandwidth Estimation Vinay Ribeiro.
Optimal Sampling Strategies for Multiscale Stochastic Processes Vinay Ribeiro Rolf Riedi, Rich Baraniuk (Rice University)
PathChirp Spatio-Temporal Available Bandwidth Estimation Vinay Ribeiro Rolf Riedi, Richard Baraniuk Rice University.
Spatio-Temporal Available Bandwidth Estimation Vinay Ribeiro Rolf Riedi, Richard Baraniuk Rice University.
Introduction Jiří Navrátil SLAC. Rice University Richard Baraniuk, Edward Knightly, Robert Nowak, Rudolf Riedi Xin Wang, Yolanda Tsang, Shriram Sarvotham,
An Efficient Gigabit Ethernet Switch Model for Large-Scale Simulation Dong (Kevin) Jin.
An Efficient Gigabit Ethernet Switch Model for Large-Scale Simulation Dong (Kevin) Jin.
Péter Hága Eötvös Loránd University, Hungary European Conference on Complex Systems 2008 Jerusalem, Israel.
Multiscale Tools for Networking Vinay J. Ribeiro Ph.D. Thesis Proposal.
Internet Measurement and Analysis Vinay Ribeiro Shriram Sarvotham Rolf Riedi Richard Baraniuk Rice University.
PathChirp & STAB Measuring Available Bandwidth and Locating Bottlenecks in Packet Networks Vinay Ribeiro Rolf Riedi, Richard Baraniuk Rice University spin.rice.edu.
Indian Institute of Technology Bombay 1 Communication Networks Prof. D. Manjunath
PathChirp Efficient Available Bandwidth Estimation Vinay Ribeiro Rice University Rolf Riedi Rich Baraniuk.
Lo Presti 1 Ne X tworking’03 June 23-25,2003, Chania, Crete, Greece The First COST-IST(EU)-NSF(USA) Workshop on EXCHANGES & TRENDS IN N ETWORKING Network.
Internet Traffic Engineering Motivation: –The Fish problem, congested links. –Two properties of IP routing Destination based Local optimization TE: optimizing.
The Impact of False Sharing on Shared Congestion Management Srinivasa Aditya Akella Joint work with Srini Seshan and Hari Balakrishnan 28 Feb, 2001.
Yiting Xia, T. S. Eugene Ng Rice University
Monitoring Persistently Congested Internet Links
Hierarchical Clustering and Network Topology Identification
Congestion Control, Quality of Service, & Internetworking
pathChirp Efficient Available Bandwidth Estimation
pathChirp Efficient Available Bandwidth Estimation
Queueing Problem The performance of network systems rely on different delays. Propagation/processing/transmission/queueing delays Which delay is affected.
Presentation transcript:

collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and Tomography Rob Nowak & Rich Baraniuk UW-Madison Rice University spin.rice.edu

too complex to measure everywhere, all the time traffic measurements expensive (hardware, bandwidth) Internet Boom

companies do not share data or performance information Proprietary Concerns

bits are bundled into packets packets go through routers queues absorb bursts of packets congestion: queues fill up, large delays, packet drops Networking 101

Network Measurement & Inference Internet equivalent model Path Modeling and Bandwidth Estimation Network Tomography

Brain Tomography unknown object statistical model measurements Maximum likelihood estimate maximize likelihood physics data prior knowledge MRF model counting & projection Poisson

unknown object statistical model measurements Maximum likelihood estimate maximize likelihood physics data prior knowledge Network Tomography queuing behavior routing & counting binomial / multinomial

Network Tomography From link-level traffic measurements, infer end-to-end traffic flow rates Vardi 96, Tebaldi & West 98 Cao, Davis, Vander Wiel, Yu 00

y = packet losses or delays measured at the edge A = routing matrix (graph) = packet loss probabilities or queuing delays for each link = randomness inherent traffic measurements likelihood function Network Tomography (MINC Project, Towsley-Duffield)

Probe packets experience similar queuing effects and may interact with each other Probing the Network probe = packet stripe cross-traffic delay

Network Tomography: The Basic Idea sender receivers

Network Tomography: The Basic Idea sender receivers

Maximum Likelihood Estimation via EM Suppose we were able to measure losses/delays on each link Expectation-Maximization (EM) alternates between computing expectation of unobserved internal measurements and the desired estimates of link-by-link loss/delay distributions Problem: How to compute maximum likelihood estimates of link-by-link loss/delay distributions from end-to-end measurements ?

Topology ID via Probe Interactions

we can infer that receivers 3 & 4 have a longer shared path than 3 & 5

Finding the Maximum Likelihood Tree Stochastic search through forest via Metropolis-Hastings

True topology estimated topology Internet measurement experiments UNO

What have we learned? Clever probing and sampling schemes reveal hidden network structure and behavior Simple inference algorithms are effective, intuitive, easy to implement, scale nicely MLE criteria are easily modified to include prior information: Bayesian or regularized MLE methods are straightforward Complex interplay between measurement/probing techniques, statistical modeling, and computational methods for optimization

Open Problems: Placement/Coverage How should measurement devices be deployed ? Logical graph coverage of physical topology ? Can random graph models shed some light ?

Open Problems: Spatio-temporal Correlation competing traffic Can we detect correlations? Can we exploit them in measurement and mapping applications? Fuse tomography and bandwidth estimation Long-range dependence of network traffic Correlations due to competing traffic flows

Open Problems: Detection and Localization Detecting and locating anomalous behavior rather than estimating everything EstimationHypothesis Testing How can we capitalize on conventional wisdom: most links are good and only a few are bad ?

Open Problems: Timing and Synchronization Hardware solutions (expensive) Software solutions (more practical) - sophisticated software clocks (Veitch 02) - crude software clocks (ICMP timestamping) and statistical averaging sender network sender monitor receiver monitor receiver How to accurately measure time ?

Open Problems: Network Security How can measurement and monitoring across the Internet help detect and prevent malicious activities ?