Hierarchical Clustering and Network Topology Identification

Slides:

Advertisements

Similar presentations

Locating Congested Segments over the Internet Based on Multiple End-to-End Path Measurements Shigehiro Ano * Atsuo Tachibana * Toru Hasegawa * Masato Tsuru.

Advertisements

Merging Logical Topologies Using End-to-end Measurements Michael Rabbat Mark Coates Robert Nowak Internet Measurement Conference 2003 Tuesday October 28,

Discussion for SAMSI Tracking session, 8 th September 2008 Simon Godsill Signal Processing and Communications Lab. University of Cambridge www-sigproc.eng.cam.ac.uk/~sjg.

Collaborators: Mark Coates, Rui Castro, Ryan King, Mike Rabbat, Yolanda Tsang, Vinay Ribeiro, Shri Sarvotham, Rolf Reidi Network Bandwidth Estimation and.

Bayesian Belief Propagation

Internet Tomography Bin Yu Statistics Department, UC Berkeley.

A Hierarchical Multiple Target Tracking Algorithm for Sensor Networks Songhwai Oh and Shankar Sastry EECS, Berkeley Nest Retreat, Jan

Budapest May 27, 2008 Unifying mixed linear models and the MASH algorithm for breakpoint detection and correction Anders Grimvall, Sackmone Sirisack, Agne.

Ragib Hasan Johns Hopkins University en Spring 2011 Lecture 11 04/25/2011 Security and Privacy in Cloud Computing.

Gibbs Sampling Qianji Zheng Oct. 5th, 2010.

Challenges posed by Structural Equation Models Thomas Richardson Department of Statistics University of Washington Joint work with Mathias Drton, UC Berkeley.

Structural Inference of Hierarchies in Networks BY Yu Shuzhi 27, Mar 2014.

Oklahoma State University Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis Xin Fan and Guoliang Fan Visual Computing and.

1 Estimating Shared Congestion Among Internet Paths Weidong Cui, Sridhar Machiraju Randy H. Katz, Ion Stoica Electrical Engineering and Computer Science.

Lo Presti 1 Network Tomography Francesco Lo Presti Dipartimento di Informatica - Università dell’Aquila.

Server-based Inference of Internet Performance V. N. Padmanabhan, L. Qiu, and H. Wang.

Network Tomography CS 552 Richard Martin. What is Network Tomography? Derive internal state of the network from: –external measurements (probes) –Some.

1 A General Introduction to Tomography & Link Delay Inference with EM Algorithm Presented by Joe, Wenjie Jiang 21/02/2004.

Network Tomography CS 552 Richard Martin. What is Network Tomography? Derive internal state of the network from: –external measurements (probes) –Some.

Maximum Likelihood Network Topology Identification Mark Coates McGill University Robert Nowak Rui Castro Rice University DYNAMICS May 5 th,2003.

Local Area Network Layer-2 Topology Mapping Local Area Network Layer-2 Topology Mapping Doron Peled Michal Rimmer Supervisor: Zigi Walter Networked Software.

Network Tomography from Multiple Senders Rob Nowak Thursday, January 15, 2004 In collaboration with Mark Coates and Michael Rabbat.

LCN 2007, Dublin 1 Non-bifurcated Routing in Wireless Multi- hop Mesh Networks by Abdullah-Al Mahmood and Ehab S. Elmallah Department of Computing Science.

LSDS-IR’08, October 30, Peer-to-Peer Similarity Search over Widely Distributed Document Collections Christos Doulkeridis 1, Kjetil Nørvåg 2, Michalis.

5/25/2005EE562 EE562 ARTIFICIAL INTELLIGENCE FOR ENGINEERS Lecture 16, 6/1/2005 University of Washington, Department of Electrical Engineering Spring 2005.

Network Tomography through End- End Multicast Measurements D. Towsley U. Massachusetts collaborators: R. Caceres, N. Duffield, F. Lo Presti (AT&T) T. Bu,

Network Tomography (A presentation for STAT 593E) Mingyan Li Radha Sampigethaya.

Using ranking and DCE data to value health states on the QALY scale using conventional and Bayesian methods Theresa Cain.

Computer vision: models, learning and inference Chapter 10 Graphical Models.

Network Tomography CS 552 Richard Martin. What is Network Tomography? Derive internal state of the network from: –external measurements (probes) –Some.

University of Kansas A KTEC Center of Excellence 1 Soshant Bali *, Yasong Jin **, Victor S. Frost * and Tyrone Duncan ** Information and Telecommunication.

Traffic Matrix Estimation for Traffic Engineering Mehmet Umut Demircin.

Multiantenna-Assisted Spectrum Sensing for Cognitive Radio

Particle Filtering in Network Tomography

1 Physical Fluctuomatics 5th and 6th Probabilistic information processing by Gaussian graphical model Kazuyuki Tanaka Graduate School of Information Sciences,

2005/10/211 A Survey on Physical Network Topology Estimation October 21, 2005 Chikayama-Taura Lab. Tatsuya Shirai.

Multiple Source, Multiple Destination Network Tomography Michael Rabbat IEEE Infocom, Hong Kong Wednesday, March 10, 2004 Co-Authors: Mark Coates and Robert.

Hierarchical Bayesian Modeling (HBM) in EEG and MEG source analysis Carsten Wolters Institut für Biomagnetismus und Biosignalanalyse, Westfälische Wilhelms-Universität.

2 Syntax of Bayesian networks Semantics of Bayesian networks Efficient representation of conditional distributions Exact inference by enumeration Exact.

Hung X. Nguyen and Matthew Roughan The University of Adelaide, Australia SAIL: Statistically Accurate Internet Loss Measurements.

Markov Random Fields Probabilistic Models for Images

Physical Layer Challenges in Telecommunication Systems: Optimum Multiuser Detection in CDMA System Fatih Alagöz Department of Computer Engineering Bogazici.

StriD 2 FA: Scalable Regular Expression Matching for Deep Packet Inspection Author: Xiaofei Wang, Junchen Jiang, Yi Tang, Bin Liu, and Xiaojun Wang Publisher:

Multiplicative Wavelet Traffic Model and pathChirp: Efficient Available Bandwidth Estimation Vinay Ribeiro.

Trajectory Sampling for Direct Traffic Oberservation N.G. Duffield and Matthias Grossglauser IEEE/ACM Transactions on Networking, Vol. 9, No. 3 June 2001.

Optimal Sampling Strategies for Multiscale Stochastic Processes Vinay Ribeiro Rolf Riedi, Rich Baraniuk (Rice University)

Tracking Multiple Cells By Correspondence Resolution In A Sequential Bayesian Framework Nilanjan Ray Gang Dong Scott T. Acton C.L. Brown Department of.

The famous “sprinkler” example (J. Pearl, Probabilistic Reasoning in Intelligent Systems, 1988)

SR: A Cross-Layer Routing in Wireless Ad Hoc Sensor Networks Zhen Jiang Department of Computer Science West Chester University West Chester, PA 19335,

Unsupervised Mining of Statistical Temporal Structures in Video Liu ze yuan May 15,2011.

Bayesian Travel Time Reliability

Multiple Target Tracking Using Spatio-Temporal Monte Carlo Markov Chain Data Association Qian Yu, Gerard Medioni, and Isaac Cohen Edwin Lei.

Overview Objective Track an unknown number of targets using a wireless sensor network of binary sensors for real-time surveillance Issues Real-time operation.

APPLICATIONS OF DIRICHLET PROCESS MIXTURES TO SPEAKER ADAPTATION Amir Harati and Joseph PiconeMarc Sobel Institute for Signal and Information Processing,

Clustering (1) Clustering Similarity measure Hierarchical clustering

Lecture 13 – Network Mapping

Zhichen Xu, Mallik Mahalingam, Magnus Karlsson

Turning Heterogeneity into an Advantage in Overlay Routing

ISP and Egress Path Selection for Multihomed Networks

On the Physical Carrier Sense in Wireless Ad-hoc Networks

Dude, where’s that IP? Circumventing measurement-based geolocation

Recovering Temporally Rewiring Networks: A Model-based Approach

Hierarchical clustering approaches for high-throughput data

Network Science, National Academies Press, 2006

Network Inference Chris Holmes Oxford Centre for Gene Function, &,

Tony Sun, Guang Yang, Ling-Jyh Chen, M. Y. Sanadidi, Mario Gerla

Expectation-Maximization & Belief Propagation

CS639: Data Management for Data Science

Yalchin Efendiev Texas A&M University

Presentation transcript:

Hierarchical Clustering and Network Topology Identification Rui Castro Mark Coates Rob Nowak Department of Electrical and Computer Engineering Copyright © 2004 - Rui Castro

Topology Identification Ratnasamy & McCanne (1999) Duffield, et al (2000,01,02) Bestravos, et al (2001) Coates, et al (2001) Shih & Hero (2002) Pairwise delay measurements reveal topology Copyright © 2004 - Rui Castro

Topology Identification Challenges: 12 % never respond,15 % multiple interfaces - Barford et al (2000) detect level-2 topology “invisible” to IP layer (e.g., switches) Copyright © 2004 - Rui Castro

Relationship between Topology ID and Hierarchical Clustering Copyright © 2004 - Rui Castro

Do not need clock synchronization!! Sandwich Probing Do not need clock synchronization!! Copyright © 2004 - Rui Castro

Sandwich Probing we can infer that receivers 3 & 4 have a longer shared path than 3 & 5 Topology imposes constraints more shared queues  larger Copyright © 2004 - Rui Castro

Delay Covariance more shared queues  larger covariance Copyright © 2004 - Rui Castro

Measurement Framework Key Assumptions: stationarity fixed (but unknown) routes temporal independence spatial independence Multiple measurements individual measurement CLT Copyright © 2004 - Rui Castro

Maximum Likelihood Tree - MLT The maximum likelihood tree (MLT) is defined as where product of Gaussian densities Two Approaches: Binary tree construction based on bottom-up, recursive selection and pair-merging process Markov Chain Monte Carlo (MCMC) tree search measurements unknown similarity metric values, measurement likelihood forest of possible trees, monotonicity constrain set, for tree Copyright © 2004 - Rui Castro

Internet Experiments – Sandwich Probing Traceroute topology MCMC topology UNO ALT topology Copyright © 2004 - Rui Castro

Internet Experiments – RTT Delay Covariance Traceroute topology Estimated topology Thanks to Yolanda Tsang & Mehmet Yildiz Copyright © 2004 - Rui Castro

Final Remarks and Comments Clever probing and sampling schemes reveal “hidden” network structure and behavior Likelihood based methods are a natural choice to account for uncertainty in the data Sampling methods relying solely on RTT can be devised R. Castro, M. Coates and R. Nowak, "Likelihood Based Hierarchical Clustering", IEEE Transactions in Signal Processing, August 2004. R. Castro, M. Coates, G. Liang, R. Nowak and B. Yu, "Network Tomography: Recent Developments", Statistical Science, 2004 (invited paper, to appear). Complex interplay between measurement/probing techniques, statistical modeling, and computational methods for optimization Copyright © 2004 - Rui Castro