1. Hierarchical Clustering and Network Topology Identification
Rui Castro
Mark Coates Rob Nowak
Department of Electrical and Computer Engineering
Copyright © Rui Castro

2. Topology Identification
Ratnasamy & McCanne (1999)
Duffield, et al (2000,01,02)
Bestravos, et al (2001)
Coates, et al (2001)
Shih & Hero (2002)
Pairwise delay measurements reveal topology
Copyright © Rui Castro

3. Topology Identification
Challenges:
12 % never respond,15 % multiple interfaces - Barford et al (2000)
detect level-2 topology “invisible” to IP layer (e.g., switches)
Copyright © Rui Castro

4. Relationship between Topology ID and Hierarchical Clustering
Copyright © Rui Castro

5. Do not need clock synchronization!!
Sandwich Probing
Do not need clock synchronization!!
Copyright © Rui Castro

6. Sandwich Probing
we can infer that receivers 3 & 4 have a longer shared path than 3 & 5
Topology imposes
constraints
more shared queues  larger
Copyright © Rui Castro

7. Delay Covariance more shared queues  larger covariance
Copyright © Rui Castro

8. Measurement Framework
Key Assumptions:
stationarity
fixed (but unknown) routes
temporal independence
spatial independence
Multiple measurements
individual
measurement
CLT
Copyright © Rui Castro

9. Maximum Likelihood Tree - MLT
The maximum likelihood tree (MLT) is defined as
where
product of Gaussian densities
Two Approaches:
Binary tree construction based on bottom-up, recursive selection and pair-merging process
Markov Chain Monte Carlo (MCMC) tree search
measurements
unknown similarity metric values,
measurement likelihood
forest of possible trees,
monotonicity constrain set, for tree
Copyright © Rui Castro

10. Internet Experiments – Sandwich Probing
Traceroute topology
MCMC topology
UNO
ALT topology
Copyright © Rui Castro

11. Internet Experiments – RTT Delay Covariance
Traceroute topology
Estimated topology
Thanks to
Yolanda Tsang & Mehmet Yildiz
Copyright © Rui Castro

12. Final Remarks and Comments
Clever probing and sampling schemes reveal “hidden” network structure and behavior
Likelihood based methods are a natural choice to account for uncertainty in the data
Sampling methods relying solely on RTT can be devised
R. Castro, M. Coates and R. Nowak, "Likelihood Based Hierarchical Clustering", IEEE Transactions in Signal Processing, August R. Castro, M. Coates, G. Liang, R. Nowak and B. Yu, "Network Tomography: Recent Developments", Statistical Science, 2004 (invited paper, to appear).
Complex interplay between measurement/probing techniques, statistical modeling, and computational methods for optimization
Copyright © Rui Castro