1. Network Tomography from Multiple Senders Rob Nowak Thursday, January 15, 2004 In collaboration with Mark Coates and Michael Rabbat

2. Brain Tomography unknown object statistical model measurements Maximum likelihood estimate maximize likelihood physics data prior knowledge counting & projection Poisson

3. unknown object statistical model measurements Maximum likelihood estimate maximize likelihood physics data prior knowledge Network Tomography queuing behavior routing & counting binomial / multinomial Why ? network optimizing, alias resolution, peeking on peering

4. y = packet losses or delays measured at the edge A = routing matrix (graph)  = packet loss probabilities or queuing delays for each link  = randomness inherent traffic measurements likelihood function Network Tomography (Y. Vardi, D. Towsley, N. Duffield)

5. Probe packets experience similar queuing effects and may interact with each other Probing the Network probe = packet stripe cross-traffic delay

6. Network Tomography: The Basic Idea sender receivers

7. Network Tomography: The Basic Idea sender receivers

8. Logical Topology Measure end-to-end (from sender to receiver) losses/delays Infer logical topology & link-level loss/delay characteristics receivers sender receivers

9. Single Sender Active Probing Tree-Structured Logical Topology A 123

10. Two Receiver Sub-problems Components 11 22 44 55 33 77 66 44 55 1 © 21 © 2 Pairs of receivers Spatial Independence Eg.  = loss, delay A 12

11. Decompose In To Components 11 22 44 55 33 77 66    ©   11    ©   Eg.  = loss, delay

12. Decompose In To Components 11 22 44 55 33 77 66    ©      ©   11 Eg.  = loss, delay And so on…

13. Back-to-Back Packet Probes A 12 Similar experience Independent experiences (Keshav, ’91) (Carter & Crovella, ’96) Repeat and average 44 55 1 © 21 © 2 Independence of behavior on unshared links allows us to separate performance effects (e.g., loss, delay) on shared and unshared portions of paths Duffield et al., ’99, Coates & Nowak, ’00, Byers et al., ’00

14. Link-Level Parameter Estimation Ex. Delay variance make repeated packet pair delay measurements

15. Topology Identification “Correlation” in packet-pairs measurements reveals topology Stronger correlation more shared links Group pairs of most correlated nodes first, building tree from bottom (receivers) to top (sender) A 123 Ratnasamy & McCanne, ’99, Duffield et al., ’02, Coates et al., ‘02

16. Topology Identification A 123 2.01.5 0.5

17. Reconstruct The Larger Network 1.5 0.5 1.02.5 1.0 2.0 1.02.51.5 1.03.01.0 1.5 Link-level characteristics (loss, delay) estimation Network topology identification Tightly coupled problems

18. Measure From Multiple Senders A 123… …B

19. Multiple Sender Tomography More topological information Mutual information, Improved estimates (Bu et al., 2002) (Rabbat et al., 2002)

20. Multiple Sender Decomposition 1-by-2 Component ? ii jj kk

21. Branching & Joining Points 1-by-2 Component 2-by-1 Component and ii jj kk aa bb cc

22. Example Decomposition 1.0 0.25 2.0 0.75 1.0 1.25 2.25 1.5 1.25 2.02.5 2.25 3.02.75 1.03.0 2.25 3.5 1.5

23. Canonical Subproblem: Two Senders & Two Receivers two sender, two receiver problem characterizes network tomography problem in general

24. Two Sender, One Receiver Probing ?? ? A 1 B Similar experiences? Independent experiences … not analogous to single sender probing Identifying joining points from probe data is very difficult

25. Shared and Non-Shared Topologies 5 Links 2 Internal Nodes 8 Links 4 Internal Nodes 11 22 33 44 55 11 44 66 22 77 88 55 33 11 44 66 22 77 88 55 33 11 44 66 22 77 88 55 33 Natural dichotomy according to “model order” Shared topologyNon-Shared topology most relevant for purposes of performance characterization easily discernable from end-to-end probes

26. Mutual Information SharedNon-Shared

27. Mutual Information Same branching point  Shared component links Different branching points  No shared component links Average Estimates! SharedNon-Shared

28. Arrival Order and Model Order Selection 1 1 Intuition: Arrival order fixed at joining point Assume: Unique routes between end-hosts Routes are stationary (5-10min) (Zhang, Paxson, Shenker, ’00) No reordering (Bellardo & Savage, ’02) Packets from each sender to receiver 1

29. Shared vs. Non-Shared 1 2 1 2 u 12 1 2 1 2 1 2 1 2 1 2 1 2 1 2 Packet pair probes from both senders with randomized offset u 1 1 2 2 1 21 2 1 2 1 2 1 2 1 2 1 2 1 2 u   

30. Shared vs. Non-Shared Arrival order always same 1 1 2 2 1 u 1 1 2 2 u Order depends on delays, offset

31. Detection of Shared Topology utut Shared: vs. Non-Shared: Repeated probing: Test:   Random offset:

32. 1.1 B.2 1.1 1.2 A.1 1.2 A.2 B.1 u Transmit many probes to receiver 1 Probability of different arrival order because of cross-traffic, Repeat to other receiver, Original measurements give Detection in Presence of Cross-Traffic Shared: vs. Non-Shared: Delays are variable: cross-traffic processing delays

33. Arrival Order Based Topology ID Rice LAN

34. Joint Performance & Topology Estimation 1 2  u  Performance Assessment Link-level parameters  1,  2, … Packet-pair measurements 1 2 1 2 1 2 Topology Characterization Different arrival order probabilities ,  1,  2 Arrival order measurements

35. Decision-Theoretic Framework HS:HS: HN:HN: Two branching, joining points  unrestricted   N 2  unrestricted   N 2 [0,1] 3 Unique joining point  2  5  3  6   S 2  1 =  2 =    S 2 [0,1] 1 11 22 33 44 55 66 11 22 33 44

36. Characterize Topology & Performance Generalized Likelihood Ratio Test:

37. Wilks Saves The Day Generalized Likelihood Ratio Test: Wilks’ Theorem (’38): Under H S : (N ! 1)

38. Asymptotic Results 100 probes1000 probes

39. ROC Curve 1000 probes Loss Only Arrival Order Only Arrival Order and Loss

40. Number of Probes Used 1000 500 200 100

41. Concluding Remarks What will make network tomography a useful tool ? www.ece.wisc.edu/~nowak