1. Network Tomography (A presentation for STAT 593E) Mingyan Li Radha Sampigethaya

2. Outline of Talk Introduction to Networks Network Monitoring Background Introduction to Network Tomography Problem addressed Clustering solutions Conclusions

3. Introduction to Networks Network is a set of interconnected hosts (PC, server, router) Internet is a network of networks Each host is modeled as functional layers for the purpose of networking Each host is identified by an IP address (Ex: 128.95.196.98) TCP is the reliable communication protocol for networking; UDP is non-reliable Internet traffic unit is assumed to be packets NOTE: TCP – Transmission Control Protocol; IP – Internet Protocol UDP – User Datagram Protocol Application (Email, HTTP) Transport (TCP/UDP) Network (IP) Network Interface (NIC, modem) Simplified host layer model

4. Introduction to Networks (2) TCP is most common on the Internet and provides reliable end-to-end (application-to-application) communication Automatic Repeat Request (ARQ): - Packets received are acknowledged by receiver - Packets are retransmitted by sender if: 1. Packets are received but with errors, at the receiver 2. ACK’s are not received by the sender indicating loss or delay of packets - TCP retransmits after waiting for the ACK for a deterministic time (round-trip-time) Flow Control mechanisms Congestion Control mechanisms (due to finite network bandwidth) Unicast communication? One user sends data to another user Multicast communication? One user sends data to many users

5. Introduction to Networks (3) IP (Internet Protocol) layer Each host identified by IP address (XXX.XXX.XXX.XXX) IP layer routes packets from source towards destination (using routing algorithm) Maintains Routing table: list of next-hop nodes (static or dynamic list) - Destination IP address (Net ID portion) determines the next-hop node chosen from table 0 1 1 1 0 Net id Multicast address Host id 0 31 Class A Class D For Internet Multicast: 224.0.0.0 through 239.255.255.255 1 8

6. Introduction to Networks (4) Internet Router or Internet Gateway Interconnects 2 networks and passes packets from one to other Router has an IP address, routing table, and handles traffic coming in and going out of network For multicast communication, router needs to be specifically enabled network1 network2

7. Introduction to Networks (5) Multicast Communication - One sender and multiple receivers Internet Applications: Video conferencing, internet gaming IP Multicast group - Each group has an IP Address - Hosts need to notify local routers about the multicast group they belong to, and routers will update tables - A host may belong to more than one multicast group! - Routers will forward multicast group packets to appropriate next-hop nodes (refers to routing table) - Any host can send packets to the multicast group by sending to the group IP address - Only members of multicast group receive the packets Multicast communication is normally best-effort; uses UDP

8. Introduction to Networks (6) Internet ISP A C D B Unicast Example (Sender-Receiver) A is sender B, C, D are receivers of the same message A, B,C,D belong to same multicast group Link Route to: LINK1128.95.X.X LINK2 Default Routing table for Router 1 128.95.X.X LINK1 LINK2

9. Introduction to Networks (7) Multicast Example (One Sender-Multiple Receiver) Internet ISP A C D E B Multicast group: A is sender B, C, D, E are receivers of same message Link Route to: LINK1, LINK2multicast group IP LINK1128.95.X.X LINK2 Default Routing table for Router 1

10. Introduction to Networks (8) Multicast Routing Tree Constructed by multicast routing algorithms Rooted at the source, with the receivers of the multicast group at the leaves Intermediate tree nodes are routers which forward packets Links constitute edges of the tree A physical topology (tree) would consist of all the nodes and links encountered in the multicast communication A logical topology (tree) would consist of subsets of links and nodes (but all the receivers and the sender) Link(s) 1 Router1 E DCB Source A A Logical Multicast Topology ISP Router Router 2 B,C,D,E are receivers

11. Where are we? Introduction to Networks Network Monitoring Background Introduction to Network Tomography Problem addressed Clustering solutions Conclusions

12. Network Monitoring Background Networks: set of nodes, links - delays, losses affect performance Networks normally are interconnected (not isolated) - hence interdependent for performance Network monitoring and management - involves collection of network performance statistics (link delay, link loss, traffic rate) - easier for isolated network compared to inter-network (such as Internet) - Challenges for Inter-network monitoring and management: Increased overhead cost, complexity, confidentiality of company network statistics 1 2 3 4 5 6 7 8 Link or group of links Node or network A Logical Network

13. Network Monitoring Background contd…. Inter-network monitoring requires: Timely, accurate, localized measurements of performance metrics without any special cooperation amongst the different networks Measurement Methods: 1. Active (sending probe packets) Adds to normal data traffic 2. Passive (traffic analysis) Temporal and spatial dependence might bias measurement Performance metrics: - Link-level: loss rate, delay; Path-level: traffic matrix - May not be directly measurable - Metrics are then inferred from other easily monitored network statistics (count of packets, time delay between received packets)

14. Where are we? Introduction to Networks Network Monitoring Background Introduction to Network Tomography Problem addressed Clustering solutions Conclusions

15. Network Tomography Inferential network monitoring and does not require any special cooperation between networks Two forms of network tomography: - link-level metric estimation based on end-to-end, traffic measurements (counts of sent/received packets, time delays between sent/received packets) - path-level (sender-receiver path) traffic intensity estimation based on link-level traffic measurements (counts of packets through nodes) Using end-to-end measurements we can infer a network’s link delay/loss statistics, and also the network connectivity or topology

16. Where are we? Introduction to Networks Network Monitoring Background Introduction to Network Tomography Problem addressed Clustering solutions Conclusions

17. Problem Addressed Logical Multicast Topology Inference using end-to- end point measurements We consider the problem of determining the connectivity structure or topology of a network and relate this to the problem of hierarchical clustering. Active measurement method (multicast probes) is used Performance statistic measured is loss rate

18. Problem Addressed (2) Multicast Topology Inference Given Sender: probes Receiver: traces (loss, delay) Goal: Identify multicast tree topology (Interconnectivity from sender to receivers) Source Receivers =or Routers

19. Problem Addressed (3) Motivation Topology: first step to infer other characteristics: link delay, link loss, and this knowledge aids multicast application design Reliable multicast has been shown to benefit from the knowledge of underlying multicast topology, and it helps scalable local recovery. MTRACE: is a tool that requires cooperation from routers.

20. Problem Addressed (4) General Approach to Multicast Topology Inference Given end-to-end measurements: loss, delay Exploit correlation in measurement to group nodes High correlation, high probability of sharing a parent node Find correlation function Increases along the path from root to leaves Can be estimated from measurements at leaves Example Prob. of probe loss Delay Build topology by recursively grouping nodes to maximize correlation function

21. Clustering Solutions Loss Based Topology Inference Why loss based? Delay based required synchronization Topology inference procedure (1) Multicast probes, record receiver loss (2) Form groups of one receiver initially (3) Merge the 2 groups that have the highest correlation in loss until there is one hierarchy group 101101 110110 110110 100100

22. Clustering Solutions (2) Multicast Topology Inference Example Real TopologyInferred Topology ab cdef g 2% 3% 2% 1% 5% 3% 1%

23. Clustering Solutions (3) Improvement in approach Approach shown builds only binary tree In practice, routers may have more than 2 children Consider: Real: Inferred:

24. Clustering Solutions (4) Loss Rate Inference A sender, 2 receivers: a & b Probe lost at link 1 won’t reach a &b Probe lost at link 2 (3) won’t reach a (b) Define P ab : Prob{ both a & b lose the probe} P a : Prob{ a loses the probe but not b} P b : Prob{ b loses the probe but not a} P 1 : Prob {probe loss seen by parent node of a&b} shared loss rate of a & b P 2 : loss rate of link 2 P 3 : loss rate of link 3 From P ab, P a, P b, compute P 1, P 2, P 3 p2p2 p1p1 p3p3 a b papa pbpb p ab Shared loss rate Link 1 Link 2 Link 3

25. Clustering Solutions (5) Loss Rate Inference (Cont.) P ab = P 1 + (1-P 1 ) P 2 P 3 P a = (1- P 1 ) P 2 (1- P 3 ) P b = (1- P 1 ) (1- P 2 ) P 3 Solve P 2 = P b / (1- P ab - P a ) P 3 = P a / (1- P ab - P b ) P 1 = 1-P a / (P 2 (1-P 3 )) P2P2 p1p1 p3p3 a b papa pbpb p ab Link 1

26. Clustering Solutions (6) Multicast Topology Inference Example Real TopologyInferred Topology ab cdef g 2% 3% 2% 1% 5% 3% 1% ab cdef g 3.1% 1.9% 2.9% 1.9% 2.0% 1.8%5.0% 2.9% 3.7% 1.0% 0.1%

27. Clustering Solutions (7) Multicast Topology Inference Approaches Two approaches Binary Loss Tree Pruning Algorithm (BLTP) Infer binary tree Estimate link loss rate Prune the link with loss rate < α (threshold) General Loss Tree Algorithm (GLT) Use P 1 (shared loss rate) as correlation function, group nodes to maximize P 1, and merge nodes based on inferred link loss rate

28. Clustering Solutions (8) Comparison of BLTP & GLT Both proved to converge to real topology with prob. 1 for sufficient large # of probes Performance (in terms of accuracy vs. # of probes) are similar Threshold (α) applied only at the end in BLTP : facilitate adaptive selection of (α)

29. Clustering Solutions (9) Limitations of the approaches Only Infer logical (not physical) topology Only identify links with significant loss rates ( 1%) Only identify routers with more than one child

30. Clustering Solutions (10) Other Approaches Maximum likelihood clustering Slow convergence, poor performance Bayesian clustering Optimal provided topology drawn from prior distribution Much more computationally complex

31. Clustering Solutions (11) Comparison of BLTP & Bayesian Bayesian can identify links with arbitrarily small loss rates BLTP (and GLT) require a parameter (α) as a threshold to account for statistical fluctuations In practice, identification of high loss links is more important and selection of (α) is application-specific.

32. Where are we? Introduction to Networks Network Monitoring Background Introduction to Network Tomography Problem addressed Clustering solutions Conclusions

33. BLTP (Binary-based clustering with pruning) offers the best combination of accuracy and computational simplicity A challenging problem remains to combine the estimated logical topology and other tools (mtrace) to infer physical network topology

34. References S. Ratnasamy and S. McCanne “Inference of multicast routing trees and bottelneck bandwidths using end-to-end measurements,” INFOCOM’99 N.G.Duffield, J.Horowitz, F.Lo Presti, D. Towsley, “Multicast Topology Inference from Measured end- to-end Loss,” IEEE Info. Theory, 2002