1. Lecture 2: Internet Measurement CS 790g: Complex Networks

2. Final Project Analyze a network What it should be More than just a measurement of network characteristics An interpretation of measurement results If applicable: discovery of community or other structures motifs weights, thresholds longitudinal data (how the network changes over time) Visualizations of the network that point out a particular feature Qualitative comparison with other networks What it should not be a literature review recapitulation of existing work raw analysis of data The data can be artificially generated or a real-world dataset 2

3. Final Project New network model What it should be Method for generating a network e.g. preferential attachment optimization wrt. different criteria Analysis of resulting network comparison with random graphs how do attributes change depending on model parameters What it should not be an already thoroughly explored model 3

4. Final Project Theory development What it should be An algorithm to analyze the network e.g. clustering or community detection algorithm webpage ranking algorithm OR a process that is influenced by the network gossip spreading games such as the prisoner’s dilemma Analysis of algorithm on several different networks What it should not be an exact replica of an existing algorithm applied to a network where it has already been studied 4

5. Final Project Epidemic Characterization What it should be In-depth study of an epidemic phenomena fads in online content; virus and worm spreading in information networks; or word-of-mouth in product marketing What it should not be a replica of an existing study 5

6. 6

7. Web of interconnected networks Grows with no central authority Autonomous Systems optimize local communication efficiency The building blocks are engineered and studied in depth Global entity has not been characterized Most real world complex-networks have non-trivial properties. Global properties can not be inferred from local ones Engineered with large technical diversity Range from local campuses to transcontinental backbone providers Internet 7

8. Need for Internet measurements arises due to commercial, social, and technical issues Realistic simulation environment for developed products, Improve network management Robustness with respect to failures/attacks Comprehend spreading of worms/viruses Know social trends in Internet use Scientific discovery Scale-free (power-law), Small-world, Rich-club, Dissasortativity,… Internet Measurements 8

9. Internet Topology Measurement CAIDA 2006 9

10. Internet Topology Measurement 10 CAIDA 2006 10

11. Internet Topology Measurement 11

12. Internet Topology Measurement 12 CAIDA 2006 12

13. Direct probing Indirect probing A DBC Internet Topology Measurements Probing IP B TTL=64 IP B IP D TTL=64 IP D Vantage Point A DBC IP B IP D TTL=2IP D TTL=1 IP C 13 http://www.caida.org/publications/animations/active_monitoring/traceroute.mpg

14. Probe packets are carefully constructed to elicit intended response from a probe destination traceroute probes all nodes on a path towards a given destination TTL-scoped probes obtain ICMP error messages from routers on the path ICMP messages includes the IP address of intermediate routers as its source Merging end-to-end path traces yields the network map S DABC Destination Internet Topology Measurement Topology Collection (traceroute) TTL=1 IP A TTL=2 IP B TTL=3 IP C TTL=4 IP D Vantage Point 14

15. Internet Topology Measurement: Background 15 S L U H C N W A s.2 l.1 s.3 u.1 l.3 u.3 h.1 k.3 h.2 h.3 a.3 u.2 k.1 c.4 a.1 a.2 w.3 c.3 w.1 c.2 n.1 n.3 w.2 l.2 K c.1 k.2 d h.4 Trace to Seattle h.4 l.3 s.2 Trace to NY h.4 a.3 w.3 n.3 Internet2 backbone

16. Internet Topology Measurement: Background 16 S L U C N A s.2 l.1 s.3 u.1 l.3 h.1 k.3 h.2 a.3 u.2 k.1 c.4 a.1 a.2 w.3 c.3 w.1 c.2 n.1 n.3 w.2 l.2 K c.1 k.2 h.3 d h.4 s.1 e f n.2 H W u.3

17. 17 Sampling to discover networks Infer characteristics of the topology Different studies considered Effect of sample size [Barford 01] Sampling bias [Lakhina 03] Path accuracy [Augustin 06] Sampling approach [Gunes 07] Utilized protocol [Gunes 08] ICMP echo request TCP syn UDP port unreachable Topology Sampling Issues

18. Anonymous Router Resolution Problem Anonymous routers do not respond to traceroute probes and appear as a  in path traces Same router may appear as a  in multiple traces. Anonymous nodes belonging to the same router should be resolved. Anonymity Types 1. Ignore all ICMP packets 2. ICMP rate-limiting 3. Ignore ICMP when congested 4. Filter ICMP at border 5. Private IP address 18

19. Anonymous Router Resolution Problem Internet2 backbone S L U K C H A W N e d Traces d -  - L - S - e d -  - A - W -  - f e - S - L -  - d e - S - U -  - C -  - f f -  - C -  -  - d f -  - C -  - U - S - e 19 f

20. Anonymous Router Resolution Problem UKCN LHAW S d e f Sampled network d e f S U L C A W Resulting network 20 Traces d -  - L - S - e d -  - A - W -  - f e - S - L -  - d e - S - U -  - C -  - f f -  - C -  -  - d f -  - C -  - U - S - e

21. 21 Graph Based Induction Common Structures Parallel nodes A x C y2 y1 y3    A x C y2 y1 y3  Star DA wx C y E z  DA wx C y E z    Complete Bipartite A C x y D w F v E z  A C x y D w F v E z       Clique A C x y D w E z  A C x y D w E z      

22. Each interface of a router has an IP address. A router may respond with different IP addresses to different queries. Alias Resolution is the process of grouping the interface IP addresses of each router into a single node. Inaccuracies in alias resolution may result in a network map that includes artificial links/nodes misses existing links Alias Resolution:.5.33.18.13.7 Denver 22

23. 23 S L U C N W A s.2 l.1 s.3 u.1 l.3 u.3 h.1 k.3 h.2 a.3 u.2 k.1 c.4 a.1 a.2 w.3 c.3 w.1 c.2 n.1 n.3 w.2 l.2 K c.1 k.2 h.3 d h.4 s.1 e f n.2 H Traces d - h.4 - l.3 - s.2 - e d - h.4 - a.3 - w.3 - n.3 - f e - s.1 - l.1 - h.1 - d e - s.1 - u.1 - k.1 - c.1 - n.1 - f f - n.2 - c.2 - k.2 - h.2 - d f - n.2 - c.2 - k.2 - u.2 - s.3 - e IP Alias Resolution Problem

24. 24 IP Alias Resolution Problem UKCN LHAW S d e f Sampled network Sample map without alias resolution s.3 s.1 s.2 l.3 l.1 u.1 u.2 k.1 c.1n.1 n.2 k.2 c.2 w.3 a.3 h.2 h.4 h.1 e d f n.3 Traces d - h.4 - l.3 - s.2 - e d - h.4 - a.3 - w.3 - n.3 - f e - s.1 - l.1 - h.1 - d e - s.1 - u.1 - k.1 - c.1 - n.1 - f f - n.2 - c.2 - k.2 - h.2 - d f - n.2 - c.2 - k.2 - u.2 - s.3 - e

25. 25 Genuine Subnet Resolution Problem Alias resolution IP addresses that belong to the same router Subnet resolution IP addresses that are connected over the same medium IP2IP3 IP4 IP1 IP6IP5 IP2 IP3 IP1 IP2IP3 IP1

26. Autonomous System Level 26

27. 27

28. 28 http://www.caida.org/publications/animations/active_monitoring/as_core.mpg

29. Traffic Measurements Monitoring and measuring network traffic to produce better models of network behavior to diagnose failures and detect anomalies to defend against unwanted traffic Live weather map Internernet2 PlanetLab 29

30. Code-Red Worm On July 19, 2001, more than 359,000 computers connected to the Internet were infected with the Code-Red (CRv2) worm in less than 14 hours Spread 30

31. Sapphire Worm was the fastest computer worm in history doubled in size every 8.5 seconds infected more than 90 percent of vulnerable hosts within 10 minutes. 31

32. Witty Worm reached its peak activity after approximately 45 minutes at which point the majority of vulnerable hosts had been infected World USA 32

33. Nyxem Email Virus Estimate of total number of infected computers is between 470K and 945K At least 45K of the infected computers were also compromised by other forms of spyware or botware Spread 33

34. Scam Hosting Study dynamics of scam hosting infrastructure 34

35. Measurement Studies Glasnost tests whether BitTorrent is being blocked or throttled BW-meter Measurement tools for the capacity and load of Internet paths NPAD Diagnostics Servers Automatic diagnostic server for troubleshooting end-systems and last-mile network problems iPlane construct a router interface-level atlas of the Internet measuring link attributes Hubble find persistent Internet black holes as they occur 35

36. Internet Measurements The Internet is man-made, so why do we need to measure it? Because we still don’t really understand it Sometimes things go wrong Malicious users Measurement for network operations Detecting and diagnosing problems What-if analysis of future changes Measurement for scientific discovery Creating accurate models that represent reality Identifying new features and phenomena 36