Presentation is loading. Please wait.

Presentation is loading. Please wait.

Comparative Analysis of Internet Topology Data Sets

Similar presentations


Presentation on theme: "Comparative Analysis of Internet Topology Data Sets"— Presentation transcript:

1 Comparative Analysis of Internet Topology Data Sets
Jay Thom

2 Outline Introduction Problem Statement Methodology Conclusion 2

3 Introduction What is Internet Topology? Why measure the Internet?
How is this done?

4 Topology Data Sets Caida-Archipelago (Ark) Measurement Lab (M-Lab)
Ripe NNC Atlas University of Washington iPlane ISI Ant Census Internet Research Lab (IRL) CIDR

5 The Problem… Big problem: Smaller problem:
What does the Internet look like right now? Smaller problem: Acquire data to infer this topology Collect data Recurring collection Python vs. C/C++ Parse data Collect statistical information Make comparisons

6 Data Collection Data stored in numerous formats…
Ripe - .json files at anchors Ark - .warts (scamper), compressed binary files iPlane - compressed binary files, iPlane.c M-Lab - Google cloud storage, nested compressed files Ant Census - Released every 2 months UCSD CAIDA (BGP Data) - compressed text files CIDR – compressed text files IRL – (BGP Data) - compressed text files Retrieve traceroute files as needed by date Python vs. C/C++

7 Data Cleaning and Parsing
Remove all un-necessary information Parse data into a common format Store in a consistent manner 30-day set vs. 5-day set 30-day set = 1TB 5-day set = 181GB reduce size to save time

8 Total Unique Source/Destination IP Addresses
For each data source, how many unique source or destination IP addresses are found? This will indicate the number of vantage points or targets the data source has access to. Question: does the number of vantage points/targets affect how much of the Internet a source can see? Question: what is the relationship between number of vantage points/targets and the number of unique traces, unique IP addresses, and unique edges found?

9 Total Unique Traces How many unique traces is each data source able to find? Why would one source find more than another? What mechanisms are present that would affect these numbers?

10 Total Unique Edges Visited
Question: what does an edge represent? Connection between two routers An ingress/egress point between two ASes

11 IP/Trace Counts Number of unique IP addresses vs. all collected IPs
Number of unique traces vs. all collected traces Question: Why is this important? How many times is a data source repeating the same measurements? How many duplicated efforts are seen? Why would this be?

12 Problem - Unresponsive Routers

13 Problem - Unresponsive Routers
Count as an edge? Keep, or disregard? If kept, how should they be noted?

14 Distribution per Source/Destination IP
Find the distribution of our data points per source and per destination Analyze this to understand the effectiveness of each platform’s approach to measurement IPs Traces Edges Sources Destinations

15 Firewalls, Loops, Repeated IP Addresses
A-B-C-C A-B-C-D-C A-B-C-C-D A-B-B-C

16 Ripe Atlas Hardware Rack mounted anchor
Small probe (connected anywhere)

17 Ripe Atlas: User Defined Measurements

18 Trace IP in traces not seen in Ant Census
Question: will some IP addresses be discovered in traces that were not found in the Ant census? Some addresses will respond to ICMP time exceeded that will not respond to ICMP echo request New IP addresses will be discovered that can then be used as active target IP addresses for future probes

19 Prefix Announcements vs. Mask Distribution
Question: What is the distribution of subnets that are announced by each AS data source: CAIDA, IRL, CIDR Why do some perform better than others?

20 Conflicts in Subnet Announcements
/30 /29 /29 /28 /28 /28 /28 Determine total number of subnets announced Combine all smaller subnets to see if they make up a complete larger subnet Compare larger subnets to see if they are announced by more than one AS in a data set Analyze to determine if sources clean up conflicts

21 Trace Data Coverage by BGP Data
Not all IP addresses found in our trace data will be visible by our AS data sources BGP data comes from RouteViews project, Univ. of Oregon May not see addresses, say somewhere in Asia Track statistics on IP addresses not found by data sources Track AS coverage per data source Track total number of prefixes announced by data source

22 AS Rank by Origin, Destination, IP, Edge
Rank ASes by the number of data points found in each per data source Compare coverage of ASes by each trace data source Question: Why are some ASes more visible to sources than others?

23 AS Coverage by Origin, Destination, IP, Edge
Track numbers of source iPs, destination IPs, total numbers of IPs, and total edges per AS by data source Rank sources based on these values (which source sees how many ASes per value) Create visual graphs of these ASes, collect and analyze graph data such as degree, centrality, etc. (use tool from CAIDA)

24 Conclusion Problem Statement Methodology Collection Parsing Statistics
Analysis Problems

25 Questions?

26 Thanks


Download ppt "Comparative Analysis of Internet Topology Data Sets"

Similar presentations


Ads by Google