1. Comparative Analysis of Internet Topology Data Sets
Jay Thom

2. Outline
Introduction
Problem Statement
Methodology
Conclusion 2

3. Introduction What is Internet Topology? Why measure the Internet?
How is this done?

4. Topology Data Sets Caida-Archipelago (Ark) Measurement Lab (M-Lab)
Ripe NNC Atlas
University of Washington iPlane
ISI Ant Census
Internet Research Lab (IRL)
CIDR

5. The Problem… Big problem: Smaller problem:
What does the Internet look like right now?
Smaller problem:
Acquire data to infer this topology
Collect data
Recurring collection
Python vs. C/C++
Parse data
Collect statistical information
Make comparisons

6. Data Collection Data stored in numerous formats…
Ripe - .json files at anchors
Ark - .warts (scamper), compressed binary files
iPlane - compressed binary files, iPlane.c
M-Lab - Google cloud storage, nested compressed files
Ant Census - Released every 2 months
UCSD CAIDA (BGP Data) - compressed text files
CIDR – compressed text files
IRL – (BGP Data) - compressed text files
Retrieve traceroute files as needed by date
Python vs. C/C++

7. Data Cleaning and Parsing
Remove all un-necessary information
Parse data into a common format
Store in a consistent manner
30-day set vs. 5-day set
30-day set = 1TB
5-day set = 181GB
reduce size to save time

8. Total Unique Source/Destination IP Addresses
For each data source, how many unique source or destination IP addresses are found?
This will indicate the number of vantage points or targets the data source has access to.
Question: does the number of vantage points/targets affect how much of the Internet a source can see?
Question: what is the relationship between number of vantage points/targets and the number of unique traces, unique IP addresses, and unique edges found?

9. Total Unique Traces
How many unique traces is each data source able to find?
Why would one source find more than another?
What mechanisms are present that would affect these numbers?

10. Total Unique Edges Visited
Question: what does an edge represent?
Connection between two routers
An ingress/egress point between two ASes

11. IP/Trace Counts Number of unique IP addresses vs. all collected IPs
Number of unique traces vs. all collected traces
Question: Why is this important?
How many times is a data source repeating the same measurements?
How many duplicated efforts are seen?
Why would this be?

12. Problem - Unresponsive Routers

13. Problem - Unresponsive Routers
Count as an edge?
Keep, or disregard?
If kept, how should they be noted?

14. Distribution per Source/Destination IP
Find the distribution of our data points per source and per destination
Analyze this to understand the effectiveness of each platform’s approach to measurement
IPs
Traces
Edges
Sources
Destinations

15. Firewalls, Loops, Repeated IP Addresses
A-B-C-C
A-B-C-D-C
A-B-C-C-D
A-B-B-C

16. Ripe Atlas Hardware Rack mounted anchor
Small probe (connected anywhere)

17. Ripe Atlas: User Defined Measurements

18. Trace IP in traces not seen in Ant Census
Question: will some IP addresses be discovered in traces that were not found in the Ant census?
Some addresses will respond to ICMP time exceeded that will not respond to ICMP echo request
New IP addresses will be discovered that can then be used as active target IP addresses for future probes

19. Prefix Announcements vs. Mask Distribution
Question: What is the distribution of subnets that are announced by each AS data source:
CAIDA, IRL, CIDR
Why do some perform better than others?

20. Conflicts in Subnet Announcements
/30
/29
/29
/28
/28
/28
/28
Determine total number of subnets announced
Combine all smaller subnets to see if they make up a complete larger subnet
Compare larger subnets to see if they are announced by more than one AS in a data set
Analyze to determine if sources clean up conflicts

21. Trace Data Coverage by BGP Data
Not all IP addresses found in our trace data will be visible by our AS data sources
BGP data comes from RouteViews project, Univ. of Oregon
May not see addresses, say somewhere in Asia
Track statistics on IP addresses not found by data sources
Track AS coverage per data source
Track total number of prefixes announced by data source

22. AS Rank by Origin, Destination, IP, Edge
Rank ASes by the number of data points found in each per data source
Compare coverage of ASes by each trace data source
Question: Why are some ASes more visible to sources than others?

23. AS Coverage by Origin, Destination, IP, Edge
Track numbers of source iPs, destination IPs, total numbers of IPs, and total edges per AS by data source
Rank sources based on these values (which source sees how many ASes per value)
Create visual graphs of these ASes, collect and analyze graph data such as degree, centrality, etc. (use tool from CAIDA)

24. Conclusion Problem Statement Methodology Collection Parsing Statistics
Analysis
Problems

25. Questions?

26. Thanks