Download presentation
Presentation is loading. Please wait.
Published byKory Edwards Modified over 6 years ago
1
Comparative Analysis of Internet Topology Data Sets
Jay Thom
2
Outline Introduction Problem Statement Methodology Conclusion 2
3
Introduction What is Internet Topology? Why measure the Internet?
How is this done?
4
Topology Data Sets Caida-Archipelago (Ark) Measurement Lab (M-Lab)
Ripe NNC Atlas University of Washington iPlane ISI Ant Census Internet Research Lab (IRL) CIDR
5
The Problem… Big problem: Smaller problem:
What does the Internet look like right now? Smaller problem: Acquire data to infer this topology Collect data Recurring collection Python vs. C/C++ Parse data Collect statistical information Make comparisons
6
Data Collection Data stored in numerous formats…
Ripe - .json files at anchors Ark - .warts (scamper), compressed binary files iPlane - compressed binary files, iPlane.c M-Lab - Google cloud storage, nested compressed files Ant Census - Released every 2 months UCSD CAIDA (BGP Data) - compressed text files CIDR – compressed text files IRL – (BGP Data) - compressed text files Retrieve traceroute files as needed by date Python vs. C/C++
7
Data Cleaning and Parsing
Remove all un-necessary information Parse data into a common format Store in a consistent manner 30-day set vs. 5-day set 30-day set = 1TB 5-day set = 181GB reduce size to save time
8
Total Unique Source/Destination IP Addresses
For each data source, how many unique source or destination IP addresses are found? This will indicate the number of vantage points or targets the data source has access to. Question: does the number of vantage points/targets affect how much of the Internet a source can see? Question: what is the relationship between number of vantage points/targets and the number of unique traces, unique IP addresses, and unique edges found?
9
Total Unique Traces How many unique traces is each data source able to find? Why would one source find more than another? What mechanisms are present that would affect these numbers?
10
Total Unique Edges Visited
Question: what does an edge represent? Connection between two routers An ingress/egress point between two ASes
11
IP/Trace Counts Number of unique IP addresses vs. all collected IPs
Number of unique traces vs. all collected traces Question: Why is this important? How many times is a data source repeating the same measurements? How many duplicated efforts are seen? Why would this be?
12
Problem - Unresponsive Routers
13
Problem - Unresponsive Routers
Count as an edge? Keep, or disregard? If kept, how should they be noted?
14
Distribution per Source/Destination IP
Find the distribution of our data points per source and per destination Analyze this to understand the effectiveness of each platform’s approach to measurement IPs Traces Edges Sources Destinations
15
Firewalls, Loops, Repeated IP Addresses
A-B-C-C A-B-C-D-C A-B-C-C-D A-B-B-C
16
Ripe Atlas Hardware Rack mounted anchor
Small probe (connected anywhere)
17
Ripe Atlas: User Defined Measurements
18
Trace IP in traces not seen in Ant Census
Question: will some IP addresses be discovered in traces that were not found in the Ant census? Some addresses will respond to ICMP time exceeded that will not respond to ICMP echo request New IP addresses will be discovered that can then be used as active target IP addresses for future probes
19
Prefix Announcements vs. Mask Distribution
Question: What is the distribution of subnets that are announced by each AS data source: CAIDA, IRL, CIDR Why do some perform better than others?
20
Conflicts in Subnet Announcements
/30 /29 /29 /28 /28 /28 /28 Determine total number of subnets announced Combine all smaller subnets to see if they make up a complete larger subnet Compare larger subnets to see if they are announced by more than one AS in a data set Analyze to determine if sources clean up conflicts
21
Trace Data Coverage by BGP Data
Not all IP addresses found in our trace data will be visible by our AS data sources BGP data comes from RouteViews project, Univ. of Oregon May not see addresses, say somewhere in Asia Track statistics on IP addresses not found by data sources Track AS coverage per data source Track total number of prefixes announced by data source
22
AS Rank by Origin, Destination, IP, Edge
Rank ASes by the number of data points found in each per data source Compare coverage of ASes by each trace data source Question: Why are some ASes more visible to sources than others?
23
AS Coverage by Origin, Destination, IP, Edge
Track numbers of source iPs, destination IPs, total numbers of IPs, and total edges per AS by data source Rank sources based on these values (which source sees how many ASes per value) Create visual graphs of these ASes, collect and analyze graph data such as degree, centrality, etc. (use tool from CAIDA)
24
Conclusion Problem Statement Methodology Collection Parsing Statistics
Analysis Problems
25
Questions?
26
Thanks
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.