On the Geographic Location of Internet Resources Mark Crovella Boston University Computer Science with Anukool Lakhina, John Byers, and Ibrahim Matta or Where on Earth is the Internet?

Some observations about the Internet Rapid, decentralized growth: –90% of Internet systems were added in the last four years –Connecting to the network can be a purely local operation This rapid, decentralized growth has opened significant questions about the physical structure of the network; e.g., –The number of hosts connected to the network –The properties of network links (delay, bandwidth) –The interconnection pattern of hosts and routers –The interconnection relationships of ISPs –The geographic locations of hosts, routers and links

Internet Science Engineering or Science? Engineering: study of things made Science: study of things found Although the Internet is a engineered artifact, it now presents us with questions that are better approached from a scientific posture

For example: where is the Internet? ? ? ? ? ? What is the relationship between geography and the network?

Why does this matter? Our motivation is in developing better generators for “representative network topologies” –Many simulation based results in networking depend critically on network topology –Topology generation is still fairly ad hoc However, there is a scientific goal as well! –We need to understand what drives Internet growth –Basic investigations will pay off in unexpected ways

Assumptions and Definitions We will treat the Internet as an undirected graph embedded on the Earth’s surface –Nodes correspond to routers or interfaces –Edges correspond to physical router-router links –Not concerned with hosts (end systems) We will ignore many higher and lower level questions –Autonomous systems –Link layers

Some Basic Questions What is the relationship between population and network geometry? What effect does distance have on network topology? Our Basic Approach Obtain IP-level router maps  Mercator and Skitter Find geographic locations of those routers  Ixia’s IxMapping service

Mercator: Govindan et al., USC/ISI, ICSI Discovers a router-level map of the Internet using active probing Uses hop-limited UDP packets Addresses a number of difficult problems: –What IP addresses represent valid hosts? –How to observe cross-links from a single vantage point? –How to resolve aliases?

Mercator: Govindan et al., USC/ISI, ICSI Skitter: Moore et al., CAIDA Based on active probing from a single site Resolves aliases Uses loose source routing to explore alternate paths Traceroutes from 19 monitors to large set of destinations Does not resolve aliases Destinations attempt to cover IP address space

Datasets Mercator Collected August ,263 routers 320,149 links Skitter Collected January ,107 interfaces 1,075,454 links

IxMapping: Moore et al., CAIDA Given an IP address, infers geographic location based on a variety of heuristics –Hostnames, DNS LOC, whois e.g., 190.ATM8-0-0.GW3.BOS1.ALTER.NET is in Boston Able to map –99% of Mercator routers –98.5% of Skitter interfaces Similar to GeoTrack [Padmanabhan] which exhibits reasonable accuracy –Median error of 64 mi –90% queries within 250 mi –for well connected nodes

Putting it together Used most recent Mercator dataset (2001) –217,000 routers (after cleanup) –37,000 unique locations –279,000 links Mapped all routers using NetGeo –88% “City”, 10% “Country”, 2% “State/Prov” –Precision on the order of tens of miles or more

Where are the routers? USA

Europe

Routers and People: World (Grid size: ~150 mi x 150 mi) Ugh!

People Per Interface (Skitter) Pop (M) Intfs (K) People / Intfc Online (M) Online / Intfc Africa8378,379100, S. America34110,13133, ,161 Mexico1544,36135, W. Europe36695,9933, ,489 Japan13637,6493, ,250 Australia1818, USA299282,0481, World5,653563,52110, Pop (M) Intfs (K) People / Intfc Online (M) Online / Intfc Africa8378,379100, S. America34110,13133, ,161 Mexico1544,36135, W. Europe36695,9933, ,489 Japan13637,6493, ,250 Australia1818, USA299282,0481, World5,653563,52110,

Interfaces and People: USA, Skitter Grid size: ~90 mi x 90 mi

Routers and People Upper, Mercator; Lower, Skitter USA Europe Japan

Router Location: Summary Router location is strongly driven by both population density and economic development Superlinear relationship between router and population density: R  k P a k varies with economic development (users online) a is greater than one More routers per person in more densely populated areas

Models for Network Topology Spatial Models Structural Models Degree-based Models

Spatial model: Waxman, 1988 Nodes are distributed randomly (uniformly) in the plane. Probability that two nodes separated by distance d are connected: P[C|d] =  exp(-d/  L) 0  ,   1; L = diameter of region  : degree of distance sensitivity  : edge density A spatially imbedded random graph

Structural Models Real networks have structure –Always connected! –Formed by interconnection of component networks –Distinction between transit and stub networks imposes a hierarchy on resulting graph Tiers: Doar, 1996 GT-ITM: Calvert, Doar, Zegura, 1997

Degree-based Models Faloutsos, Faloutsos, & Faloutsos, 1999: –Empirically measured networks show a power law in degree distribution: P[node has degree d] = k d -a Barabasi & Albert, 1999: –This property will be present in a graph where: Nodes and links are added incrementally Probability of connecting to a node is proportional to its degree (preferential connectivity) BRITE: Medina, Matta, Byers, 2001

Empirical Evidence Interested in influence of distance on link formation: f(d) = P[C|d] i.e., Probability two nodes separated by distance d are connected Estimated as: number links of length d f(d) = number of router pairs separated by d

f(d) for USA (Skitter) Distance Sensitive Distance Insensitive

Link Distance Preference for USA Skitter, d < 250, semi-log plot  L  140 mi.

Link distance preference: all regions Upper, Mercator; Lower, Skitter; small d USA  L  140 mi Europe  L  80 mi Japan  L  140 mi

Large d: distance insensitivity F(d) =  f(u) d u=1 USA data, Skitter

Distance insensitivity, all regions Upper, Mercator; Lower, Skitter; large d USA Europe Japan

Limits to distance sensitivity MercatorSkitter Limit% Links < Limit Limit% Links < Limit USA820 mi.82.1%818 mi.77.2% Europe383 mi.97.3%366 mi.95.4% Japan165 mi.91.5%116 mi.92.8%

Link Formation: Summary Link formation seems to be a mixture of distance-dependent and –independent processes Waxman (exponential) model remarkably good for large fraction of all links! –But, crucial difference is that we are using a very irregular spatial distribution of nodes Small fraction of non-local links are very important (structural)

Generating Topologies: a new recipe 1.Good models for population density exist –CIESIN’s Gridded Population of the World –2.5 arc minutes (<5 mi 2 ) … very high quality –Time trends also available 2.Router density then follows from population relationship 3.Link formation driven by hybrid process –Distance-dependent and –independent

Related Work Matrix.net: –Uses DNS hostname allocations –proprietary location methods Akamai –Extensive peering and measurement infrastructure Padmanabhan and Subramanian, 2000: –assessed accuracy of geographic mapping techniques Yook, Jeong, and Barabasi, 2001: –Similar goals –Find linear (not exponential) distance dependence

Final Thoughts The Internet has fully interpenetrated human society Scientific understanding of the net is essential Applications: –Simulation –Security –Reliability –Planning

Thanks! CAIDA: –David Moore –k claffy –Andre Broido Notre Dame: –Lazslo Barabasi USC: –Ramesh Govindan –Hongsuda Tangmunarunkit

Routers and People: North America

Subdividing the Data N. US C. A. S. US

Economic Heterogeneity

What Influences the Formation of Links? Waxman, 1988: spatial model Zegura et al., 1997: structural model –Explicitly captures hierarchical structure Barabasi et al., 1999: degree-based preferential connectivity –Matches observed power-law node degree –Inspired by Faloutsos et al., 1999 Strogatz & Watts, 1998: small-world properties –Captures “six degrees of separation”

Routers and Economics Matrix.net: hosts Mercator: routers

Penultimate Geographic Map, Oct 1980

Last Complete Geographic Map, Aug 1982