A First-Principles Approach to Understanding the Internet’s Router-level Topology 2004 ACM SIGCOMM Portland, OR

Evaluate performance of protocols Protect Internet Resource provisioning Understand large scale networks Why Topology Challenges Large Size Real topologies are not publicly available Incredible variability in many aspects

Trends in Topology Modeling Observation Modeling Approach Real networks are not random, but have obvious hierarchy. Structural models (GT-ITM Calvert/Zegura, 1996) Long-range links are expensive Random graph models (Waxman, 1988) Internet topologies exhibit power law degree distributions (Faloutsos et al., 1999) Degree-based models replicate power-law degree sequences

A few nodes have lots of connections Rank R(d) Degree d Source: Faloutsos et al. (1999) Power Laws and Internet Topology Router-level graph & Autonomous System (AS) graph Led to active research in degree-based network models Most nodes have few connections R(d) = P (D>d) x #nodes

Degree-Based Models of Topology Preferential Attachment –Growth by sequentially adding new nodes –New nodes connect preferentially to nodes having more connections –Examples: Inet, GPL, AB, BA, BRITE, CMU power-law generator

Degree-Based Models of Topology Expected Degree Sequence –Based on random graph models that skew probability distribution to produce power laws in expectation –Examples: Power Law Random Graph (PLRG), Generalized Random Graph (GRG) Preferential Attachment (PA) –Growth by sequentially adding new nodes –New nodes connect preferentially to nodes having more connections –Examples: Inet, GPL, AB, BA, BRITE, CMU power-law generator

Features of Degree-Based Models Degree sequence follows a power law (by construction) High-degree nodes correspond to highly connected central “ hubs ”, which are crucial to the system Achilles ’ heel: robust to random failure, fragile to specific attack “ scale-free ” in complex networks Preferential AttachmentExpected Degree Sequence

Our Approach Consider the explicit design of the Internet –Annotated network graphs (capacity, bandwidth) –Technological and economic limitations –Network performance –Heuristic optimized tradeoffs (HOT) Seek a theory for Internet topology that is explanatory and not merely descriptive. –Explain high variability in network connectivity –Ability to match large scale statistics (e.g. power laws) is only secondary evidence

Trends in Topology Modeling Observation Modeling Approach Real networks are not random, but have obvious hierarchy. Structural models (GT-ITM Calvert/Zegura, 1996) Long-range links are expensive Random graph models (Waxman, 1988) Internet topologies exhibit power law degree distributions (Faloutsos et al., 1999) Degree-based models replicate power-law degree sequences Physical networks have hard technological (and economic) constraints. Optimization-driven models topologies consistent with design tradeoffs of network engineers

Degree Bandwidth (Gbps) 15 x 10 GE 15 x 3 x 1 GE 15 x 4 x OC12 15 x 8 FE Technology constraint Total Bandwidth Bandwidth per Degree Router Technology Constraint Cisco GSR, circa 2002 high BW low degree high degree low BW

approximate aggregate feasible region Aggregate Router Feasibility Source: Cisco Product Catalog, June 2002 core technologies edge technologies older/cheaper technologies

Rank (number of users) Connection Speed (Mbps) 1e-1 1e-2 1 1e1 1e2 1e3 1e4 1e211e4 1e6 1e8 Dial-up ~56Kbps Broadband Cable/DSL ~500Kbps Ethernet Mbps Ethernet 1-10Gbps most users have low speed connections a few users have very high speed connections high performance computing academic and corporate residential and small business Variability in End-User Bandwidths

Heuristically Optimal Topology Hosts Edges Cores Mesh-like core of fast, low degree routers High degree nodes are at the edges.

SOX SFGP/ AMPATH U. Florida U. So. Florida Miss State GigaPoP WiscREN SURFNet Rutgers U. MANLAN Northern Crossroads Mid-Atlantic Crossroads Drexel U.U. Delaware PSC NCNI/MCNCMAGPI UMD NGIX DARPA BossNet GEANT Seattle Sunnyvale Los Angeles Houston Denver Kansas City Indian- apolis Atlanta Wash D.C. Chicago New York OARNET Northern Lights Indiana GigaPoP Merit U. Louisville NYSERNet U. Memphis Great Plains OneNet Arizona St. U. Arizona Qwest LabsUNM Oregon GigaPoP Front Range GigaPoP Texas TechTulane U. North Texas GigaPoP Texas GigaPoP LaNet UT Austin CENICUniNet WIDE AMES NGIX Pacific Northwest GigaPoP U. Hawaii Pacific Wave ESnet TransPAC/APAN Iowa St. Florida A&M UT-SW Med Ctr. NCSA MREN SINet WPI StarLight Intermountain GigaPoP Abilene Backbone Physical Connectivity (as of December 16, 2003) Gbps Gbps Gbps Gbps

Cisco 750X Cisco Cisco OC-3 (155 Mb/s) OC-12 (622 Mb/s) GE (1 Gb/s) OC-48 (2.5 Gb/s) 10GE (10 Gb/s) CENIC Backbone (as of January 2004) dc1 dc2 dc3 hpr dc1 dc3 hpr dc2 dc1 dc2 hpr SAC OAK SVL LAX SDG SLO dc1 FRG dc1 FRE dc1 BAK dc1 TUS dc1 SOL dc1 COR dc1 hpr dc1 dc2 dc3 hpr Abilene Los Angeles Abilene Sunnyvale Corporation for Education Network Initiatives in California (CENIC) CENIC Router Configurations, Jan. 2004

Two Different Perspectives Degree-based perspective –Match aggregate statistics –Suggest high-degree central hubs First principles perspective –Technology and economic constraints –Performance –Suggest fast, low-degree core routers –Consistent with physical design of real networks How to reconcile these two perspectives?

Same Degree Distribution PA

PLRG Same Degree Distribution

HOT Same Degree Distribution

PAPLRG/GRG HOTAbilene-inspiredSub-optimal

Network Performance Given realistic technology constraints on routers, how well is the network able to carry traffic? Step 1: Constrain to be feasible Abstracted Technologically Feasible Region degree Bandwidth (Mbps) Step 3: Compute max flow BiBi BjBj x ij Step 2: Compute traffic demand

PAPLRG/GRGHOT Structure Determines Performance P(g) = 1.19 x P(g) = 1.64 x P(g) = 1.13 x 10 12

Likelihood-Related Metric Easily computed for any graph Depends on the structure of the graph, not the generation mechanism Measures how “hub-like” the network core is Define the metric( d i = degree of node i ) For graphs resulting from probabilistic construction (e.g. PLRG/GRG), LogLikelihood (LLH)  L(g) Interpretation: How likely is a particular graph (having given node degree distribution) to be constructed?

L max l(g) = 1 P(g) = 1.08 x P(g) Perfomance (bps) PA PLRG/GRG HOT Abilene-inspiredSub-optimal l(g) = Relative Likelihood

L max l(g) = 1 P(g) = 1.08 x P(g) Perfomance (bps) PA PLRG/GRG HOT Abilene-inspiredSub-optimal l(g) = Relative Likelihood

Points to Emphasize Same Degree distribution can have different core structures Same Core structure can have different degree distributions

Abilene-inspired core Uniform high BW users Low variability deg dist. Abilene-inspired core High variability at edges Power-law deg distribution Abilene-inspired core Uniform low BW users low variability deg dist.

Conclusions 1.Probabilistic degree-based generation mechanisms are likely to produce networks that have poor performance and are very unlikely to generate realistic networks. 2.Models of router-level topology should consider inherent technological and economic tradeoffs in network design, and they should consider issues such as performance over simple connectivity. 3.Realistic router-level topology generators will require additional work to incorporate other key features (e.g. geography, population density) into the framework

Future Work Validation –rely on cooperative ISPs to validate router technology constraints –combine with heuristics and supplementary measurements –develop annotated prototype ISP topology generator

A Tier 1 Router Bandwidth-Degree (Courtesy: Matt Roughan)

Future Work Validation –rely on cooperative ISPs to validate router technology constraints –combine with heuristics and supplementary measurements –develop annotated prototype ISP topology generator –Contact: David Alderson HOT-inspired framework for protocol stack –consider layering as optimization decomposition –study properties of `natural' decompositions –Contact: Lun Li