Stable Internet Routing Without Global Coordination Jennifer Rexford AT&T Labs--Research

Internet Architecture  The Internet is –Decentralized: loose confederation of peers –Self-configuring: no global registry of topology –Stateless: limited information in the routers –Connectionless: no fixed connection between hosts  These attributes contribute –To the success of Internet –To the rapid growth of the Internet –… and the difficulty of controlling the Internet! senderreceiver

Research Overview: Internet Control Plane  Managing a single Autonomous System –Measurement of traffic, routing, and configuration –Traffic engineering techniques and tools –Configuration checking and automation  Interdomain routing between ASes –Routing protocol convergence –Inference of commercial relationships –Characterization of routing dynamics  End-to-end troubleshooting –Root-cause analysis of routing changes –Measuring the AS-level forwarding path –DARPA Knowledge Plane seedling This talk

Internet Routing Architecture  Divided into Autonomous Systems (ASes) –Distinct regions of administrative control –Routers/links managed by a single institution –Service provider, company, university, …  Hierarchy of Autonomous Systems –Nation-wide tier-1 provider –Medium-sized regional provider –Small university or corporate network  Interaction between Autonomous Systems –Internal topology is not shared between ASes –… but, neighboring ASes interact to coordinate routing

Autonomous Systems (ASes) Client Web server Path: 6, 5, 4, 3, 2, 1

Interdomain Routing: Border Gateway Protocol  ASes exchange info about who they can reach –IP prefix: block of destination IP addresses –AS path: sequence of ASes along the path  Policies configured by the AS’s network operator –Path selection: which of the paths to use? –Path export: which neighbors to tell? “I can reach /24” “I can reach /24 via AS 1” data traffic

Interdomain Routing Challenges  Scale –Destination prefixes: 150,000 and growing –Autonomous Systems: 17,000 visible ones, and growing –AS paths and routers: at least in the millions…  Policy –Path selection: selecting which path your AS wants to use –Path export: controlling who can send packets through your AS  Convergence –VoIP and video games need convergence in tens of milliseconds –Routing protocol convergence can take several (tens of) minutes –… and the routing system doesn’t necessarily converge at all!

Conflicting Policies Cause Convergence Problems Pick the highest-ranked path consistent with your neighbors’ choices. Only choice! Top choice! Only choice! Better choice! Only choice! Better choice!

Global Control is Not Workable  Create a global Internet routing registry –Keeping the registry up-to-date would be difficult  Require each AS to publish its routing policies –ASes may be unwilling to reveal BGP policies  Check for conflicting policies, and resolve conflicts –Checking for convergence problems is NP-complete –Link/router failure may result in an unstable system Need a solution that does not require global coordination.

Think Globally, Act Locally  Key features of a good solution –Flexibility: allow complex local policies for each AS –Privacy: do not force ASes to divulge their policies –Backwards-compatibility: no changes to BGP –Guarantees: convergence even when system changes  Restrictions based on AS relationships –Path selection rules: which route you prefer –Export policies: who you tell about your route –AS graph structure: who is connected to who

Customer-Provider Relationship  Customer pays provider for access to the Internet –Provider exports its customer’s routes to everybody –Customer exports provider’s routes only to downstream customers d d provider customer provider Traffic to the customerTraffic from the customer advertisements traffic

Peer-Peer Relationship  Peers exchange traffic between their customers –AS exports only customer routes to a peer –AS exports a peer’s routes only to its customers peer Traffic to/from the peer and its customers d advertisements traffic

Hierarchical AS Relationships  Provider-customer graph is a directed, acyclic graph –If u is a customer of v and v is a customer of w –… then w is not a customer of u u v w

Our Local Path Selection Rules  Classify routes based on next-hop AS –Customer routes, peer routes, and provider routes  Rank routes based on classification –Prefer customer routes over peer and provider routes  Allow any ranking of routes within a class –Do not impose ranking between customer routes, or between peer and provider routes  Consistent with traffic engineering practices –Customers pay for service, and providers are paid –Peer relationship contingent on balanced traffic load

Solving the Convergence Problem  Assumptions –Export policies based on AS relationships –Path selection rule that favors customer routes –Acyclic provider-customer graph  Result –Guaranteed convergence of the routing protocol –Holds under link/router failures and policy changes  Sketch of (constructive) proof –Activation sequence that leads to a stable state –Any “fair” activation sequence includes this sequence

Proof, Phase 1: Selecting Customer Routes  Activate ASes in customer-provider order –AS picks a customer route if one exists –Decision of one AS cannot cause an earlier AS to change its mind d An AS picks a customer route when one exists

Proof, Phase 2: Selecting Peer and Provider Routes  Activate rest of ASes in provider-customer order –Decision of one phase-2 AS cannot cause an earlier phase-2 AS to change its mind –Decision of phase-2 AS cannot affect a phase 1 AS AS picks a peer or provider route when no customer route is available d

Economic Incentives Affect Protocol Behavior  ASes already follow our rules, so system is stable –High-level argument »Export and topology assumptions are reasonable »Path selection rule matches with financial incentives –Empirical results [IMW’02] »BGP routes for popular destinations are stable for ~10 days »Most churn due to small number of unpopular destinations  ASes should follow our rules to make system stable –Need to encourage operators to obey these guidelines –… and provide ways to verify the network configuration –Need to consider more complex relationships and graphs

Different Rules: More Flexible Import Policies  Allowing more flexibility in ranking routes –Allow the same rank for peer and customer routes with the same AS path length –Never choose a peer route over a shorter customer route  Stricter AS graph assumptions –Hierarchical provider-customer relationship (as before) –No private peering with (direct or indirect) providers Peer-peer

Backup Relationships [INFOCOM’01]  Backups: more liberal export policies –Primary and a backup provider –Peers giving backup service to each other  Generalize rule: prefer routes with fewest backup links backup path primary provider backup provider failure Backup Provider backup path failure peer provider Peer-Peer Backup

More Complex Scenarios  Complex AS relationships –AS pair with different relationship for different prefixes –AS pair with both a backup and a peer relationships –AS providing transit service between two peer ASes  Stability under changing AS relationships –Customer-provider to/from peer-peer –Customer-provider to/from provider-customer

Conclusions  Avoiding convergence problems –Hierarchical AS relationships –Export policies based on commercial relationships –Guidelines for import policies based on relationships  Salient features –No global coordination (locally implementable) –No changes to BGP protocol or decision process –Guaranteed convergence, even under failures –Guidelines consistent with financial incentives

Broader Influence of the Work  AS relationships and BGP convergence –Algebraic framework and design principles for policy languages –Fundamental limits on relaxing the assumptions  Internal BGP inside an AS –Sufficient conditions for iBGP convergence inside an AS –“What-if” tool for traffic engineering inside an AS  AS-level analysis of the Internet –Inference of AS relationships from routing data –Characterization of AS-level topology and growth  Network design and operations (e.g., at AT&T) –Analyzing competitors and changing BGP policies –Setting protective route filters on BGP sessions

Longer-Term Research Plans: Internet Control Plane  Internet routing architecture –Routing Control Point for moving intelligence out of the routers –Distributed troubleshooting service based on measurement  Router, protocol, and language extensions –Extra routing information to help in troubleshooting –Lightweight support for measurement in routers –Better router and network configuration languages  Campus, enterprise, and regional networks –Fertile ground for new research problems –New sources of measurement data and “tech transfer”