BGP: Inter-Domain Routing Protocol Noah Treuhaft U.C. Berkeley

The need for routing in the Internet Need to get packets from source to destination How do you do this? –Network is a collection of point-to-point links connected by routers –Routers’ decisions determine which links you transit –Routing proceeds hop by hop (contrast with source routing) –How do you determine the next hop? Could configure it statically But the Internet needs a routing system and protocol to exchange complex and changing routing info

External and internal gateway protocols Autonomous System (AS) – a single administrative domain (ISP, customer) External Gateway Protocols exchange routing information between routers of different AS’s. –Goal: support routing policies, scale Internal Gateway Protocols exchange routing information among an AS’s own routers –Goal: optimize route taken

Distance vector protocols messages relative to sender –Essentially a routing table Contrast with link state protocols – messages “flooded” to all nodes –Shortest path first (Dijkstra) algorithm builds routing table

Classless inter-domain routing Internet routing was once based on network classes Trading classes for variable-length prefixes allows aggregation –Greater flexibility in address allocation –Less routing information required

BGP communication A Border Gateway Protocol (BGP) session consists of a TCP connection between two routers –If connection fails, associated state is dropped. Message types: OPEN, UPDATE, NOTIFICATION, KEEPALIVE UPDATE format:

BGP attributes ORIGIN – BGP speaker’s unique identifier AS-PATH – AS’s that relayed this message NEXT-HOP MULTI-EXIT-DISCRIMINATOR (MED) – metric for multiple paths between 2 AS’s LOCAL-PREF – metric for multiple paths to same prefix COMMUNITY – update categorization

Internal-BGP Same messages, attributes as External-BGP Different rules for readvertising prefixes –Does not readvertise routes from one I-BGP speaker to another –Prevents looping (E-BGP uses the ASN and AS-PATH for this) Route reflection: adding hierarchy for scalability AS confederation: subdivision of a logical AS into multiple AS’s

Route Information Propagation In the Internet Using BGP Matthew Denny U.C. Berkeley

Introduction Internet consists of different Autonomous Systems (ASs), which consist of admin. defined domains of hosts (e.g. ISPs, universities, companies, etc.) Hosts in each AS must be able to send packets to any other host on the Internet ASs have routers which exchange routing info. with other ASs using BGP –How do ASs ensure “full reachability” of the Internet, given no central authority? –How well does the current route information propagation scheme perform?

Outline Introduction Structure of ASs BGP policies of ASs Why does it work? Now, does it really work?

Structure of ASs Somewhat hiearchical (but becoming less so) 3 Types of relationships –Customer-Provider: customer AS pays provider AS for access to rest of Internet: provider provides transit service End customers pay ISPs, and ISPs in lower “tiers” pay ISPs in higher tiers –Peers: ASs that allow each other transit service ISPs on same tier, usually involves no fees –Customer-Backup Provider: Provider if primary provider fails. May be peers otherwise Use BGP to communicate route info. at Network Exchange Points (NAPs) and private peering points

AS BGP Policies Customers export all of their routes and routes of their customers to providers, but not routes from peers or other providers Peers export their routes and routes of their customers to other peers, but not routes from peers or other providers Providers export all of their routes to customers Usually, backup providers “promoted” to provider from peer upon failure of primary If an AS recieves 2 routes for same prefix, usually exports the best by some path selection algorithm

AS BGP Policies AS1AS2 AS3AS / / / / / / / / / / / / / / / / / / / /26

AS BGP Policy Details Export Policy –To indicate priority of route, most ASs use communities [Labovitz 2000b] Import Policy –AS Path loop detection, not usually selective. Use communities to infer local pref Path Selection –If AS has multiple routes for same prefix, best route decided by local pref; AS Path and MED are tiebreakers

How does it work? Full Reachability –Provider/Customer relationships form a DAG Assumes everyone below tier 1 has a provider Convergence –Can diverge, and checking for convergence is an NP Complete Problem [Griffin 99] –Assuming strict preferences on route selection and the above structure, [Gao 2000] proves that BGP systems will converge –Will this hold as peering becomes more common?

Now, Does it Really Work? Potential Problems –Route Instability Large number of unneeded messages leads to router CPU flooding –Routers lose Keep-Alive messages go “down” “Route Flap” Problem –Route Convergence Routes that change (e.g. failover to a backup provider) may take a long time to propagate correctly through system Can cause intermittent loss of connectivity

Route Instability Study Labovitz et. al. performed a study to measure instability in BGP Updates [Labovitz 1997, 1999] –Logged BGP update messages at 5 NAPs , and analyzed instability events Routes withdrawn that are re-announced, and “pathological” withdraws Some events due to route or policy instability, or pathological behavior

Initial Instability Findings In 1996, 45,000 prefixes, 1,500 unique AS paths, 1,300 ASs, 3-6 million BGP update messages/day Messages dominated by pathological withdraws Redundant updates have strong periodicity of 60 sec. Redundant Updates correspond with network usage Instability not dominated by small number of ASs or routes

Many Problems due to Router Software Implementation Pathological withdraws due to “stateless BGP” –Announce withdraws to router peers that did not originally receive an announcement Periodicity due to min. advertisement timer that was fixed in one BGP implementation New methods developed to prevent route flaps –BGP messages have higher priority than data, esp Keep-Alives. Labovitz et. al. contacted router vendors, who released patches to fix these bugs

Follow-up Results June 1996, 2M pathological withdrawls/day, 10K in June 1998 –Due to at least partial “stateful BGP” in most routers In 1998, duplicate announcements 40% of traffic –Bug in router software where non-transitive attribute is errantly “propagated” –Min. advertisement timer allows routes to change back to original value before transmission; router still sends these routes In 1998, vast majority of route fluctuation due to MED changes –Come from 2 ISPs that dynamically assign MEDs from IGP. Effectively make IGP changes globally visible

Route Convergence For a set of real host addresses, Labovitz et. al. inject routing events and observe convergence behavior [Labovitz 2000] –Route failures, new routes, and routes with new path –Simulated backup routes by inflating AS path –Analyzed BGP traffic –Tested faults affect on routes by sending ICMP messages to web sites from set of addresses

Convergence Findings Delay in failovers ave. 3 min., but up to 15 min. (more than 30 sec. expected) Messages/event and convergence time/event varies from ISP to ISP Significant increase in packet loss and latency around faults Routers use per peer min. advertisement timers, which delays convergence Wait for min. advertisement timer to send updates that have loops in them; should use sender side loop detection instead

Conclusions Describe structure of ASs in Internet and how they exchange routing information via BGP Discuss work that shows that this structure should work given specific assumptions Discuss work that shows, in practice, Internet had instability and convergence problems, but many of these were due to implementation problems

References [Gao 2000] L. Gao and J, Rexford “Stable Internet Routing Without Global Coordination”, SIGMETRICS 2000 [Labovitz 1997] C. Labovitz, G.R. Malan, F. Jahanian, “Internet Routing Instability”, SIGCOMM 97. [Labovitz 1999] C. Labovitz, G.R. Malan, F. Jahanian, “Origins of Internet Routing Instability”, INFOCOMM 1999 [Labovitz 2000] C. Labovitz, A. Ahuja, A. Bose, F. Jahanian. “Delayed Internet Routing Convergence”, SIGCOMM 1999 [Labovitz 2000b] C. Labovitz, R. Wattenhofer, S. Venkatachary, A. Ahuja. “The Impact of Internet Policy and Topology on Delayed Routing Convergence”. Microsoft Tech. Report MSR-TR , 2000.