Measuring the Autonomous System Path Through the Internet Jennifer Rexford Internet and Networking Systems AT&T Labs - Research; Florham Park, NJ Joint work with Z. Morley Mao, David Johnson, Jia Wang, and Randy Katz

Research Overview: Mature Internet Infrastructure  Managing a single Autonomous System –Measurement of traffic, routing, and configuration –Traffic engineering techniques and tools –Configuration checking and automation  BGP routing between Autonomous Systems –Routing protocol convergence –Inference of commercial relationships –Characterization of routing dynamics  End-to-end troubleshooting of network problems –Intradomain root-cause analysis of routing changes –DARPA Knowledge Plane architecture –Autonomous System traceroute…

IP Forwarding Path Path packets traverse through the Internet  Why important?  Characterize end-to-end network paths  Discover the router-level Internet topology  Detect and diagnose reachability problems IP traffic Internet source destination

Traceroute: Measuring the Forwarding Path  Time-To-Live field in IP packet header –Source sends a packet with a TTL of n –Each router along the path decrements the TTL –“TTL exceeded” sent when TTL reaches 0  Traceroute tool exploits this TTL behavior source destination TTL=1 Time exceeded TTL=2 Send packets with TTL=1, 2, 3, … and record source of “time exceeded” message

Example Traceroute Output (Berkeley to CNN) * * Hop number, IP address, DNS name inr-daedalus-0.CS.Berkeley.EDU soda-cr-1-1-soda-br-6-2 vlan242.inr-202-doecev.Berkeley.EDU gigE6-0-0.inr-666-doecev.Berkeley.EDU qsv-juniper--ucb-gw.calren2.net POS1-0.hsipaccess1.SanJose1.Level3.net ? pos8-0.hsa2.Atlanta2.Level3.net pop2-atm-P0-2.atdn.net ? pop1-atl-P4-0.atdn.net www4.cnn.com No response from router No name resolution

AS A AS B AS C AS D Autonomous System (AS) Autonomous System Forwarding Path Example: Pinpoint forwarding loop & responsible AS IP traffic Internet source destination

Border Gateway Protocol (BGP) BGP path may differ from forwarding AS path –Routing loops and deflections –Route aggregation and filtering –BGP misconfiguration AS AAS BAS C prefix d Signaling path: control traffic d: path=[C] Forwarding path: data traffic d: path=[BC] Origin AS

Map Traceroute Hops to ASes * * Traceroute output: (hop number, IP) AS25 AS11423 AS3356 AS1668 AS5662 Berkeley CNN Calren Level3 AOL Need accurate IP-to-AS mappings (for network equipment).

Candidate Ways to Get IP-to-AS Mapping  Routing address registry –Voluntary public registry such as whois.radb.net –Used by prtraceroute and “NANOG traceroute” –Incomplete and quite out-of-date »Mergers, acquisitions, delegation to customers  Origin AS in BGP paths –Public BGP routing tables such as RouteViews –Used to translate traceroute data to an AS graph –Incomplete and inaccurate… but usually right »Multiple Origin ASes (MOAS), no mapping, wrong mapping

Refining Initial IP-to-AS Mapping  Start with initial IP-to-AS mapping –Mapping from BGP tables is usually correct –Good starting point for computing the mapping  Collect many BGP and traceroute paths –Signaling and forwarding AS path usually match –Good way to identify mistakes in IP-to-AS map  Successively refine the IP-to-AS mapping –Find add/change/delete that makes big difference –Base these “edits” on operational realities

Extra AS due to Internet eXchange Points  IXP: shared place where providers meet –E.g., Mae-East, Mae-West, PAIX –Large number of fan-in and fan-out ASes A B C D E F G Traceroute AS pathBGP AS path Physical topology and BGP session graph do not always match. B C F G AE

Extra AS due to Sibling ASes  Sibling: organizations with multiple ASes: –E.g., Sprint AS 1239 and AS 1791 –AS numbers equipment with addresses of another Traceroute AS pathBGP AS path A B C D E F G H A B C D E F G Sibling ASes “belong together” as if they were one AS.

Weird Paths Due to Unannounced Addresses AB C A C B A C B C C does not announce part of its address space in BGP (e.g., /24) /8 Fix the IP-to-AS map to associate /24 with C

Reasons BGP and Traceroute Paths May Differ  IP-to-AS mapping is inaccurate (fix these!) –Internet eXchange Points (IXPs) –Sibling ASes owned by the same institution –Unannounced infrastructure addresses  Forwarding and signaling paths differ (study these!) –Forwarding loops and deflections –Route aggregation and filtering  Traceroute inaccuracies (don’t overreact to these!) –Forwarding path changing during measurement –Address assignment to border links between ASes –Outgoing link identified in “time exceeded” message

Optimization Framework  Start with initial IP-to-AS map A(x) –IP address x maps to A(x), a set of ASes  Iterative refinement –Apply A(x) to the hops in each traceroute path –Compare the traceroute hops to the BGP AS path –Compute mismatch statistics for each entry x –Modify A(x) depending on a small set of rules  Terminate when no further modifications

Matching Function and Unavoidable Error  Matching function m for BGP/traceroute pair –Traceroute path: t 1, t 2, …, t n of n IP addresses –BGP path: b 1, b 2, …, b l of l AS numbers –Matching: associate IP hop t i with AS hop b m(i)  Find the matching m that minimizes error –Number of traceroute hops with b m(i) not in A(t i ) –Dynamic programming algorithm to find best m t: b: A B C

Rules for Modifying the IP-to-AS Mapping  Computing match statistics across paths –Focusing on path pairs with at most two errors  Example rules –Create a mapping: A(x) is null »Assign to the AS y that appears in the most matchings –Replace a mapping: A(x) has one entry »If an AS y not in A(x) accounts for > 55% of matchings –Delete from a mapping: A(x) has multiple entries »If an AS y in A(x) accounts for < 10% of matchings  Algorithm converges in less than ten iterations

Measurement Data: Eight Vantage Points OrganizationLocationUpstream Provider AT&T ResearchNJ, USUUNET, AT&T UC BerkeleyCA, USQwest, Level3, Internet 2 PSG home networkWA, USSprint, Verio Univ of WashingtonWA, USVerio, Cable&Wireless ArosNetUT, USUUNET NortelON, CanadaAT&T Canada Vineyard.NETMA, USUUNET, Sprint, Level3 Peak Web HostingCA, USLevel 3, Global Crossing, Teleglobe Sweep the routable IP address space –~200,000 IP addresses –160,000 prefixes –15,000 destination ASes

Initial Analysis of BGP and Traceroute Paths  Traceroute paths: initial mapping A from BGP –Unmapped hops: match no ASes (1-3% of paths) –MOAS hops: match any AS in the set (10-13% of paths) –“*” hops: match any AS (7-9% of paths)  BGP paths: discard 1% of prefixes with AS paths –Routing changes based on BGP updates –Private AS numbers (e.g., 65100) –Empty AS paths (local destinations) –Apparent AS-level loops from misconfiguration –AS_SET instead of AS sequence

Comparison of IP-to-AS Mappings  Whois: unmapped hops cause half of mismatches  BGP tables: mostly match, as our algorithm assumes  Refined mapping: change 2.9% of original mapping –Robust to reducing # of probes and introducing noise Whois BGP origins Refined mapping Match47%85%95% Mismatch53%15%5% Ratio Comparing BGP and Traceroute AS paths for various IP-to-AS mappings

Validating the Changes to the Mapping  AT&T’s tier-1 network (AS 7018) –Dump of configuration state from each of the routers –Explains 45 of 54 changes involving AS 7018 »E.g., customer numbered from AT&T addresses »E.g., Internet exchange point where AT&T connects  Whois query on prefix or AS –Look for “exchange point” or “Internet exchange” »Explains 24 of the changes to the mappings –Look for ASes with similar names (Sprintlink vs. Sprintlink3) »Explains many of the changes to the mappings  List of known Internet eXchange Points –Explains 24 of the MOAS inferences –Total of 38 IXPs contributed to mapping changes

Exploring the Remaining Mismatches  Route aggregation –Traceroute AS path longer in 20% of mismatches –Different paths for destinations in same prefix  Interface numbering at AS boundaries –Boundary links numbered from one AS –Verified cases where AT&T (AS 7018) is involved BGP path: B C Traceroute path: B C D B CC D D E E B CB D D BGP path: B C D Traceroute path: B D

Contributions  Problem formulation –AS-level traceroute tool for troubleshooting –Compute an accurate IP-to-AS mapping  Optimization approach –Compute matchings using dynamic programming –Improve mapping through iterative refinement  Measurement methodology –Traceroute and BGP paths from many locations  Validation of our results –Changes to the IP-to-AS mappings –Remaining mismatches between traceroute and BGP

Future Work on AS Traceroute  Lower measurement overhead –Avoid traceroute probes that would discover similar paths –Work with BGP routing tables rather than live feeds  Limiting the effects of traceroute inaccuracies –Catch routing changes through repeat experiments –Use router-level graphs to detect AS boundaries –Detect routers using outgoing link in “time exceeded”  Public AS traceroute tool –Periodic data collection and computation of IP-to-AS mapping –Software to apply mapping to traceroute output  Network troubleshooting –Analyze valid differences between forwarding and signaling paths –Use the AS traceroute tool to detect and characterize anomalies

Longer-Term Research Plans: Mature Infrastructure  Network architecture –Distributed troubleshooting service –Routing “czar” for controlling IP routing  Router and protocol extensions –Extra information to help in troubleshooting –Traffic and routing protocol measurement –Better router configuration languages  Campus, enterprise, and regional networks –Fertile ground for new research problems –New sources of measurement data and “tech transfer”