1. Internet Routing: Measurement, Modeling, and Analysis Dr. Jia Wang jiawang@research.att.com AT&T Labs Research Florham Park, NJ 07932, USA http://www.research.att.com/~jiawang/ Prof. Zhuoqing Morley Mao zmao@umich.edu Department of EECS University of Michigan Ann Arbor, MI 48109, USA http://www.eecs.umich.edu/~zmao/ ACM Sigmetrics 2005 Tutorial

2. 2 Outline 1.Overview of Inter-domain routing 2.Measuring inter-domain paths 3.BGP Measurement 4.BGP Modeling Our opinions should not be taken to represent AT&T policies

3. Part I: Overview of Inter- domain Routing

4. 4 Internet  Loose cooperative effort of Internet Service Providers (ISPs)  E.g., AT&T, Sprint, UUNet, AOL  Best effort service  Connectedness  Anyone connected to the Internet can exchange traffic with anyone else connected to the Internet

5. 5 : Routing session routes Control plane: exchange routes Internet routing rusty.cs.berkeley.edu IP=169.229.62.116 Prefix=169.229.0.0/16 www.cnn.com IP=64.236.16.52 Prefix=64.236.16.0/20 Internet IP traffic Data plane: forward traffic Fail over to alternate route

6. 6 Internet routing domain  Autonomous routing domain  Network devices under same technical and administrative control  Common routing policy  E.g., ISPs, enterprise networks  Autonomous system  Autonomous routing domain with an AS number (ASN)  AS numbers: 16 bits integer  Public AS number: 1 – 64511  Private AS number: 64512 – 65535  Examples  AT&T: 7018, 6431, …  Sprint: 1239, 1240, …  MIT: 3

7. 7 More than 20,000 ASes today Berkeley Internet CNN Calren Level3 GNN IP traffic QwestSprintUUnet University company AT&T business ISP Autonomous System Berkeley Calren Level3 QwestSprintUUnet University company AT&T business ISP Berkeley Calren Level3 QwestSprintUUnet University company AT&T business ISP

8. 8 Internet routing architecture IP traffic Berkeley CNN Level3 Internet CalrenGNN Inter-domain routing Intra-domain routing

9. 9  Run within a certain network infrastructure  Optimize routes taken between points within a network  Internal Gateway Protocols (IGPs)  Metrics based  OSPF (Open Shortest Path First)  RIP (Routing Information Protocol)  IS-IS (Intermediate System to Intermediate System)

10. 10 Inter-domain routing  Run between networks  Provide full connectivity of entire Internet  External Gateway Protocol (EGP)  Policy based  BGP (Border Gateway Protocol)

11. 11 Link state protocols  Examples: OSPF, IS-IS  Based on Dijkstra’s shortest path computation  Each router periodically floods immediate reachability information to other routers  Fast convergence  High communication and computation overhead  Not scalable for large networks  Requires periodic refreshes

12. 12 Vectoring protocols  Distance vs. Path Vector  Distance: hop count (RIP)  Path: entire path (BGP)  Helps identify loops  Supports policy-based routing based on path  Minimal communication overhead  Takes longer to converge, i.e., in proportion to the maximum path length

13. 13 Link state vs. vectoring OSPF IS-IS RIP BGP IGP EGP Link stateVectoring BGP is a path vector protocol

14. 14 Classful addressing  IPv4: 32 bits  Five classes of networks ClassAddressMask# of networks# of hosts A0*255.0.0.0128~1.6M B10*255.255.0.01638465535 C110*255.255.255.0~2.1M255 DUsed for multicast EReserved and currently unused Improve scaling factor of routing in the Internet => classless

15. 15 CIDR: Classless Inter-domain Routing (RFC1519)  No implicit mask based on the class of the network  Explicit masks passed in the routing protocol  Allow aggregation and hierarchical routing 00001100 00100110 00000000 00000000 11111111 11111111 11000000 00000000 IP address: 12.70.0.0Mask: 255.255.252.0 CIDR representation: 12.70.0.0/22 Address Mask Network prefix Host identifier 00001100 00100110 00000000 00000000 11111111 11111111 11000000 00000000

16. 16 Address aggregation Internet 12.70.1.0/24 12.70.2.0/24 12.70.3.0/24 12.70.0.0/24 ISP A ISP B 12.70.0.0/22 12.71.0.0/16

17. 17 Routing and forwarding  Routing  The decision process of choosing optimal path that is consistent with the administrative or technical policy  Forwarding  The act of receiving a packet, doing a lookup, and copying a packet to the next hop

18. 18 Classless forwarding Internet 135.120.0.1 12.70.0.20 IP traffic PrefixNext hop 12.70.0.0/2410.20.0.1 12.70.0.0/1610.20.1.1 12.0.0.0/810.20.128.1 0.0.0.0 10.20.128.10 10.20.0.1 10.20.1.1 10.20.128.1 10.20.128.10

19. 19 Inter-domain routing with CIDR support  BGP-4 [RFC1771]  De facto EGP  Carry routing information between ASes  Path vector protocol  Policy based routing  Run on top of TCP for reliability  Basic operations  Set up BGP session  Exchange all candidate routes  Send incremental updates

20. 20 Establish BGP session 12.10.0.112.10.0.2 Establish neighboring session between 12.10.0.1 and 12.10.0.2 PrefixNext hop 12.70.0.0/2410.20.0.1 12.9.0.0/1610.20.1.1 PrefixNext hop 135.120.0.0/2410.128.0.1 68.35.0.0/1610.192.1.1 TCP 179

21. 21 Exchange all candidate routes 12.10.0.112.10.0.2 PrefixNext hop 12.70.0.0/2410.20.0.1 12.9.0.0/1610.20.1.1 135.120.0.0/2410.128.0.1 68.35.0.0/1610.192.1.1 PrefixNext hop 135.120.0.0/2410.128.0.1 68.35.0.0/1610.192.1.1 12.70.0.0/2410.20.0.1 12.9.0.0/1610.20.1.1 12.70.0.0/2410.20.0.1 12.9.0.0/1610.20.1.1 135.120.0.0/2410.128.0.1 68.35.0.0/1610.192.1.1

22. 22 Send incremental updates 12.10.0.112.10.0.2 PrefixNext hop 12.70.0.0/2410.20.0.1 12.9.0.0/1610.20.1.1 135.120.0.0/2410.128.0.1 68.35.0.0/1610.192.1.1 PrefixNext hop 135.120.0.0/2410.128.0.1 68.35.0.0/1610.192.1.1 12.70.0.0/2410.20.0.1 12.9.0.0/1610.20.1.1 Withdraw 12.9.0.0/16

23. 23 BGP messages  OPEN: set up a peering session  UPDATE: announce new routes or withdraw previously announced routes  NOTIFICATION: shut down a peering session  KEEPALIVE: confirm active connection at regular interval

24. 24 Internal vs. external BGP Internet I-BGP E-BGP AS A AS B AS C E-BGP update I-BGP update I-BGP update

25. 25 Scaling I-BGP for large AS  Route reflectors  Confederations E-BGP update RR Only best paths being sent by RR AS 1000 EBGP IBGP AS 65010 AS 65020

26. 26 Establish connectivity 135.120.0.0/16 12.10.0.1 12.10.0.2 PrefixNext hop AS path 135.120.0.0/1612.10.0.11 EBGP IBGP EBGP 12.10.0.5 12.10.0.6 AS 1 AS 2 AS 3 PrefixNext hop AS path 135.120.0.0/1612.10.0.52 1 PrefixNext hop AS path 135.120.0.0/1612.10.0.11

27. 27 IGP and BGP working together 135.120.0.0/16 12.10.0.1 12.10.0.2 PrefixNext hop AS path 135.120.0.0/1612.10.0.11 EBGP IBGP EBGP 12.10.0.5 12.10.0.6 AS 1 AS 2 AS 3 PrefixNext hop AS path 135.120.0.0/1612.10.0.11 10.10.0.1 PrefixNext hop 12.10.0.0/3010.10.0.1 135.120.0.0/1610.10.0.1 12.10.0.0/30

28. 28 Policy routing ISP1 ISP4ISP3 Cust1Cust2 ISP2 traffic Connectivity DOES NOT imply reachability! Policy determines how traffic can flow on the Internet

29. 29 BGP routing process Apply input policy Routes received from peers Select best route Best routes Apply output policy Routes advised to peers Routing table Forwarding table BGP is not shortest path routing!

30. 30 Best route selection  Highest local preference  Shortest AS path  Lowest MED (Multi-Exit-Discriminator)  I-BGP < E-BGP  Lowest I-BGP cost to E-BGP egress  Tie breaking rules

31. 31 Best route selection  Highest local preference  To enforce economical relationships between domains  Shortest AS path  Lowest MED (Multi-Exit-Discriminator)  I-BGP < E-BGP  Lowest I-BGP cost to E-BGP egress  Tie breaking rules

32. 32 Best route selection  Highest local preference  Shortest AS path  Compare the quality of routes, assuming shorter AS-path length is better  Lowest MED (Multi-Exit-Discriminator)  I-BGP < E-BGP  Lowest I-BGP cost to E-BGP egress  Tie breaking rules

33. 33 Best route selection  Highest local preference  Shortest AS path  Lowest MED (Multi-Exit-Discriminator)  To implement “cold potato” routing between neighboring domains  I-BGP < E-BGP  Lowest I-BGP cost to E-BGP egress  Tie breaking rules

34. 34 Best route selection  Highest local preference  Shortest AS path  Lowest MED (Multi-Exit-Discriminator)  I-BGP < E-BGP  Prefer EBGP routes to IBGP routes  Lowest I-BGP cost to E-BGP egress  Tie breaking rules

35. 35 Best route selection  Highest local preference  Shortest AS path  Lowest MED (Multi-Exit-Discriminator)  I-BGP < E-BGP  Lowest I-BGP cost to E-BGP egress  Prefer routes via the nearest IGP neighbor  To implement “hot potato” routing  Tie breaking rules

36. 36 Best route selection  Highest local preference  Shortest AS path  Lowest MED (Multi-Exit-Discriminator)  I-BGP < E-BGP  Lowest I-BGP cost to E-BGP egress  Tie breaking rules  Router ID based: lowest router ID  Age based: oldest route

37. 37 BGP route propagation  Not all possible routes propagate  Commercial relationships determine policies for  Route import  Route selection  Route export

38. 38 Typical AS relationships  Provider-customer  customer pay money for transit  Peer-peer  typically exchange respective customers’ traffic for free  Siblings  Mutual transit agreement  Provide connectivity to the rest of the Internet for each other

39. 39 AS relationships translate into BGP export rules  Export to a provider or a peer  Allowed: its routes and routes of its customers and siblings  Disallowed: routes learned from other providers or peers  Export to a customer or a sibling  Allowed: its routes, the routes of its customers and siblings, and routes learned from its providers and peers

40. 40 Which AS paths are legal?  Valley-free:  After traversing a provider-customer or peer-peer edge, cannot traverse a customer-provider or peer-peer edge  Invalid path: >= 2 peer links, downhill- uphill, downhill-peer, peer-uphill

41. 41 Example of valley-free paths X X [1 2 3], [1 2 6 3] are valley-free [1 4 3], [1 4 5 3] are not valley free

42. 42 Inferring AS relationships  Identify the AS-level hierarchy of Internet  Not shortest path routing  Predict AS-level paths  Traffic engineering  Understand the Internet better  Correlate with and interpret BGP update  Identify BGP misconfigurations  E.g., errors in BGP export rules

43. 43 Existing approaches  On inferring Autonomous Systems Relationships in the Internet, by L. Gao, IEEE Global Internet, 2000.  Characterizing the Internet hierarchy from multiple vantage points, by L. Subramanian, S. Agarwal, J. Rexford, and R. Katz, IEEE Infocom, 2002.  Computing the Types of the Relationships between Autonomous Systems, by G. Battista, M. Patrignani, and M. Pizzonia, IEEE Infocom, 2003.  On AS-level Path Inference, by Z. Mao, L. Qiu, J. Wang, and Y. Zhang, ACM Sigmetrics, 2005.

44. 44 Policy routing causes path inflation  End-to-end paths are significantly longer than necessary  Why?  Topology and routing policy choices within an ISP, between pairs of ISPs, and across the global Internet  Peering policies and interdomain routing lead to significant inflation  Interdomain path inflation is due to lack of BGP policy to provide convenient engineering of good paths across ISPs

45. 45 Path inflation  Based on [Mahajan03]  Comparing actual Internet paths with hypothetical “direct” link