1. Routing Policy CS 6250 Nick Feamster Fall 2011

2. BGP Policies in ISP Networks Introduced as fairly simple path vector protocol Many incremental modifications added over time Many policies used by operators; cant enumerate them all –Nevertheless, some patterns emerge 2

3. How is Policy Implemented? Preference: Which route will be chosen for each destination? –Adding/deleting/modifying route attributes Filtering: Eliminates certain routes from consideration –Can be done on inbound or outbound Tagging: Adding state to a route –Tagging with community attribute 3

4. 4 Filtering and Rankings Ranking: route selection Filtering: route advertisement Customer Competitor Primary Backup

5. 5 Internet Business Relationships(Simplified) Customer/Provider: One AS pays another for reachability to some set of destinations Settlement-free Peering: Bartering. Two ASes exchange routes with one another. Provider Peer Customer Preferences implemented with local preference manipulation Destination Pay to use Get paid to use Free to use

6. 6 Rankings Routes from customers over routes from peers Routes from peers over routes from providers provider peer customer

7. Traffic Engineering Outbound traffic control: Control the way traffic leaves the network –Via local preference and IGP costs Inbound traffic control: Control the way traffic enters the network –Via AS prepending and MED Remote control: Control the preferences of a remote AS –Through the community attribute 7

8. 8 Policy Interactions 1 23 1 3 0 1 0 3 2 0 3 0 2 1 0 2 0 0 Varadhan, Govindan, & Estrin, Persistent Route Oscillations in Interdomain Routing, 1996

9. 9 Strawman: Global Policy Check Require each AS to publish its policies Detect and resolve conflicts Problems: ASes typically unwilling to reveal policies Checking for convergence is NP-complete Failures may still cause oscillations

10. 10 Think Globally, Act Locally Key features of a good solution –Safety: guaranteed convergence –Expressiveness: allow diverse policies for each AS –Autonomy: do not require revelation/coordination –Backwards-compatibility: no changes to BGP Local restrictions on configuration semantics –Ranking –Filtering

11. 11 Can BGP Be Made Stable? Permit only two business arrangements –Customer-provider –Peering Constrain both filtering and ranking based on these arrangements to guarantee safety Surprising result: these arrangements correspond to todays (common) behavior Gao & Rexford, Stable Internet Routing without Global Coordination, IEEE/ACM ToN, 2001

12. 12 Relationship #1: Customer-Provider Filtering –Routes from customer: to everyone –Routes from provider: only to customers providers customer From the customer To other destinations advertisements traffic From other destinations To the customer customer providers

13. 13 Relationship #2: Peering Filtering –Routes from peer: only to customers –No routes from other peers or providers advertisements traffic customer peer

14. How Do Business Relationships Appear in Policy? Local preference: influence the decision process to prefer customer routes Controlling route export: prevent neighbors from sending traffic to some destinations Defensive programming: prevent neighboring AS from influencing its choice of routes 14

15. Physical Interconnections Public peering (IXP): Across a layer-2 access technology (typically, port on shared fabric). –Larger number of smaller peers –Trial peering Private peering: Direct interconnection between only two networks. –Previously: Circuit between two facilities –Today: Typically occur at carrier hotels 15

16. 16

17. The Art of Peering: The Peering Playbook William B. Norton Co-Founder & Chief Technical Liaison Equinix, Inc. NaMeX Member Meeting October 7, 2005 Rome, Italy

18. Research The Art of Peering Follow up to the first three white papers. Q: When e-mail to peering@.net generates no response, what do Seasoned Peering Coordinators do? Smartest Peering Coordinator: Tricks of the Trade 20 Tactics successfully used to obtain Peering where you otherwise might not be able to. Disclaimer: These are NOT recommended tactics…I am simply documenting what has been successfully used in the field to obtain peering.

19. P? Graphical Notation of Tactics BA ISP InitiatorISP Target AB ISP A Customers ISP B Customers Larger Circle=More Customer Prefixes Thicker Lines=More Traffic P?=Peering Request w/ Peering Coordinator Peering Negotiation T?=Transit Request To Sales Person Transit Negotiations A PC B PC P? A PC BSBS T? A PC B PC A PC BSBS T? $ $ To Portray Peering Plays Pictorially…

20. Transit and Peering Sessions AB T AB P T=Established Transit Session (Selling Access to entire Internet) Size indicates effective size of transport Supporting the session P=Established Peering Session (Reciprocal Access to each others customers) Size indicates effective size of transport Supporting the session AB P AB T $ $ Graphical Display of Routing Announcements Represents the rest of the Internet

21. Traffic over Transit and Peering Sessions AB T AB P Traffic showed as directed lines Thickness of line indicates amount of Traffic in relevant direction AB P AB T $ $ Other Variations P->T = Transition of Relationship P | T = Either Peering or Transit apply = Traffic destined anywhere = Fictitious Traffic = Packet Loss ridden Traffic = Traffic destined to green network = Traffic destined to brown network

22. Other Graphical Symbols Peering Point Exchange Point, Telco Hotel Tied with Indicates two or more Elements tied with relationship Indicates a ordering: a sequence to be followed in the Peering Tactic or

23. 1) The Direct Approach uses peering@.net, phone calls, face to face meetings, or otherwise direct interactions with Peering Coordinators to establish peering. P? P?=Peering Request To Peering Coordinator(s) Peering Negotiation Leading to Peering Session A PC B PC P? A PC B PC AB P {null} -or- {No,null}

24. 2) The Transit with Peering Migration tactic leverages an internal advocate to buy transit with a contractual migration to peering at a later time. A PC BSBS T? A PC BSBS T? A PC B S+ B PC T->P? AB T->P $ AB P Transit Negotiations with Sales leads to Peering (…if peering prerequisites be met…) $ $ $

25. 3) The End Run Tactic minimizes the need for transit by enticing a direct relationship with the target ISPs largest traffic volume customers. A PC B T?/P? $

26. 4) In Europe the Dual Transit/Peering separates the peering traffic from the transit traffic using separate interface cards and/or routers. AB P T$

27. A 5) Purchasing Transit *Only* from Large Tier 2 ISPs is an approach to reduce the risk of being a customer of a potential peer on the road to Tier 1 status. AB T XYZ P Tier 1 ISPs (who do not buy Transit) Tier 2 ISPs (who buy Transit) P? $ Since A peering with any Tier 1 wont affect transit revenue with Tier 1s There is no financial consequence to peering. (One less barrier to overcome during peering negotiations.)

28. 6) Paid Peering as a maneuver is positioned by some as a stepping stone to peering for those who dont immediately meet the peering prerequisites. AB P AB P $

29. Peering Point 7) In the Partial Transit tactic, the routes learned at an exchange point are exchanged with the peer for a price slightly higher than transport costs. $ AB E G D C F H I J K L M Routing Announcements Forwarding all customer & Peering Pt Routes (almost peering – maybe costs less) Geographically Remote Router pt

30. 8) The Chicken tactic involves de-peering in order to make the other peer adjust the relationship. AB P AB P Who will blink first? A B Traffic has to go somewhere YX T T $ $ AB T or $

31. 31 The Business Game and Depeering Cooperative competition (brinksmanship) Much more desirable to have your peers customers –Much nicer to get paid for transit Peering tiffs are relatively common 31 Jul 2005: Level 3 Notifies Cogent of intent to disconnect. 16 Aug 2005: Cogent begins massive sales effort and mentions a 15 Sept. expected depeering date. 31 Aug 2005: Level 3 Notifies Cogent again of intent to disconnect (according to Level 3) 5 Oct 2005 9:50 UTC: Level 3 disconnects Cogent. Mass hysteria ensues up to, and including policymakers in Washington, D.C. 7 Oct 2005: Level 3 reconnects Cogent During the outage, Level 3 and Cogents singly homed customers could not reach each other. (~ 4% of the Internets prefixes were isolated from each other)

32. 32 Depeering Continued Resolution… …but not before an attempt to steal customers! As of 5:30 am EDT, October 5th, Level(3) terminated peering with Cogent without cause (as permitted under its peering agreement with Cogent) even though both Cogent and Level(3) remained in full compliance with the previously existing interconnection agreement. Cogent has left the peering circuits open in the hope that Level(3) will change its mind and allow traffic to be exchanged between our networks. We are extending a special offering to single homed Level 3 customers. Cogent will offer any Level 3 customer, who is single homed to the Level 3 network on the date of this notice, one year of full Internet transit free of charge at the same bandwidth currently being supplied by Level 3. Cogent will provide this connectivity in over 1,000 locations throughout North America and Europe.

33. http://66.249.93.104/search?q=cache:NOS3HJhX9jcJ:www.merit.edu/mail.archives/nanog/msg11606.html+level3 +network+status+cogent&hl=it

35. What tactic is this?

36. 36

37. AB L G P T T T $ $ $ A forces traffic Over Bs transit 9) In the Traffic Manipulation tactic, ISPs or content players force traffic along the network path that makes peering appear most cost effective. A PC B PC L G P T T T $ $ $ P? 1 MONTH LATER Contact PC-We should Peer! B hears As route for free through Peer L CH ISP i.e.Yahoo! AB L G P T T T $ $ $

38. $ $ $ A PC B PC L G P T T T AB L G P T T T AB L G P T T T 9b) For Access Heavy Guys…In the Traffic Manipulation tactic, Access ISP a) stop announcing routes, or b) insert Target AS# into announcement to trigger BGP Loop Suppression to force traffic along the network path that makes peering appear most cost effective. $ $ $ $ $ $ P? 1 MONTH LATER Contact PC-We should Peer! B hears As route for free through Peer L A forces traffic Over Bs transit Access ISP i.e. Verizon

39. 10) The Bluff maneuver is simply overstating future traffic volumes or performance issues to make peering appear more attractive. AB P? ABLG T P T $ $ Fictitious Performance Problems Overstating Traffic FuturesYou better peer with me now cause… Lots of transit fees coming otherwise!

40. 11) The Wide Scale Open Peering Policy as a tactic signals to the Peering Coordinator Community the willingness to peer and therefore increases the likelihood of being contacted for peering by other ISPs. A PC P? From the highest mountain We will Peer with Anyone! To anyone who will listen!

41. 12) The Massive Colo Build tactic seeks to meet the collocation prerequisites of as many ISPs as possible by building POPs into as many exchange points as possible. A IX A A A A A A A A A A A A A A Meet us in 3 Time Zones Pacific TimeZone Eastern TimeZone M C

42. 13) The Aggressive Traffic Buildup tactic increases the traffic volume by large scale market and therefore traffic capture to make peering more attractive. AB P? Cheap Transit for sale $20/Mbps!

43. 14) Friendship-based Peering leverages contacts in the industry to speed along and obtain peering where the process may not be in place for a peering. A PC P? B PC Forums to meet Peering Coordinators GPF NANOG APRICOT RIPE IETF :

44. 15) The Spam Peering Requests tactic is a specific case of the Wide Scale Open Peering tactic using the exchange point contact lists to initiate peering. A PC P? IX Participants List ::::

45. 16) The Honey Approach Easier to lure flies with honey…than with Vinegar Publicly promote the attractiveness of Peering with the candidate. Example: Yahoo! –Policy=Yes, millions of streaming hours Example: Rogers –650K Internet subs, 2.3M cable subs –Largest Cable company in Canada

46. 17) Purchasing Legacy Peering provides an immediate set of peering partners. GAGAU P PAPAB P AGU P PB P A A Purchases G and P A Legacy (early Internet day) Peering

47. 18) The Bait and Switch tactic leverages a large corporate identity to obtain peering even though ultimately only a small subset or unrelated set of routes are actually announced. AB P? AB AB P a New Startup Subsidiary

48. 19) The False Peering Outage tactic involves deceiving an ill- equipped NOC into believing a non-existing peering session is down. A NOC B NOC Peering Point X A NOC : Hey – Emergency! A NOC : Our Peering Session with you Went Down! B NOC : Strange. I dont see it configured. A NOC : It was. Dont make me escalate to B NOC : Ah – I bet is was that last config run that trashed it. B NOC : Give me a few minutes to fix it on both ends.

49. 20) The Leverage Broader Business Arrangement takes advantage of other aspects of the relationship between two companies to obtain peering in exchange for something else. A PC BSBS P? AB P AB Other A PC BSBS P? Peering Tied with Other +Fiber deal +Dial-in deal +Racks +Transport +Strategic deal : Tied with

50. 50 Additional Assumption: Hierarchy Disallowed!

51. 51 Safety: Proof Sketch System state: the current route at each AS Activation sequence: revisit some routers selection based on those of neighboring ASes

52. 52 Activation Sequence: Intuition Activation: emulates a message ordering –Activated router has received and processed all messages corresponding to the system state Fair activation: all routers receive and process outstanding messages

53. 53 Safety: Proof Sketch State: the current route at each AS Activation sequence: revisit some routers selection based on those of neighboring ASes Goal: find an activation sequence that leads to a stable state Safety: satisfied if that activation sequence is contained within any fair activation sequence

54. 54 Proof, Step 1: Customer Routes Activate ASes from customer to provider –AS picks a customer route if one exists –Decision of one AS cannot cause an earlier AS to change its mind An AS picks a customer route when one exists

55. 55 Proof, Step 2: Peer & Provider Routes Activate remaining ASes from provider to customer –Decision of one Step-2 AS cannot cause an earlier Step- 2 AS to change its mind –Decision of Step-2 AS cannot affect a Step-1 AS AS picks a peer or provider route when no customer route is available