Internet Routing (COS 598A) Today: Multi-Homing Jennifer Rexford Tuesdays/Thursdays 11:00am-12:20pm

Outline Multi-homing –Motivations: reliability, performance, and financial –Do you really need to use a routing protocol? Controlling outbound traffic –Shortest-path routing –Primary and backup providers –Load balancing over multiple links Controlling inbound traffic –Primary and backup providers –Selective advertising –BGP communities State-of-the-art today

Why Connect to Multiple Providers? Reliability –Reduced fate sharing –Survive ISP failure Performance –Multiple paths –Select the best Financial –Leverage through competition –Game 95 th -percentile billing model Provider 1 Provider 2

The Stub AS Doesn’t Need to Speak BGP… Sending traffic –Assume both providers can reach everyone –Split traffic however you want (e.g., 50%/50%) –But… what if a provider can’t reach someone? –But… what if one provider has a better path? Provider 1 Provider 2 L1 L /0  L1, L2 One static route

The Stub AS Doesn’t Need to Speak BGP… Receiving traffic –Both providers can announce the prefix into BGP –Ensures that everyone else can reach you –But… what if traffic load is very uneven? Provider 1 Provider /24 Advertise /24 traffic

Controlling Outbound Traffic

Outbound Traffic: Pick a BGP Route Easier to control than inbound traffic –IP routing is destination based –Sender determines where the packets go Control only by selecting the next hop –Border router can pick the next-hop AS –Cannot control selection of the entire path Provider 1 Provider 2 “(2, 7, 8, 4)” “(1, 3, 4)”

Outbound Traffic: Shortest AS Path No import policy on border router –Pick route with shortest AS path –Arbitrary tie break (e.g., smallest router-id) Performance? –Shortest AS path is not necessarily best –Could have long propagation delay or congestion Load balancing? –Could lead to uneven split in traffic –E.g., one provider with shorter paths –E.g., too many ties with skewed tie-break

Outbound Traffic: Primary and Backup Single policy for all prefixes –High local-pref for session to primary provider –Low load-pref for session to backup provider Outcome of BGP decision process –Choose the primary provider whenever possible –Use the backup provider when necessary But… –What if you want to balance traffic load? –What if you want to select better paths?

Outbound Traffic: Load Balancing Selectively use each provider –Assign local-pref across destination prefixes –Change the local-pref assignments over time Useful inputs to load balancing –End-to-end path performance data E.g., active measurements along each path –Outbound traffic statistics per destination prefix E.g., packet monitors or router-level support –Link capacity to each provider –Billing model of each provider

Outbound Traffic: Load, Performance, and Cost Balance traffic based on link capacity –Measure outbound traffic per prefix –Select provider per prefix for even load splitting –But, might lead to poor performance and high bill Balance traffic based on performance –Select provider with best performance per prefix –But, might lead to congestion and a high bill Balance traffic based on financial cost –Select provider per prefix over time to minimize the total financial cost –But, might lead to bad performance

Outbound Traffic: What Kind of Probing? Lots of options –HTTP transfer –UDP traffic –TCP traffic –Traceroute –Ping Pros and cons for each –Accuracy –Overhead –Dropped by routers –Sets off intrusion detection systems

Outbound Traffic: Getting Probes on Paths Problem –Router selects one path per prefix –How to measure the alternate paths? Solution #1: special sources (source routing) –Special IP addresses for probe traffic –Router configured to forward probe traffic Solution #2: special destinations –Special destination servers in various locations –At least one destination per provider AS –Probe traffic sent to each destination

Outbound Traffic: How Much Probing? How often? –Continuously, at some rate –In response to a perceived problem How diverse of destinations? –Per destination prefix –Just for popular/important prefixes –Select servers throughout the Internet

Outbound Traffic: How Often to Change Routes? ASes with downstream customers –Each change leads to BGP updates –If not, then no new BGP updates occur TCP flows that switch paths –Out-of-order packets during transition –Change in round-trip-time (RTT) Impact on the providers –Uncertainty in the offered load –Interaction with their own traffic engineering? Impact on other end users –Good: move traffic off of congested paths –Bad: potential oscillation as stub ASes adapt?

Controlling Inbound Traffic

Inbound Traffic: Influencing What Others Do Harder to control than outbound traffic –IP routing is destination based –Sender determines where the packets go Control only by influencing others’ decisions –Static configuration of the providers –BGP route attributes sent by the stub –Selective advertising of destination prefixes Provider 1 Provider 2

Inbound Traffic: Primary and Backup Providers Ask your provider to be a backup –Provider violates “prefer customer” policy –… by assigning lower local-pref to customer –Backup link is only used if the primary link fails Provider 1 Provider /24 local-pref=90 local-pref=100 traffic

Inbound Traffic: AS Prepending Make one path look longer –Advertise short path one way –… and longer path another –In the hope of influencing choices –But, how much prepending to do? Provider 1 Provider 2 “ : (3)” “ : (3, 3, 3)”

Inbound Traffic: Prepending and Prefer-Cust Example where prepending doesn’t work –Customer does prepending of AS path –Provider has a “prefer customer” policy –Provider 2 prefers the longer path Provider 1 Provider 2 “ : (3)” “ : (3, 3, 3)” “ : (1, 3)”

Inbound Traffic: Programming Your Provider Better to have selective control over provider –Tell the provider whether to prefer your route –… on a per-prefix basis, with changes over time –Enables adaptive load balancing –… without asking provider to reconfigure policy Provider 1 Provider /24

Inbound Traffic: RFC 1998 on BGP Communities Provider and customer agree on a “tag” –One tag mean “primary” and the other “backup” –Customer includes tags in BGP advertisements –Provider sets local preference based on tags BGP community attribute –Opaque attribute with no real meaning Two numbers: usually AS number and arbitrary number –Sprint example ( 1239:70 means “assign local pref of 70” … 1239:110 means “assign local pref of 110”

Example: Tier-1 ISP Setting Local-Preference Customers –110: Primary path –100: Secondary path –80: Primary backup path –70: Secondary backup path Peers –81-99: In between –Range for traffic engineering Peer Customer

Inbound Traffic: Not Enough Prefixes Stub ASes usually have only a few prefixes –E.g., one prefix, or at most a handful –Not enough granularity to control traffic Solutions: advertise smaller subnets of prefix –Essentially, create a bunch of smaller prefixes –And apply the load-balancing techniques Advertise selectively AS prepending Communities to set local-pref …

Inbound Traffic: Selective Advertising Divide the destination prefix –Advertise one subnet to each provider –Advertise the supernet to both providers –Traffic splits due to the longest-prefix match –Supernet ensures backup connectivity after failure Provider 1 Provider / / / /17

Inbound Traffic: Small Subnets, Big Debate The players –Stub ASes want more control Advertise smaller subnets –ISPs want to limit table size Filter BGP advertisements for small blocks –ARIN/RIPE/APNIC Publish guidelines for acceptable block sizes Problems –ISPs not getting paid for their routing tables –Risk of network crashes when memory is full –Risk of black-holing a small subnet you filter

Project Ideas Intelligent route-control techniques –Survey approaches to measuring performance –Evaluation of different measurement approaches Techniques for controlling inbound traffic –Negotiation scheme between ASes –Economic approaches for balancing the tension between fine-grain control and table size Source routing –Scalable techniques for stub AS to pick the end-to- end route (not just the next-hop AS)

Next Class: Convergence Delay Two papers, intradomain and interdomain –“Analysis of Link Failures in an IP Backbone” –“Delayed Internet Routing Convergence” Reviews of both papers –Summary –Why accept? –Why reject? –New research directions Optional NANOG video –“Toward Millisecond IGP Convergence”