Instability of BGP ASPP Supervised by Prof. Chiu and Prof. John Presented by Hui Wang

Outline  Introduction to Routing and BGP  AS Relationship and Routing Policy  Inbound Traffic Engineering and ASPP  ASPP at Different Level (my work)  Instability of BGP ASPP (my work)  Future Work  Conclusion

Routing In Internet  Intradomain vs. Interdomain Routing Intradomain:  Router only knows the topology of its domain.  Three types of IGP (Interior Gateway Protocol) Static routing: Only useful in very small domains Distance vector routing : RIP (used in small domain) Link-state routing:  OSPF (enterprise networks)  Intermediate System- Intermediate-System (IS- IS): (widely used by ISPs)

Routing In Internet  Intradomain vs. Interdomain Routing Interdomain Routing (EGP)  Router knows the domain topology of Internet  But every domain is a blackbox for EGP  EGP should be scalable (13,000 AS, 120,000 routes)  Border Gateway protocol Designed in early 1990s Current version 4: 1995

Introduction: BGP Basics  The two variants of BGP eBGP: between border routers of distinct AS Used to distribute the interdomain routes to all border routers over eBGP sessions. iBGP: between BGP routers inside AS Used to distribute the interdomain routes to all routers in same AS over iBGP sessions. In small networks, these iBGP sessions are established in a fullmesh. In larger networks, this full-mesh is replaced by the utilization of route reflectors or confederations.

Introduction: BGP Basics  An Example: eBGP vs. iBGP R1R2 R3R4 R5R6 iBGP eBGP

Introduction: BGP Basics  Autonomous Systems (AS) An AS is defined as a set of routers under a single technical administration. It means that an AS presents a consistent picture of what destinations are reachable through it. Each AS is identified by its AS number (16 bits) The private AS numbers (range through 65535) are reserved for private use and should not be advertised on the global Internet. (currently, more than ASes) A domain is often equivalent to an AS A domain may be composed of several ASes Many domains do not have an AS number

Introduction: BGP Principle  Based on TCP  Distance vector protocol - BGP router advertises its best route to each neighbor - Advertisements are only sent when their routes change (No periodic advertisement of routes as with RIP)  Five Types of Message - Update - Withdrawal - Control Message : Open, Notification and Keepalive

Introduction: BGP Message  Some Fields in BGP Update Message IP prefixes :List of reachable IP prefixes AS-PATH: the list of AS through which the announcement passed NEXT-HOP: the IP address of the router that advertised the route LOCAL-PREF: can be used for traffic control purposes MED (Multi-Exit-Discriminator): used for traffic control between neighbors ORIGIN: how the route was learned (IGP, EGP, Incomplete) ATOMIC AGGREGATE and AGGREGATOR

Introduction: BGP Message  An Example AS1 AS2 If AS1 send Update Message to AS2: /16, AS1, (AS3,AS1) AS3 Prefix: /16 It means AS1 says: I can reach this prefix through AS3, if you have traffic to this prefix, you can send to AS1. AS4

Introduction: BGP Feature  An important feature: BGP allows each AS to define its own routing policy... But how does BGP support different policies?

Introduction: BGP Router Organization  Three tables and two filters BGP RIB Import Filter Export Filter Import PolicyExport Policy Receive routes from neighbors Determines which BGP Msgs are acceptable from neighbors. (maybe assign local-pref to each route) All acceptable routes BGP decision process selects the best route towards each destination 1.Prefer routes with highest local-pref 2. shortest ASPath 3. with the lowest ORIGIN attribute 3. smallest MED 4.Prefer routes learned via eBGP over via iBGP 5. closest next-hop 6. learned from router with lowest router id best route to each destination Only send to some (not all) destination

Introduction: BGP Feature  Why BGP want to support different routing policies? Provider Customer AS2 AS1 AS1 should announce AS2 the route to other ASes AS2 can choose to not announce AS1 the route to AS3 AS3 It is because BGP is a routing protocol between ASes. Different ASes have different commercial interests. The routing policy is constrained by commercial agreements. Customer AS2 AS1 AS3

Summary: Introduction to BGP  Routing in Internet Intradomain vs. Interdomain  BGP Basics eBGP vs. iBGP AS vs. Domain  BGP Principle TCP Distance Vector  BGP Message Five types of Msgs Some Fields (attribute)  BGP Feature Support Different Routing Policy  BGP Router Organization Three Tables

 I mentioned that BGP want to support different routing policies because there are different agreements between two ASes. Now: How the agreements (AS relationship) affect the routing policy?

Outline  Introduction to Routing and BGP  AS Relationship and Routing Policy  Inbound Traffic Engineering and ASPP  ASPP at Different Level (my work)  Instability of BGP ASPP (my work)  Future Work  Conclusion

AS Relationship:  Agreement = Relationship  In practice: Classify AS relationship into Provider to Customer Peer to Peer Provider to Customer: Customer c buys Internet connectivity from provider P. Peer to Peer: AS x and y agree to exchange traffic between their customers free of charge.

Routing Policy  Two kinds of Routing Policy Export Routing Policy vs. Import Routing Policy Import filter -Specifies which routes can be accepted by the router among all the received routes from a given neighbor Export filter -Specifies which routes can be advertised by the router to a given neighbor

Export Routing Policy  It is a conclusion from real world An AS does not provide transit service between its providers and peers. AS1AS2 AS3 AS4 customer Consider: AS u AS v is provider(u) or peer(u) For each best route r of u If first(r.as_path) is provider(u) or peer(u) Then: export(v,u)[{r}] = {}; It ’ s Selective Export Rule!

Export Routing Policy  It has import influence on Routing Table Entry Pattern AS1 AS2 AS4 AS3 AS1AS2 AS3AS1 AS2 AS1,AS2,AS3,AS4 AS1,AS2,AS3 Look at some AS Paths in routes of a Routing Table It will not appear in BGP routing table! This one is OK!

Export Routing Policy  All AS Path should be Valley Free After traversing a provider to customer edge or a peer to peer edge, the AS Path can not traverse a customer to provider or peer to peer edge. AS Path in Routing Table Reference will be given in the last part

Export Routing Policy  A example in a simulation Topology See Simulation Tool

Import Routing Policy  It is implemented by this attribute: local-preference AS4 AS2 AS3 AS1 Prefix: /16 ( /16, AS2, (AS1,AS2)) ( /16, AS3, (AS1,AS3)) If AS4 prefer link AS2-AS4,assign high local preference to this route! (for example: 100) If AS4 prefer link AS2-AS4,assign low local preference to this route! (for example: 80) Then AS2-AS4 will be selected as the best route by AS4!

Import Routing Policy  In real world, how do the router assign local preference to affect routing? In most case, the settings corresponds to the economical relationships between the ASes. Typical Local Preference! -Customer Routes have higher local preference than other routes. (Since provider is paid to carry packets to customer. Also AS thinks customer is closer to the destination ) - Peer Routes have higher local preference than Provider Routes. (Since he has to pay for the traffic to his provider) Due to the utilization of the local preference attribute, some paths on the Internet are longer than their shortest length.

Import Routing Policy  A example in a simulation Topology See Simulation Tool

Summary: Relationship and Policy  Relationship Provider to Customer Peer to Peer  Routing Policy Export Routing Policy  Selective Export Rule  Valley Free Routing Table Entry Pattern Import Routing Policy  Typical Local Preference

Outline  Introduction to Routing and BGP  AS Relationship and Routing Policy  Inbound Traffic Engineering and ASPP  ASPP at Different Level (my work)  Instability of BGP ASPP (my work)  Future Work  Conclusion

Inbound Traffic Engineering: Mutihomed  With the development of Internet, many AS became multihomed, which means it has more than one physical links to its provider or has more than one providers.  So it will try to control the distribution of inbound traffic and outbound traffic on each link  Only talk Inbound Traffic Engineering in this presentation. It has many ways to do inbound traffic engineering!

Inbound Traffic Engineering: Ways  Obviously, in order to do inbound traffic engineering, AS needs to send different BGP messages on different links to influence the BGP decision process of routers in distant AS. Selective announcements Prefix splitting: announce a large prefix on all links for redundancy, but prefer some links for parts of this prefix Remember: When forwarding an IP packet, a router will always select the longest match in its routing table AS Path Prepending: artificially increase the length of AS- Path

Inbound Traffic Engineering: Ways  Advantages and drawbacks Selective announcements always work, but if one prefix is advertised on a single link, it may become unreachable in case of failure Prefix Splitting better than selective announcements in case of failure. but increases significantly the size of all BGP tables. some ISPs filter announcements for long prefixes AS-Path prepending useful for backup link, but besides that, it is difficult to find the amount of prepending...

AS Path Prepending  AS Path Prepending: artificially increase the length of AS-Path AS1 AS2 AS4 AS3 AS5 Here, the traffic from AS5 to AS4 will follow this Path AS5-AS2-AS4 If AS4 want to shift the traffic to link AS3-AS4, it will send route to AS2 in this way: (Prefix, AS4, (AS4,AS4,AS4)) So AS5 will select other way because he thinks AS2- AS4-AS4-AS4 is longer than AS1-AS3-AS4 It is called “ AS4 prepend link AS2-AS4 twice (or by two) ”. (Note: prepending is directional, AS2 can prepend link AS4-AS2)

Community-based ASPP  AS can ask its provider, to do prepending for it AS1 AS2 AS4 AS3 AS5 BGP support the attribute ” community ”. AS can attach special community value to request down stream router to perform a special action, for example, to ask the router prepend for him. So it is called “ Community-based AS-Path prepending ”. In this case, AS4 can ask AS2 to prepend AS-Path when announcing to AS5. It is called “ AS2 prepends AS5-AS2 for AS4 ”

Summary: Inbound Traffic Engineering and ASPP  Multihomed  Three Way to Do Inbound Traffic Engineering  AS Path Prepending  Community-based ASPP

 Because BGP support “ Community-based ASPP ”, so we can do ASPP at different levels, for example, prepending at its parents or prepending at its grandfather. How does ASPP affect the whole network? How does ASPP at different levels affect the network?

Outline  Introduction to Routing and BGP  AS Relationship and Routing Policy  Inbound Traffic Engineering and ASPP  ASPP at Different Level (my work)  Instability of BGP ASPP (my work)  Future Work  Conclusion

ASPP at Different Levels  How to evaluate the influence of ASPP? The change of total traffic in the network. Add up all the traffic on all links. The number of affected AS pairs Affected AS pairs: the routing path between them changes because of ASPP.  Network traffic model Uniform Distribution: means for any ASi and ASj, the traffic from ASi to ASj is 10 units.  See Simulation Tools

ASPP at Different Levels  Findings ASPP will increase the total traffic of the network. ASPP at high level has less influence on local traffic and the whole Internet. Attention: These are simulation results of some random topology, not general conclusion. We need a good network topology model for real- world Internet!

ASPP at Different Levels  If it is true in real Internet When we want to shift a little traffic to the other link, we can ask the parents to do ASPP for us. It has less influence on the whole Internet.

Outline  Introduction to Routing and BGP  AS Relationship and Routing Policy  Inbound Traffic Engineering and ASPP  ASPP at Different Level (my work)  Instability of BGP ASPP (my work)  Future Work  Conclusion

Instability of BGP ASPP  My Network Model Every AS obeys Exporting Routing Policy  AS does not provide transit service between any two of its providers or peers. So all AS paths in routing table should be valley-free. Every AS obeys Importing Routing Policy  Customer route is preferred over peer route, and peer route over provider route. Uniform Traffic Distribution

Instability of BGP ASPP  Network Model: Every AS wants to do inbound traffic load balance through ASPP. So he will try to prepend the most heavy-loaded link and predict what will happen. If the prepending helps him to balance his traffic, he will do it. Otherwise, he will give up and thinks he has got optimal result.

Instability of BGP ASPP  It is an abstract simple example: Internet AS1AS2 Link1 Link3Link2 Link4 Time 1Time 2Time 3 Time Slot 1Time Slot 2 ……… At time1, Internet is stable. In time slot 1, AS1 find link1 has heavy traffic and prepending this link is helpful, he do it. This prepending affects the whole Internet. After some time, at time2, Internet become stable. But AS2 find his traffic is skewed, so he do prepending. AS2 also affects AS1, then in time slot 3, AS1 do prepending again … If every AS find he has had nothing to do to get better results, I call this process “ converged ”. Will this process converge after some time?

Instability of BGP ASPP  Distributed Greedy ASPP Algorithm : Does ASi have more than one providers?  If it doesn ’ t have, stop;  Otherwise, continue. Find the heavy-loaded link  Predict if it will have better balance after prepend this link by one.  If it will have, then do it.  Otherwise, predict the result of prepending it by two. If it helps him, do it.  Otherwise, predict the result of prepending it by three. If it helps him, do it.  If all these prepending don ’ t help, it has got best results, it stops. If all ASes stop, the process is converged.

Instability of BGP ASPP  How to evaluate the traffic balance? Formula: (AS has n links to providers) Balance coefficient =

Instability of BGP ASPP  See the simulation Two cases which will never converge.  Analyze these cases Do prepending in sequence will reduce the probability of non-convergence.

Outline  Introduction to Routing and BGP  AS Relationship and Routing Policy  Inbound Traffic Engineering and ASPP  ASPP at Different Level (my work)  Instability of BGP ASPP (my work)  Future Work  Conclusion

Future Work on ASPP  Try to find a way to model the AS topology of real-world Internet, so I can draw some general conclusions.  Based on the model, try to conclude the reason and the pattern of ASPP cycle (or loop).  Based on it, try to find if there is a practical way to prevent ASPP cycle.  If there is no practical way, try to find all the AS that form the cycle. (John said maybe they could negotiate and find some solution:-)

My Current Question:  Some papers said ASPP is not a effective way to do inbound traffic engineering, so will my future work be useful in real world? However, BGP is really an important research area. Any successful research results in this area will be significant for the development of Internet. In fact, it has been a hot topic in these three years.

Why BGP is “ hot ” : Growth in Table Size The reasons for the recent growth  Fraction of IPv4 address space advertised 24 % of total IPv4 space in % of total IPv4 space in April 2003  Increase in number of ASes About 3000 ASes in early 1998 More than ASes in April 2003  Increase in multi-homing Less than 1000 multi-homed stub ASes in early 1998 More than 6000 multi-homed stub ASes April 2003  Increase in advertisement of small prefixes Number of IPv4 addresses advertised per prefix In late 1999, 16k IPv4 addr. per prefix in BGP tables In April 2003, 8k IPv4 addr. per prefix in BGP tables

Why BGP is “ hot ” : Issues and Challenges  How to sustain the growth of the Internet?  In theory anyone can announce its routes with BGP, In practice, BGP routing tables cannot be infinite...  BGP should react quickly to link failures.  An ISP should be able to control the flow of its interdomain traffic.  Security of interdomain routing.  Path length was a good indicator only for 50% of the considered paths.

Conclusion  '' BGP is running on more than 100K routers (my estimate), making it one of the world's largest and most visible distributed system. Global dynamics and scaling principles are still not well understood...'' Tim Griffin, AT&T Research

Reference  1. Lixin Gao, On inferring autonomous system relationships in the internet, IEEE/ACM Transactions on Networking (TON), v.9 n.6, p , December 2001  2. Feng Wang, Lixin Gao, On inferring and characterizing internet routing policies, Proceedings of the conference on Internet measurement conference, October 27-29, 2003, Miami Beach, FL, USA

Thank You.