1. Announcement  Slides and reference materials available at http://www.cs.purdue.edu/homes/yau/tsinghua-course2006  Slides and reference materials available at http://www.cs.purdue.edu/homes/yau/tsinghua-course2006

2. Stable Internet Routing Without Global Coordination (based on Gao and Rexford) Application of materials learned in Lecture 1

3. Guaranteed Convergence by Global Coordination  Internet routing registry  Require all routing policies be registered  Check for consistency  Problems  ISPs not willing to reveal their policies  Consistency check is NP complete  Possible divergence after policy change or node / link failure  Internet routing registry  Require all routing policies be registered  Check for consistency  Problems  ISPs not willing to reveal their policies  Consistency check is NP complete  Possible divergence after policy change or node / link failure

4. Guaranteed Convergence by Distributed Approach  Prescription of set of guidelines for ASes to follow  Certain policies are disallowed  Observance of guidelines ensures convergence  Independent of underlying network topology  Guidelines based on  Internet hierarchical structure  Commercial relationships between ASes  Prescription of set of guidelines for ASes to follow  Certain policies are disallowed  Observance of guidelines ensures convergence  Independent of underlying network topology  Guidelines based on  Internet hierarchical structure  Commercial relationships between ASes

5. AS Commercial Relationships  Customer-provider  Customer pays provider for service  Peer-to-peer  Agreement between ASes to exchange traffic between their customers  Frequently no exchange of money  Relationship assumed equal usage  Peering agreements usually treated as business secrets  Backup  Service in case of failures  Customer-provider  Customer pays provider for service  Peer-to-peer  Agreement between ASes to exchange traffic between their customers  Frequently no exchange of money  Relationship assumed equal usage  Peering agreements usually treated as business secrets  Backup  Service in case of failures

6. Guidelines for Convergence  Routing preferences  Routing via customer link preferred over provider / peer link  Backup links always least preferred  AS free to choose local policies within each preference class  Guidelines not in BGP, but agree with ISP routing practice  Registry can be simplified to check only relationships between AS pairs  Routing preferences  Routing via customer link preferred over provider / peer link  Backup links always least preferred  AS free to choose local policies within each preference class  Guidelines not in BGP, but agree with ISP routing practice  Registry can be simplified to check only relationships between AS pairs

7. Internet Architecture  ASes are autonomous  Detailed knowledge of topology within AS  Limited knowledge of topology about other ASes  ASes interconnected at public IXPs or dedicated point-to-point links  Example IXPs: Mae-East, Mae-West  Connectivity does not imply flow of traffic  Routes must be established through BGP  ASes are autonomous  Detailed knowledge of topology within AS  Limited knowledge of topology about other ASes  ASes interconnected at public IXPs or dedicated point-to-point links  Example IXPs: Mae-East, Mae-West  Connectivity does not imply flow of traffic  Routes must be established through BGP

8. BGP Route Maintenance  Routes constructed by propagation of BGP advertisements  advertisement = prefix + attributes  Upon receipt of advertisement, BGP speaker decides whether to use path, and whether to propagate path  Routes removed by BGP withdrawals  Withdrawal leads to sequence of withdrawals by upstream ASes using path withdrawn  Routes constructed by propagation of BGP advertisements  advertisement = prefix + attributes  Upon receipt of advertisement, BGP speaker decides whether to use path, and whether to propagate path  Routes removed by BGP withdrawals  Withdrawal leads to sequence of withdrawals by upstream ASes using path withdrawn

9. BGP Route Selection  BGP allows many routing policies  Local preference (reflecting AS relationships)  Community attribute (hint to neighbor on preference that should be given to a route)  MED (control over how traffic from neighbors enters AS)  Otherwise, neighbors use hot potato routing  AS path prepending (ingress traffic engineering)  BGP allows many routing policies  Local preference (reflecting AS relationships)  Community attribute (hint to neighbor on preference that should be given to a route)  MED (control over how traffic from neighbors enters AS)  Otherwise, neighbors use hot potato routing  AS path prepending (ingress traffic engineering)

10. Advertisement Processing  Import policies  Which routes to consider for use  Path selection  Which imported route to use as best route  Export policies  Whether best route is advertised to a neighbor  If so, what to advertise? (route can change since their attributes can change)  Import policies  Which routes to consider for use  Path selection  Which imported route to use as best route  Export policies  Whether best route is advertised to a neighbor  If so, what to advertise? (route can change since their attributes can change)

11. BGP Protocol Dynamics  Route convergence not guaranteed  Group of ASes may continually advertise and withdraw routes to a prefix because of policy conflicts  Convergence concepts  Group of ASes in stable state when no AS would change its routes  Safe BGP system guarantees that group of ASes will reach stable state  Route convergence not guaranteed  Group of ASes may continually advertise and withdraw routes to a prefix because of policy conflicts  Convergence concepts  Group of ASes in stable state when no AS would change its routes  Safe BGP system guarantees that group of ASes will reach stable state

12. BGP Model  Clustered graph G = (N, V, E)  a(i) denotes AS of BGP speaker i  Clustered graph G = (N, V, E)  a(i) denotes AS of BGP speaker i AS 1 AS 2 AS 3 eBGP iBGP BGP speaker

13. Route Update Notation  For route update r  r.prefix: destination prefix  r.next_hop: next hop address  r.as_path: as path  r.local_pref: local preference  r.med: multiplex exit discriminator  r.c_set: community set  For route update r  r.prefix: destination prefix  r.next_hop: next hop address  r.as_path: as path  r.local_pref: local preference  r.med: multiplex exit discriminator  r.c_set: community set

14. BGP Processing Notations  BGP speaker v, eBGP session between two BGP speakers  Import  For set of updates R, set of updates remaining after applying implicit import policy of v on edge l: im_import(l,v)[R]  For explicit update policy: ex_import(l,v)[R]  Overall import policy:  import(l,v)[R] = ex_import(l,v)[im_import(l,v)[R]]  Route selection  For set of route updates S, best route for each prefix picked from S: Select(S)  BGP speaker v, eBGP session between two BGP speakers  Import  For set of updates R, set of updates remaining after applying implicit import policy of v on edge l: im_import(l,v)[R]  For explicit update policy: ex_import(l,v)[R]  Overall import policy:  import(l,v)[R] = ex_import(l,v)[im_import(l,v)[R]]  Route selection  For set of route updates S, best route for each prefix picked from S: Select(S)

15. BGP Processing Notations (cont’d)  Export  BGP speaker u applies implicit export policy on link l to neighbor v: im_export(l,u)  For explicit export policy: ex_export(l,u)  Overall export policy:  export(l,u)[R] = ex_export(l,u)[im_export(l,u)[R]]  Export  BGP speaker u applies implicit export policy on link l to neighbor v: im_export(l,u)  For explicit export policy: ex_export(l,u)  Overall export policy:  export(l,u)[R] = ex_export(l,u)[im_export(l,u)[R]]

16. Distributed Path Selection  Distributed and asynchronous process by all BGP speakers, for a prefix d  Triggered by advertisements / withdrawals  Sufficient for BGP speaker to remember only its own best route  System state is vector s = (s 1,…s n ), where s i is route chosen by speaker i  Distributed and asynchronous process by all BGP speakers, for a prefix d  Triggered by advertisements / withdrawals  Sufficient for BGP speaker to remember only its own best route  System state is vector s = (s 1,…s n ), where s i is route chosen by speaker i

17. Speaker Activation  System state changes when BGP speaker(s) apply route selection process  Speaker is activated when it has considered everything for triggering route s; i.e., it has applied  Export policies of all its neighbors  Import policies of itself  BGP route selection  Activation order not deterministic due to asynchronous execution of protocol  At given time, a subset A of speakers are activated  System state changes when BGP speaker(s) apply route selection process  Speaker is activated when it has considered everything for triggering route s; i.e., it has applied  Export policies of all its neighbors  Import policies of itself  BGP route selection  Activation order not deterministic due to asynchronous execution of protocol  At given time, a subset A of speakers are activated

18. Evolution of System State  System state s changes into next state s’  For i  A  s’ i = BestRoute(i,s) (best route chosen by i based on routes chosen by each speaker)  For i  A  s’ i = s i  s -> A s’ denotes state change from s to s’ based on activation set A  State s is stable iff s -> A s for any A  System state s changes into next state s’  For i  A  s’ i = BestRoute(i,s) (best route chosen by i based on routes chosen by each speaker)  For i  A  s’ i = s i  s -> A s’ denotes state change from s to s’ based on activation set A  State s is stable iff s -> A s for any A

19. Activation Sequence   (j) denotes j-th activation in activation sequence   (Infinite) sequence is fair if it has infinitely many j s.t. i   (j), for each speaker i  BGP system converges for  and s 0 if there is finite j s.t. s 0 ->  (1) s 1 … ->  (j) s j and s j is stable  BGP system may have a stable state but is not converging   (j) denotes j-th activation in activation sequence   (Infinite) sequence is fair if it has infinitely many j s.t. i   (j), for each speaker i  BGP system converges for  and s 0 if there is finite j s.t. s 0 ->  (1) s 1 … ->  (j) s j and s j is stable  BGP system may have a stable state but is not converging

20. (2,0) (0) (1,0) (0) AS 0 AS 1 AS 2 originates prefix d Stable state: AS 1 (2,0); AS 2 (0)

21. Safe BGP System  BGP system is safe if it has a stable state and converges under any fair activation sequence and any initial state  BGP system is inherently safe it it is safe and remains safe after removing any nodes / edges  BGP system is safe if it has a stable state and converges under any fair activation sequence and any initial state  BGP system is inherently safe it it is safe and remains safe after removing any nodes / edges

22. AS Relationships  Customers, providers, and peers of AS a: customer(a), provider(a), peer(a)  Route r is customer (provider, peer) route of a if next hop AS in r.as_path is in customer(a) (provider(a), peer(a))  Customers, providers, and peers of AS a: customer(a), provider(a), peer(a)  Route r is customer (provider, peer) route of a if next hop AS in r.as_path is in customer(a) (provider(a), peer(a))

23. Rules for BGP Export Policies  Export to provider  Can export own routes and routes of customers, but not routes from providers / peers  AS does not provide transit for its provider  Export to customer  Can export own routes, and routes from providers / peers  AS provides transit for its cusomers  Export to peer  Same as for provider  Export to provider  Can export own routes and routes of customers, but not routes from providers / peers  AS does not provide transit for its provider  Export to customer  Can export own routes, and routes from providers / peers  AS provides transit for its cusomers  Export to peer  Same as for provider

24. Hierarchical Relationships  Customer-provider relationship assumed hierarchical  In practice, AS chooses a larger AS as provider  Any direct / indirect provider of u cannot be a customer of u  Provider-to-customer graph is a DAG  Customer-provider relationship assumed hierarchical  In practice, AS chooses a larger AS as provider  Any direct / indirect provider of u cannot be a customer of u  Provider-to-customer graph is a DAG

25. Guideline A  AS prefers a route via a customer to a route via a provider / peer  Set by r.loc_pref  Theorem 1: For a BGP system that has only customer-provider and peer-peer relationshps, if all ASes follow A, the system is inherently safe  AS prefers a route via a customer to a route via a provider / peer  Set by r.loc_pref  Theorem 1: For a BGP system that has only customer-provider and peer-peer relationshps, if all ASes follow A, the system is inherently safe

26. Lemma 1: The BGP system has a stable state  Activate ASes in a two phase activation sequence  *  In phase 1, AS selects a customer route if one is available, following guideline A; accomplished by activating ASes by customer-provider partial order  In phase 2, ASes that do not have a customer route after Phase 1 get provider or peer routes; accomplished by activating ASes in provider-customer partial order  Activate ASes in a two phase activation sequence  *  In phase 1, AS selects a customer route if one is available, following guideline A; accomplished by activating ASes by customer-provider partial order  In phase 2, ASes that do not have a customer route after Phase 1 get provider or peer routes; accomplished by activating ASes in provider-customer partial order

27. Claim 1 for Lemma 1: A Phase 1 BGP speaker reaches a stable state after its activation in phase 1  Proof by induction on the order that Phase 1 speakers are activated in phase 1  Let Phase 1 BGP speaker i belong to AS n  Suppose all Phase 1 speakers that belong to an AS preceding AS n in Phase 1 reach a stable state after activation (induction hypothesis)  Speaker i selects best route from amongst its customer routes  Each such customer precedes AS n in  *; either has reached a stable state or does not get a customer route in Phase 1  Proof by induction on the order that Phase 1 speakers are activated in phase 1  Let Phase 1 BGP speaker i belong to AS n  Suppose all Phase 1 speakers that belong to an AS preceding AS n in Phase 1 reach a stable state after activation (induction hypothesis)  Speaker i selects best route from amongst its customer routes  Each such customer precedes AS n in  *; either has reached a stable state or does not get a customer route in Phase 1

28. Claim 1 (cont’d)  A customer that does not get a customer route in Phase 1 does not export its route to i (by export policy rule)  Such a customer’s routing decisions do not affect i’s decisions  A customer that does not get a customer route in Phase 1 does not export its route to i (by export policy rule)  Such a customer’s routing decisions do not affect i’s decisions

29. Claim 2 for Lemma 1: A Phase 2 speaker reaches a stable state after its activation in Phase 2  Let AS 0 be first Phase 2 speaker activated in Phase 2  Since AS 0 is in Phase 2, its BGP speakers can only get routes from its peers / providers  For the peers, they are either stable after Phase 1 (if they got a customer route), or they do not export their routes to AS 0 (if their best route is a provider / peer route)  For the providers, they must be phase 1 providers, and hence already stable when AS 0 is activated  Let AS 0 be first Phase 2 speaker activated in Phase 2  Since AS 0 is in Phase 2, its BGP speakers can only get routes from its peers / providers  For the peers, they are either stable after Phase 1 (if they got a customer route), or they do not export their routes to AS 0 (if their best route is a provider / peer route)  For the providers, they must be phase 1 providers, and hence already stable when AS 0 is activated

30. Claim 2 (cont’d)  Let Phase 2 speaker i belong to AS n  Suppose all BGP speakers that belong to an AS preceding AS n in Phase 2 reach a stable state after their activation in Phase 2 (induction hypothesis)  Since no customer route was learned in Phase 1, i must select route from its providers / peers  Each provider has already reached stable state (by  *)  Each peer is either Phase 1 AS or Phase 2 AS  If phase 1, peer’s route is already available to i when i is activated  If phase 2, peer selects route from its providers / peers; such a route will not be announced to i  Let Phase 2 speaker i belong to AS n  Suppose all BGP speakers that belong to an AS preceding AS n in Phase 2 reach a stable state after their activation in Phase 2 (induction hypothesis)  Since no customer route was learned in Phase 1, i must select route from its providers / peers  Each provider has already reached stable state (by  *)  Each peer is either Phase 1 AS or Phase 2 AS  If phase 1, peer’s route is already available to i when i is activated  If phase 2, peer selects route from its providers / peers; such a route will not be announced to i

31. Lemma 2: The BGP System Converges to Stable State for Any Initial State and Fair Activation Sequence  Given any fair activation sequence , proof by induction on ASes, in same order of ASes in  *  Originating AS stable after first activation  Suppose all speakers stable for ASes preceding AS n after  (t)  Find t’ s.t. all speakers in AS n activated at least once between t and t’; these speakers all become stable after  (t’) (received all updates as in  *)  Given any fair activation sequence , proof by induction on ASes, in same order of ASes in  *  Originating AS stable after first activation  Suppose all speakers stable for ASes preceding AS n after  (t)  Find t’ s.t. all speakers in AS n activated at least once between t and t’; these speakers all become stable after  (t’) (received all updates as in  *)

32. Proof of Theorem 1  Follows from Lemmas 1 and 2, and  Lemmas 1 and 2 are not affected by either node or link removals  Follows from Lemmas 1 and 2, and  Lemmas 1 and 2 are not affected by either node or link removals

33. Guideline B  Assumption P (constrained peer-peer relationships)  Peering only between similar ASes; therefore …  ASes do not peer with their direct / indirect providers  Guideline A can be relaxed to Guideline B:  Peer route never more preferred than customer route  Provider route always less preferred than customer route  Under Assumption P, BGP system inherently safe if all ASes follow Guideline B  Assumption P (constrained peer-peer relationships)  Peering only between similar ASes; therefore …  ASes do not peer with their direct / indirect providers  Guideline A can be relaxed to Guideline B:  Peer route never more preferred than customer route  Provider route always less preferred than customer route  Under Assumption P, BGP system inherently safe if all ASes follow Guideline B

34. What About Backup Links?  If route does not have a backup link, follow Guidelines A and B, and give them higher local preferences than the backup local preference  If route has a backup link, give it the backup local preference  All backup links have the same local preference  Shortest paths first routing among the backup paths  Restriction can be relaxed  If route does not have a backup link, follow Guidelines A and B, and give them higher local preferences than the backup local preference  If route has a backup link, give it the backup local preference  All backup links have the same local preference  Shortest paths first routing among the backup paths  Restriction can be relaxed

35. Robustness of Guidelines  Safety guidelines are independent of underlying network topology  Node / link failures  Relationship changes  Failures / changes trigger exchanges of route information, but guidelines ensure convergence after the changes  Safety guidelines are independent of underlying network topology  Node / link failures  Relationship changes  Failures / changes trigger exchanges of route information, but guidelines ensure convergence after the changes

36. Discussions  AS relationships between different destination prefixes do not interact  Possible to have different policies for different prefixes (as long as guidelines are obeyed)  Approach avoids instability  Peace of mind; effective despite dynamic operating conditions; but  Can be overly conservative; prevents otherwise useful policies  Alternative dynamic approach possible to detect and resolve conflicts when necessary  AS relationships between different destination prefixes do not interact  Possible to have different policies for different prefixes (as long as guidelines are obeyed)  Approach avoids instability  Peace of mind; effective despite dynamic operating conditions; but  Can be overly conservative; prevents otherwise useful policies  Alternative dynamic approach possible to detect and resolve conflicts when necessary