Lessons Learned TEIN2 and CERNET Xing Li 2007-01-22

Outline Introduction TEIN2 routing policy CERNET BGP Experience Lessons learned

Simple Case (where BGP can handle things easily) Global transit To tier 1 or tier 2 commodity networks Care the aggregation Care the load balancing Don’t care the symmetry Peering (no transit, except for the down streams) To domestic ISPs (bi-literal or via IX) Care the business model To academic partners Care the performance Care the symmetry

Complicated Case (where BGP cannot handle things easily) Global transit To tier 1 or tier 2 commodity networks Care the aggregation Care the load balancing Don’t care the symmetry Academic transit To multiple transit backbones within academic scope Care the performance Care the symmetry Etc. Peering (no transit, except for the down streams) To domestic ISPs (bi-literal or via IX) Care the business model To academic partners

Two Steps to Implement the Policy Identification IP prefix AS path regular expression Community tag Path selection AS path (inbound and outbound) Local-preference (outbound) More specific (inbound)

For Transit Network TEIN2 Example

TEIN2 Topology

The Principle of Routing Design for the TEIN2 network To provide interconnection among TEIN2 partners and between TEIN2 partners and EU NRENs. To provide back-up paths within the TEIN2 network and/or via partner networks for service resilience when possible. To provide a flexible and transparent routing policy to TEIN2 NRENs. To avoid being selected by GÉANT, Abilene and other R&E networks outside TEIN2 as the preferred transit network. To minimize the adjustment of the external peers’ routing policy outside TEIN2 networks, e.g. GÉANT and APAN.

TEIN2 Routing Policy Enable additive community tagging to mark the prefix announcements. Adopt AS number prepending as the preferred BGP policy for TEIN2 traffic adjustment within TEIN2 backbone. Use ingress AS number prepending for outbound traffic adjustment, including traffic from TEIN2 POP to NRENs, GÉANT and APAN. Use egress AS number prepending for inbound traffic adjustment, including traffic from NRENs, GÉANT and APAN to TEIN2 POP. May use Local-Preference amendment as the last resort of mechanism for fine tuning on TEIN2 traffic over the backbone.

For NRN CERNET Example

CERNET Topology

CERNET Peering DRAGONTAP CNGI-BJIX DRAGONLIGHT Internet CERNET TEIN2 STARLIGHT HARNET ASNET DRAGONLIGHT CNGI-BJIX DRAGONTAP CERNET 2 Internet Domestic Peering CNGI APAN KOREN 3 G 12G 155M 100M 1G 2x155M 622M 10G

CERNET Routing Policy Outbound Inbound Use AS number prepending if possible Heavily use Local-Preference Enable additive community tagging to mark the prefixes Inbound Announce more specifics

Case 1 TAIWAN Earthquake

Earthquake on 26th DEC 2006

Why did not include this policy before the earthquake?

Case 2 Routing and End-to-end performance

Ping and dvping beacons

Here in the APAN venue WLAN

Lessons Learned (1) The nature of BGP is reachability Stupid routing happen Policy based routing makes thing very complicated The routing and topology are very dynamic environment The key words are: simple, open and controllability For transit network Simple Open For NRN Controllability Why did not include this policy before the earthquake? Because it is a NP problem and there are many contradict requirements Mission impossible What should be the solution?

Lessons Learned (2) It seems that we still need to do a lot manual BGP policy adjustment, case by case with the help of Multi-site collaborations Routeviews We have to compare the routing table with the end-to-end performance matrix dvping tool