TEIN3 Inter-Domain Routing Guideline Author: Xing Li, Jilong Wang, Zhonghui Li, Version: Draft 1.3 Date: 8 July 2011

Abstract This document describes the inter-domain routing guideline for TEIN3. It contains the principle of inter-domain routing design, topology of TEIN3 networks, outline of unicast routing policy, suggestions on advanced routing implementation and some complicated routing policy scenarios.

Document Revision History Date Version Number Author/Editor Summary of main changes 1 January 2009 Draft 1.0 Xing Li Draft 1.0 revised from TEIN2 version 10 January 2010 Draft 1.1 Modification on topology, communities and specific routing policy. 19 May 2010 Draft 1.2 Added community-signalling part for partner-initialized fine tuning 8 July 2011 Draft 1.3 Modification on topology, specific routing policy among TEIN3, CNGI-6IX and GEANT

Content Principle of Routing Design Topology of TEIN3 Networks Outline of Unicast Routing Policy Suggestions on Advanced Routing Implementation Configuration Examples for NRENs

Principle of Routing Design To provide interconnection among TEIN3 partners and between TEIN3 partners and EU NRENs To provide back-up paths within the TEIN3 network and/or via partner networks for service resilience when possible. To provide a flexible and transparent routing policy to TEIN3 NRENs

Principle of Routing Design (continued) To avoid being selected by GEANT, Internet2 and other R&E networks outside TEIN3 as the preferred transit network To minimize the adjustment of the external peers’ routing policy outside TEIN3 networks, e.g. GEANT and APAN-JP To provide advanced routing service to TEIN3 NRENs, e.g. multicast and MPLS

Topology of TEIN3 Networks

Illustration for BJ-COP STM16 link migration from TEIN3 backbone to CNGI-6IX (Appendix 2) Before the migration, BJ-COP link is terminated to TEIN3 BJ PoP router CERNET←→ EU: CERNET(4538)←→CNGI-6IX(23911)←→TEIN3-NORTH (24489)←→GEANT(20965) CSTNET ←→ EU: CSTNET(7497)←→CERNET(4538)←→CNGI-6IX(23911)←→TEIN3-NORTH (24489)←→GEANT(20965) TEIN3 ←→ EU: TEIN3-NORTH (24489)←→GEANT(20965)

Illustration for BJ-COP STM16 link migration from TEIN3 backbone to CNGI-6IX (Appendix 2) After the migration, BJ-COP link is terminated to CNGI-6IX BJ PoP router CERNET←→ EU: CERNET(4538)←→CNGI-6IX(23911)←→GEANT(20965) CSTNET ←→ EU: CSTNET(7497)←→CNGI-6IX(23911)←→GEANT(20965) TEIN3 ←→ EU: TEIN3-NORTH(24489)←→TEIN3-SOUTH(24490)←→GEANT(20965) TEIN3 ←→ EU (backup): TEIN3-NORTH(24489)←→CNGI-6IX(23911)←→GEANT(20965)

Outline of Unicast Routing Policy Routing Policy Overview Description of Common Routing Policy Description of Specific Routing Policy Suggestions to Routing Policy on TEIN3 NREN/partner (transit network) sides Suggestions to Routing Policy on TEIN3 NREN (non-transit networks) sides Illustrations for traffic between TEIN3 NRENs and GEANT/EU & Internet2/US

Routing Policy Overview Routing Control Definition of BGP communities

Routing Control Enable additive community tagging to mark the prefix announcements Adopt AS number prepending as the preferred BGP policy for TEIN3 traffic adjustment within TEIN3 backbone Use ingress AS number prepending for outbound traffic adjustment, including traffic from TEIN3 PoP to NRENs, GEANT and APAN-JP Use egress AS number prepending for inbound traffic adjustment, including traffic from NRENs, GEANT and APAN-JP to TEIN3 PoP May use local-preference amendment as the last resort of mechanism for fine tuning on TEIN3 traffic over the backbone

Definition of BGP communities In TEIN3 backbone (PoP routers) Name Value Explanation tein-self 24489:0 Routes generated by TEIN3 backbone itself tein-nrn 24489:65155 Routes received from any TEIN3 NREN (non-transit) tein-nrn-benef 24489:65156 Routes received from any TEIN3 beneficiary NREN (non-transit) tein-geant 24489:20965 Routes received from GEANT tein-apan-jp 24489:7660 Routes received from APAN-JP tein-aarnet 24489:7575 Routes received from AARNet for backup transit tein-koren 24489:9270 Routes received from KOREN for backup transit tein-singaren 24489:23855 Routes received from SingAREN for backup transit (IPv4 only) tein-caren 24489:65157 Routes received from CAREN tein-backbone 24489:65200 Routes advertised from TEIN3 PoP routers tein-no-exp 24489:65201 Routes advertised from TEIN3 PoP routers which should not be exported outside NREN tein-low-tran 24489:65202 Routes advertised from TEIN3 PoP routers which should be taken as lower priority for transit tein-high-tran 24489:65203 Routes advertised from TEIN3 PoP routers which should be taken as higher priority for transit tein-benef 24489:65500 Routes received from any TEIN3 peers which should be announced to beneficiary NREN only tein-blocked 24489:65501 Routes received from any TEIN3 peers which should not be announced to any other TEIN3 peer

Definition of BGP communities In any TEIN3 peer network for community-signalling (router peering with TEIN3 BJ/HK/SG/MB PoP) Name Value Explanation peer-low-tran 24490:65202 Routes received from peer which should be taken as lower priority for transit by TEIN3 backbone and other TEIN3 peers peer-high-tran 24490:65203 Routes received from peer which should be taken as higher priority for transit by TEIN3 backbone and other TEIN3 peers peer-high-tran-spec 24490:xxxxx Routes received from peer which should be taken as higher priority for transit by the TEIN3 peer whose AS number is xxxxx peer-benef 24490:65500 Routes received from peer which should be announced to beneficiary NREN only peer-blocked 24490:65501 Routes received from peer which should not be announced to any other TEIN3 peer peer-blocked-spec 65300:xxxxx Routes received from peer which should not be announced to the TEIN3 peer whose AS number is xxxxx peer-prep-1-spec 65301:xxxxx Routes received from peer which should be prepended 1 time of TEIN3 AS number before announced to the TEIN3 peer whose AS number is xxxxx peer-prep-2-spec 65302:xxxxx Routes received from peer which should be prepended 2 times of TEIN3 AS number before announced to the TEIN3 peer whose AS number is xxxxx peer-prep-3-spec 65303:xxxxx Routes received from peer which should be prepended 3 times of TEIN3 AS number before announced to the TEIN3 peer whose AS number is xxxxx peer-prep-6-spec 65306:xxxxx Routes received from peer which should be prepended 6 times of TEIN3 AS number before announced to the TEIN3 peer whose AS number is xxxxx

Definition of BGP communities In GEANT network (routers peering with TEIN3 MB PoP) Name Value Explanation geant-nrn 20965:155 Routes received from any GEANT NREN geant-eumed 20965:21320 Routes of EUMEDCONNECT NRENs geant-tenet 20965:2018 Routes received from TENET (RSA) geant-internet2 20965:11537 Routes received from Internet2 for transit geant-canari 20965:6509 Routes received from Canarie for transit geant-clara 20965:27750 Routes received from RedClara for transit geant-esnet 20965:293 Routes received from ESnet for transit

Definition of BGP communities In APAN-JP network (routers peering with TEIN3 HK/SG/JP PoP) Name Value Explanation apan-jp-nrn 7660:65155 Routes received from any APAN-JP NREN apan-jp-transpac2 7660:22388 Routes received from TransPAC2 for transit apan-jp-internet2 7660:11537 Routes received from Internet2 for transit apan-jp-starlight 7660:10764 Routes received from StarLIGHT for transit apan-jp-sinet 7660:2907 Routes received from SINET for transit apan-jp-ren-v6 7660:500 Routes received from R&E peers (IPv6) apan-jp-other-v6 7660:4 Routes received from non-R&E peers (IPv6)

Definition of BGP communities In AARNET network (router peering with TEIN3 SG PoP) Name Value Explanation aarnet-ren 7575:1000 Routes received from AARNet customer networks aarnet-peer 7575:1001 Routes received from R&E peers with AARNet aarnet-au 7575:6001 Routes (with 7575:1001) received from Australasian network aarnet-na 7575:6003 Routes(with 7575:1001) received from North American network (IPv4 only) aarnet-nas 7575:6004 Routes(with 7575:1001) received from North Asian network aarnet-sas 7575:6005 Routes(with 7575:1001) received from South Asian network

Definition of BGP communities In KOREN network (router peering with TEIN3 HK PoP) Name Value Explanation koren-ren 9270:65155 Routes received from KOREN R&E networks koren-apan-jp 9270:7660 Routes received from APAN-JP for transit

Definition of BGP communities In CNGI-6IX network (router peering with TEIN3 BJ PoP) Name Value Explanation cngi-6ix-ren 23911:65155 Routes received from CNGI-6IX R&E networks cngi-6ix-cst 23911:7497 Routes received from CSTNET via CNGI-6IX for transit cngi-geant 23911:20965 Routes received from GEANT via CNGI-6IX for backup transit

Definition of BGP communities In SingAREN network (router peering with TEIN3 SG PoP) Name Value Explanation singaren-ren 23855:900 Routes received from SingAREN R&E networks singaren-ntu 23855:12 Routes received from NTU in SG singaren-nus 23855:22 Routes received from NUS in SG singaren-bii 23855:32 Routes received from BII in SG singaren-ihpc 23855:42 Routes received from IHPC in SG singaren-apan-jp 23855:7660 Routes received from APAN-JP for transit (IPv4 only)

Definition of BGP communities In ThaiREN network (router peering with TEIN3 SG PoP) Name Value Explanation thairen-ren 24475:65155 Routes received from ThaiREN R&E networks

Definition of BGP communities In MYREN network (router peering with TEIN3 SG PoP) Name Value Explanation myren-ren 24514:65155 Routes received from MYREN R&E networks

Definition of BGP communities In VinaREN network (router peering with TEIN3 HK PoP) Name Value Explanation vinaren-ren 24175:65155 Routes received from VinaREN R&E networks

Definition of BGP communities In ASTI network (router peering with TEIN3 HK PoP) Name Value Explanation asti-ren 9821:65155 Routes received from ASTI R&E networks

Definition of BGP communities In HARNET network (router peering with TEIN3 HK PoP) Name Value Explanation harnet-ren 3662:65155 Routes received from HARNET R&E networks

Definition of BGP communities In ITB network (router peering with TEIN3 HK PoP) Name Value Explanation itb-ren 18007:65155 Routes received from ITB R&E networks

Definition of BGP communities In NKN network (router peering with TEIN3 MB PoP) Name Value Explanation nkn-ren 55824:65155 Routes received from NKN R&E networks

Definition of BGP communities In NREN (Nepal) network (router peering with TEIN3 MB PoP) Name Value Explanation nren-ren 45170:65155 Routes received from NREN (Nepal) R&E networks

Definition of BGP communities In LEARN network (router peering with TEIN3 MB PoP) Name Value Explanation learn-ren 38229:65155 Routes received from LEARN R&E networks

Definition of BGP communities For Internet2 backbone (PoP routers peering with TEIN3-JP, APAN-JP, KOREN, SingAREN and AARNet which provide transit service for TEIN3-Internet2 traffic) Name Value Explanation internet2-block-com 11537:2002 Routes advertised to Internet2 (via TEIN3 peers) which should not be announced to any commercial peer of Internet2

Description of Common Routing Policy BOGON Filters Private AS number Filtering Prefix Length Filtering Authenticated BGP Sessions Maximum Prefixes Community Filtering Community Signalling

BOGON Filters For IPv4, networks that should be filtered include the following ones RFC1918 addresses (reserved: 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16, etc.) Loopback address (127.0.0.0/8) Class D and E addresses (224.0.0.0/3) Various other reserved addresses (0.0.0.0/0, etc.) Your own IP address aggregates (import only) For IPv6, the prefixes permitted should belong to 2000::/3 except 2002::/32 (IETF recommendation)

Private AS number Filtering Private AS number, from 64512 to 65534, is not supposed to appear in BGP AS-path attribute. Although there are some customer networks that use private AS number for BGP peering, we must reject those routes and recommend customer networks to apply for public AS number.

Prefix Length Filtering Filtering based on prefix length is a common practice. This provides protection against IGP leaking, de-aggregation, routing table pollution and other accidents. A /24 is generally the longest mask for IPv4 that should be accepted, a /48 is generally the longest prefix length for IPv6, and these will be the cases for TEIN3. Even if TEIN3 were to not implement this then many other networks (even R&E networks) further afield will do so.

Authenticated BGP Sessions MD5 authentication can be used between BGP peers to protect the peering sessions. Each TCP segment is authenticated using an MD5 hash, and a password which is only known to each peer. Authentication can be used to prevent TCP spoofing and injection of bogus routing information. Peerings will not come up if the password is wrong or missing. All TEIN3 iBGP and eBGP peerings will be authenticated.

Maximum Prefixes This option limits the maximum number of prefixes that can be received from a peer. It protects against a peer network’s instabilities that cause an extraordinary number of prefixes to be advertised. For example, a peer network may usually advertise only five prefixes but then turns up a transit peer and accidentally advertises 100,000 prefixes. For both access and transit peers, the reasonable limit should be carefully evaluated and investigated from corresponding peers respectively for IPv4 and IPv6. As an interim solution, we plan to apply different limitation on maximum number of prefixes to transit and non-transit network, just over IPv4 prefixes. Network Type Maximum number of prefixes transit network 20000 non-transit network 1000

Community Filtering Since the implementation of TEIN3 routing policy is mainly based on additive community-tagging, it is highly necessary for us to apply community filtering over TEIN3 backbone, so as to reduce the impact of mis-tagging TEIN3 dedicated communities (inattentively or even maliciously) outside TEIN3 backbone

Community Filtering (continued) Considering there is no other partner network who provides transit service for the traffic among three ASs of TEIN3 backbone (including TEIN3 JP PoP, AS24287), i.e. any route with TEIN3 dedicated communities tagged should not be learned from external peers outside TEIN3 BJ/HK/SG/MB/JP PoP, so we can simply apply community filter (import only) on all BGP sessions between TEIN3 PoP and TEIN3 NRENs/partners to reject any route from external peers with TEIN3 dedicated communities (24489:*) tagged

Community Filtering (continued) By doing this, we can ensure that all routes advertised from TEIN3 backbone to direct external BGP peers are correctly tagged with TEIN3 dedicated communities, then TEIN3 NRENs and partners can deploy their routing policy (based on the authentic TEIN3 dedicated communities) for the routes learned from direct BGP peerings with TEIN3 backbone, without worry of misidentification caused by TEIN3 community mis-tagging.

Community signalling For the purpose of automatic processing of partner-initialized fine tuning on specific prefix routing, we introduce community-based signalling mechanism for TEIN3 partners, by which TEIN3 backbone will apply predefined action on the specific prefix with particular community tagged by TEIN3 as follows

Community signalling (continued) 24490:65202 (peer-low-tran) In the import policy of partner, decrease the local-preference of prefix by 10 and add community of 24489:65202 (tein-low-tran) on the specific prefix 24490:65203 (peer-high-tran) In the import policy of partner, increase the local-preference of prefix by 10 and add community of 24489:65203 (tein-high-tran) on the specific prefix 24490:xxxxx (peer-high-tran-spec) In the export policy of peer whose AS number is xxxxx, add community of 24489:65203 (tein-high-tran) on the specific prefix

Community signalling (continued) 24490:65500 (peer-benef) In the import policy of partner, add community of 24489:65500 (tein-benef) on the specific prefix 24490:65501 (peer-blocked) In the import policy of partner, add community of 24489:65501 (tein-blocked) on the specific prefix 65300:xxxxx (peer-blocked-spec) In the export policy of peer whose AS number is xxxxx, prevent the specific prefix from being advertised

Community signalling (continued) 65301:xxxxx (peer-prep-1-spec) In the export policy of peer whose AS number is xxxxx, prepend 1 time of TEIN3 AS number 65302:xxxxx (peer-prep-2-spec) In the export policy of peer whose AS number is xxxxx, prepend 2 times of TEIN3 AS number 65303:xxxxx (peer-prep-3-spec) In the export policy of peer whose AS number is xxxxx, prepend 3 times of TEIN3 AS number 65306:xxxxx (peer-prep-6-spec) In the export policy of peer whose AS number is xxxxx, prepend 6 times of TEIN3 AS number

Community Signalling (continued) Note For the peer with 4 bytes AS number, specific communities need to be defined, e.g. 24490:65157 defined as peer-high-tran-spec community for CAREN (AS197118). Due to the sensitivity of TEIN3 backbone on the communities above, we strongly suggest TEIN3 partners help to avoid mis-tagging of those communities on customer prefixes before announcing to TEIN3 PoPs, so as to prevent TEIN3 backbone from applying inappropriate action on those prefixes

Description of Specific Routing Policy Routing Policy between TEIN3 PoP and NREN (non-transit network) Routing Policy between TEIN3 PoP (MB) and GEANT Routing Policy between TEIN3 PoP (JP) and APAN-JP (transit network to Internet2 and other North American networks, backup transit network to KOREN) Routing Policy between TEIN3 PoP (HK/SG) and NICT (backup transit network to KOREN, APAN-JP & North American networks) Routing Policy between TEIN3 PoP (SG) and AARNet (backup transit network to North American networks) Routing Policy between TEIN3 PoP (HK) and KOREN (backup transit network to APAN-JP & North American networks) Routing Policy between TEIN3 PoP (BJ) and CNGI-6IX (backup transit network to GEANT) Routing Policy between TEIN3 PoP (SG) and SingAREN (backup transit network to APAN-JP & North American networks, IPv4 only)

Routing Policy between TEIN3 PoP and NREN (non-transit network) Note 1: With the exception of routes tagged with tein-blocked, which should not be announced to any other TEIN3 peer. With the exception of routes tagged with tein-benef, which should not be announced to non-beneficiary networks. Note 2: For routes originated from TEIN3 backbone, tag with tein-self and tein-backbone; for other routes, tag with tein-no-exp. Note 3: For NREN with more than one AS number, i.e. there are customer networks (non-transit) connected to the NREN, the routes should not be limited to self-originated (^$), and these routes should be tagged with particular community of the NREN, i.e. AS_NREN:65155. Note 4: For NREN with more than one AS number, i.e. there are customer networks (non-transit) connected to the NREN, the routes should not be limited to self-originated (^AS_NREN$), but be filtered based on particular community of the NREN, i.e. AS_NREN:65155. Note 5: For routes from beneficiary NREN, tagged with tein-nrn and tein-nrn-benef; for routes from non-beneficiary NREN, tagged with tein-nrn.

Routing Policy between TEIN3 PoP (MB) and GEANT Note 1: With the exception of routes tagged with tein-blocked, which should not be announced to any other TEIN3 peer. With the exception of routes from North American networks, as well as routes tagged with tein-benef or apan-jp-other-v6 (IPv6), which should not be announced to non-beneficiary or European R&E networks. Note 2: All routes should be tagged with tein-backbone, except for routes with cernet-cst community on TEIN3 BJ PoP, which should be tagged with both tein-backbone and tein-high-tran, so as to let GEANT take TEIN3 backbone as the preferred network for CSTNET IPv4 transit traffic (via CERNET) by raising local-preference of those routes. For routes originated from TEIN3 backbone, tag with tein-self and tein-backbone Note 3: To prepend 3 times of ASN on routes tagged with geant-internet2, geant-canari, geant-clara and geant-esnet, so as to prevent TEIN3 NRENs from selecting GEANT as preferred transit network to Internet2 and other North American networks.

Routing Policy between TEIN3 PoP (JP) and APAN-JP (transit network to Internet2 and other North American networks, backup transit network to KOREN) Note 1: With the exception of routes tagged with tein-blocked, which should not be announced to any other TEIN3 peer. Note 2: For routes originated from TEIN3 backbone, tag with tein-self and tein-backbone; for other routes, tag with tein-backbone. Note 3: For routes from ASTI, tag with tein-nrn. The others are tagged with tein-apan-jp.

Routing Policy between TEIN3 PoP (HK/SG) and NICT (backup transit network to KOREN, APAN-JP & North American networks) Note 1: With the exception of routes tagged with tein-blocked, which should not be announced to any other TEIN3 peer. To prepend 2 times of ASN on all routes advertised to NICT, so as to prevent KOREN, APAN-JP and North American networks from selecting HK-NICT or SG-NICT as preferred transit link to TEIN3 NRENs. Note 2: For routes originated from TEIN3 backbone, tag with tein-self and tein-backbone; for other routes, tag with tein-backbone. Note 3: To prepend 2 times of ASN on all routes from NICT, so as to prevent TEIN3 NRENs from selecting HK-NICT or SG-NICT as preferred transit link to KOREN, APAN-JP and North American networks. Note 4: For routes from ASTI, tag with tein-nrn. The others are tagged with tein-apan-jp.

Routing Policy between TEIN3 PoP (SG) and AARNet (backup transit network to North American networks) Note 1: With the exception of routes tagged with tein-blocked, which should not be announced to any other TEIN3 peer. With the exception of routes tagged with tein-benef, which should not be announced to non-beneficiary networks. Note 2: For routes originated from TEIN3 backbone, tag with tein-self and tein-backbone; for other routes, tag with tein-backbone. Note 3: To prepend 4 times of ASN on routes tagged with both aarnet-peer and aarnet-na, so as to prevent TEIN3 NRENs from selecting AARNet as preferred transit network to Internet2 and other North American networks. Note 4: To prepend 4 times of ASN on IPv6 routes tagged with aarnet-peer, so as to prevent TEIN3 NRENs from selecting AARNet as preferred transit network to GEANT, Internet2 and other North American networks. Note 5: For routes tagged with aarnet-ren, tag with tein-nrn; for routes tagged with aarnet-peer or both aarnet-peer and aarnet-na, tag with tein-aarnet and tein-benef.

Routing Policy between TEIN3 PoP (HK) and KOREN (backup transit network to APAN-JP & North American networks) Note 1: With the exception of routes tagged with tein-blocked, which should not be announced to any other TEIN3 peer. With the exception of routes tagged with tein-benef, which should not be announced to non-beneficiary networks. Note 2: For routes originated from TEIN3 backbone, tag with tein-self and tein-backbone; for other routes, tag with tein-backbone. Note 3: To prepend 1 time of ASN on routes tagged with koren-apan-jp, so as to prevent TEIN3 NRENs from selecting KOREN as preferred transit network to APAN-JP and North American networks. Note 4: For routes tagged with koren-ren, tag with tein-nrn; for routes tagged with koren-apan-jp, tag with tein-koren.

Routing Policy between TEIN3 PoP (BJ) and CNGI-6IX (backup transit network to GEANT) Note 1: With the exception of routes tagged with tein-blocked, which should not be announced to any other TEIN3 peer. Note 2: For routes originated from TEIN3 backbone, tag with tein-self and tein-backbone; for other routes, tag with tein-backbone. Note 3: For routes tagged with cngi-geant, prepend 2 times of AS number before announcing to TEIN3 backbone. Note 4: For routes tagged with cngi-6ix-ren or cngi-6ix-cst, tag with tein-nrn and tein-nrn-benef. For routes tagged with cngi-geant, tag with tein-geant

Routing Policy between TEIN3 PoP (SG) and SingAREN (backup transit network to APAN-JP & North American networks, IPv4 only) Note 1: With the exception of routes tagged with tein-blocked, which should not be announced to any other TEIN3 peer. With the exception of routes tagged with tein-benef, which should not be announced to non-beneficiary networks. Note 2: For routes originated from TEIN3 backbone, tag with tein-self and tein-backbone; for other routes, tag with tein-backbone. Note 3: To prepend 1 time of ASN on routes tagged with singaren-apan-jp, so as to prevent TEIN3 NRENs from selecting SingAREN as preferred transit network to APAN-JP and North American networks. Note 4: For routes tagged with singaren-ren or singaren-bii or singaren-ihpc or singaren-ntu or singaren-nus, tag with tein-nrn; for routes tagged with singaren-apan-jp, tag with tein-singaren.

Suggestions to Routing Policy on TEIN3 NREN/partner (transit network) sides First of all, TEIN3 NOC doesn’t want to interfere in the deployment of routing policy on TEIN3 NREN/partner sides who act as transit networks for TEIN3 traffic, and TEIN3 NRENs/partners are fully free to design their own routing policy. However, in order to guarantee the overall interests of TEIN3 NRENs and partners, TEIN3 NOC still need cooperation from TEIN3 NRENs/partners, by which we can improve the availability, stability and efficiency of TEIN3 networks.

Suggestions to Routing Policy on TEIN3 NREN/partner (transit network) sides (continued) Accept all the routes advertised from TEIN3 PoP routers, which are tagged with tein-backbone or tein-no-exp, and set higher priority for the routes tagged with tein-self Avoid the leaking of inappropriate routes into TEIN3 backbone, e.g. routes of commercial networks Avoid misoperation on community tagging for BGP routes For routes learned from TEIN3 backbone, which are tagged with tein-low-tran, set them with lower priority (local-preference in BGP) For routes learned from TEIN3 backbone, which are tagged with tein-high-tran, set them with higher priority (local-preference in BGP)

Suggestions to Routing Policy on TEIN3 NREN/partner (transit network) sides (continued) GEANT On GEANT border router peering with TEIN3 MB PoP, all routes, tagged with geant-nrn or geant-eumed or geant-tenet or geant-internet2 or geant-canari or geant-clara or geant-esnet (7 types of communities totally), should be announced to TEIN3 MB PoP. In order to prevent Internet2 and other North American networks from selecting GEANT as preferred transit network to TEIN3 NRENs, on GEANT border router peering with Internet2 and other North American networks, 3 times of ASN should be prepended on routes tagged with tein-nrn, before advertised to Internet2 and other North American networks

Suggestions to Routing Policy on TEIN3 NREN/partner (transit network) sides (continued) APAN-JP On APAN-JP border router peering with TEIN3 JP PoP, all routes, tagged with apan-jp-nrn or apan-jp-transpac2 or apan-jp-internet2 or apan-jp-starlight or apan-jp-sinet or apan-jp-ren-v6 or apan-jp-other-v6 (7 types of communities totally), should be announced to TEIN3 JP PoP. In order to take KOREN as a backup transit network for TEIN3 backbone’s access to APAN-JP and North American networks, on APAN-JP border router peering with KOREN, all routes advertised from KOREN, which are tagged with tein-backbone, should be accepted and advertised to APAN-JP NRENs and North American networks; in the meantime, all routes, tagged with 7 types of communities aforementioned, should be advertised to KOREN. In order to take APAN-JP as a backup transit network for KOREN’s access to TEIN3 backbone, in case of HK-KR link outage, on APAN-JP border router peering with KOREN, all routes advertised from KOREN, which are tagged with koren-ren, should be accepted and advertised to TEIN3 JP PoP with community of apan-jp-nrn; in the meantime, all routes learned from TEIN3 JP PoP, which are tagged with tein-backbone, should be advertised to KOREN. In order to take SingAREN as a backup transit network for TEIN3 backbone’s access to APAN-JP and North American networks, on APAN-JP border router peering with SingAREN, all routes advertised from SingAREN, which are tagged with tein-backbone, should be accepted and advertised to APAN-JP NRENs and North American networks; in the meantime, all routes tagged with 7 types of communities aforementioned, should be advertised to SingAREN.

Suggestions to Routing Policy on TEIN3 NREN/partner (transit network) sides (continued) NICT On NICT border router peering with TEIN3 HK/SG PoP, all routes, tagged with apan-jp-nrn or apan-jp-transpac2 or apan-jp-internet2 or apan-jp-starlight or apan-jp-sinet or apan-jp-ren-v6 or apan-jp-other-v6 (7 types of communities totally), should be announced to TEIN3 HK/SG PoP.

Suggestions to Routing Policy on TEIN3 NREN/partner (transit network) sides (continued) AARNet On AARNet border router peering with TEIN3 SG PoP, all IPv4 routes, tagged with aarnet-ren or both aarnet-peer and aarnet-na, and all IPv6 routes, tagged with aarnet-ren or aarnet-peer (3 types of community sets for both IPv4 and IPv6 totally), should be announced to TEIN3 SG PoP. In order to take AARNet as a backup transit network for TEIN3 backbone’s access to Internet2 and other North American networks, on AARNet border router peering with Internet2 and other North American networks, all routes advertised from Internet2 and other North American networks, should be accepted and tagged with aarnet-peer and aarnet-na, then advertised to TEIN3 SG PoP; in the meantime, all routes learned from TEIN3 SG PoP, which are tagged with tein-nrn, should be advertised to Internet2 and other North American networks. In order to prevent Internet2 and other North American networks from selecting AARNet as preferred transit network to TEIN3 NRENs, on AARNet border router peering with Internet2 and other North American networks, 4 times of ASN should be prepended on the routes tagged with tein-nrn, before advertised to Internet2 and other North American networks.

Suggestions to Routing Policy on TEIN3 NREN/partner (transit network) sides (continued) KOREN On KOREN border router peering with TEIN3 HK PoP, all routes, tagged with koren-ren or koren-apan-jp (2 types of communities totally), should be announced to TEIN3 HK PoP. In order to take KOREN as a backup transit network for TEIN3 backbone’s access to APAN-JP and North American networks, on KOREN border router peering with APAN-JP, all routes advertised from APAN-JP, which are tagged with 7 types of communities aforementioned in section 3.4.2, should be accepted and tagged with koren-apan-jp, then advertised to TEIN3 HK PoP; in the meantime, all routes learned from TEIN3 HK PoP, which are tagged with tein-nrn, should be advertised to APAN-JP. In order to prevent APAN-JP and North American networks from selecting KOREN as preferred transit networks to TEIN3 NRENs, on KOREN border router peering with APAN-JP, 1 time of ASN should be prepended on routes tagged with tein-nrn, before advertised to APAN-JP. In order to take APAN-JP as a backup transit network for KOREN’s access to TEIN3 backbone, in case of HK-KR link outage, on KOREN border router peering with APAN-JP, all the routes originated from KOREN R&E networks, which are tagged with koren-ren, should be announced to APAN-JP; in the meantime, all routes learned from APAN-JP, which are tagged with tein-backbone, should be accepted.

Suggestions to Routing Policy on TEIN3 NREN/partner (transit network) sides (continued) CNGI-6IX On CNGI-6IX border router peering with TEIN3 BJ PoP, all routes, tagged with cngi-6ix-ren or cngi-6ix-cst or cngi-geant (3 types of communities totally), should be announced to TEIN3 BJ PoP. In order to take CNGI-6IX as the backup transit network for TEIN3 NREN’s access to GEANT, on CNGI-6IX border router peering with GEANT, all routes learned from GEANT should be accepted and tagged with cngi-geant, then announced to TEIN3 BJ PoP (2 times AS-prepending); in the meantime, all routes learned from TEIN3 BJ PoP, which are tagged with tein-backbone, should be announced to GEANT (2 times AS-prepending).

Suggestions to Routing Policy on TEIN3 NREN/partner (transit network) sides (continued) SingAREN On SingAREN border router peering with TEIN3 SG PoP, all routes, tagged with singaren-ren or singaren-bii or singaren-ihpc or singaren-ntu or singaren-nus or singaren-apan-jp (6 types of communities totally), should be announced to TEIN3 SG PoP. In order to take SingAREN as a backup transit network for TEIN3 backbone’s access to APAN-JP and North American networks, on SingAREN border router peering with APAN-JP, all routes advertised from APAN-JP, which are tagged with 5 types of communities aforementioned in section 3.4.3, should be accepted and tagged with singaren-apan-jp, then advertised to TEIN3 SG PoP; in the meantime, all routes learned from TEIN3 SG PoP, which are tagged with tein-nrn, should be advertised to APAN-JP. In order to prevent APAN-JP and North American networks from selecting SingAREN as preferred transit networks to TEIN3 NRENs, on SingAREN border router peering with APAN-JP, 1 time of ASN should be prepended on routes tagged with tein-nrn, before advertised to APAN-JP.

Suggestions to Routing Policy on TEIN3 NREN (non-transit networks) sides Accept all the routes advertised from TEIN3 PoP routers, which are tagged with tein-backbone or tein-no-exp, and set higher priority for the routes tagged with tein-self Avoid the leaking of inappropriate routes into TEIN3 backbone, e.g. routes of commercial networks Avoid misoperation on community tagging for BGP routes Estimate and provide appropriate maximum number of prefixes (IPv4/v6) to TEIN3 NOC for limitation on prefix number learned from NREN at PoP router For routes learned from TEIN3 backbone, which are tagged with tein-no-exp, avoid announcing them to external peers outside NREN and NREN’s customer networks For routes learned from TEIN3 backbone, which are tagged with tein-low-tran, set them with lower priority (local-preference in BGP) For routes learned from TEIN3 backbone, which are tagged with tein-high-tran, set them with higher priority (local-preference in BGP)

Illustrations for traffic between TEIN3 NRENs and GEANT/EU & Internet2/US Path Selection of Traffic from TEIN3 NRENs to GEANT/EU & Internet2/US

Illustrations for traffic between TEIN3 NRENs and GEANT/EU & Internet2/US Path Selection of Traffic from GEANT/EU & Internet2/US to TEIN3 NRENs

Suggestions on Advanced Routing Implementation Suggestions on inter-domain multicast implementation Suggestions on inter-domain MPLS implementation

Suggestions on inter-domain multicast implementation (IPv4) Enable PIM-SM IPv4 on interface of border router connecting with TEIN3 PoP Disable IPv4 PIM BSR flooding on border router peering with TEIN3 backbone Bring up MSDP peering with TEIN3 PoP Need collaboration between TEIN3 NOC and NREN/partner Provide TEIN3 NOC with the IP address for MSDP peering on NREN/partner side TEIN3-North(AS24489) HK PoP: 202.179.240.3 TEIN3-South(AS24490) SG PoP: 202.179.248.2 MD5 authentication (if possible) Same as the one for BGP peering Apply MSDP SA filter Enable BGP IPv4 multicast peering with TEIN3 PoP If NREN/partner wants multicast traffic to go through different ingress/egress from unicast traffic If NREN/partner has a topology with partial PIM configuration If MBGP is the must of inter-AS RPF checking The same community-tagging operation should be applied on MBGP routes as what is done for BGP unicast peering Because TEIN3 PoP currently deploy the same routing policy for MBGP routes as the one for BGP unicast peering

Suggestions on inter-domain multicast implementation (IPv6) Enable PIM-SM IPv6 on interface of border router connecting with TEIN3 PoP Disable IPv6 PIM BSR flooding on border router peering with TEIN3 backbone Enable embedded-RP If NREN/partner wants to support ASM Enable BGP IPv6 multicast peering with TEIN3 PoP If NREN/partner wants multicast traffic to go through different ingress/egress from unicast traffic If NREN/partner has a topology with partial PIM configuration If MBGP is the must of inter-AS RPF checking The same community-tagging operation should be applied on MBGP routes as what is done for BGP unicast peering Because TEIN3 PoP currently deploy the same routing policy for MBGP routes as the one for BGP unicast peering

Suggestions on inter-domain MPLS implementation NREN/partner border router is the ASBR for inter-AS MPLS deployment Enable intra-AS MPLS P/PE: MPLS + LDP PE-ASBR (NREN/partner border router): iBGP with labeled-unicast enabled Enable MPLS on interface of border router connecting with TEIN3 PoP Enable exchanging of IPv4 labeled route in eBGP session Juniper router family labeled-unicast inet.3 Cisco router neighbor a.b.c.d send-label (IOS) allocate-label & address-family ipv4 labeled-unicast (IOS-XR) Allow exchanging /32 routes for P/PE inside/outside NREN/partner AS Remember to announce ASBR's /32 route, if the ASBR is also acting as a PE The same community-tagging operation should be applied on labeled routes as what is done for BGP unicast peering Because TEIN3 PoP currently deploy the same routing policy for MBGP routes as the one for BGP unicast peering   Note: The Option-C of inter-AS MPLS deployment is the preferred solution for TEIN3 networks, but not the must. For example, the LSP stitching might be another solution, especially for inter-AS MPLS L2VPN

Configuration Examples for NRENs Notes Exemplary topology for special cases BGP policy for CNGI-6IX BGP policy for KOREN BGP policy for SingAREN Case study on community-signalling

Notes In order to facilitate TEIN3 NRENs’ routing design, here list some more complicated configuration examples for several special cases, which are mainly community-oriented. Next slide show a complicated scenario with CNGI-6IX, KOREN and SingAREN inside as exemplary NRENs for special cases (the topology in the figure may be different from actual situation). Note that the examples are mainly focused on outbound traffic control, i.e. the traffic from NREN to its peer networks. As for the inbound traffic control, i.e. the traffic from peer networks to NREN, NREN may need to announce more specific routes, exploit community-signalling, or may need the collaboration from external peer networks on reverse route tuning, e.g. AS-prepending or local-preference adjustment.

Exemplary topology for special cases

BGP policy for CNGI-6IX In the figure, besides the connection to TEIN3 BJ PoP, CNGI-6IX also has dedicated circuits to GEANT, Internet2, APAN-JP and KOREN. Suppose CNGI-6IX wants to access Internet2 and APAN-JP through dedicated links, then CNGI-6IX router needs no extra configuration for that, because of the shortest AS path. Suppose CNGI-6IX wants to access GEANT via TEIN3 backbone, then CNGI-6IX router should raise the local-preference of routes learned from TEIN3 BJ PoP with community geant-nrn or geant-eumed or geant-tenet tagged. Suppose CNGI-6IX wants to access all of the TEIN3 NRENs, including KOREN, through TEIN3 backbone, what CNGI-6IX router should do is just to raise the local-preference of routes learned from TEIN3 BJ PoP with community tein-nrn tagged.

BGP policy for KOREN In the figure, other than the link to TEIN3 HK PoP, KOREN also has dedicated circuits to Internet2, APAN-JP and CNGI-6IX. Suppose KOREN wants to access Internet2 and APAN-JP through TEIN3 backbone, i.e. via TEIN3 HK PoP, then KOREN router needs to raise the local-preference of routes learned from TEIN3 HK PoP with community tein-apan-jp tagged. Note that although routes with community tein-apan-jp tagged might also include routes to GEANT via APAN-JP - TransPAC2 - Internet2 path, but the outbound traffic from KOREN to GEANT will not follow that way, as the local-preference in KOREN can’t affect the normal routing on TEIN3 HK PoP, that is, the traffic from KOREN to GEANT will go through the link between TEIN3 HK PoP and GEANT. Suppose KOREN wants to access CNGI-6IX via dedicated circuit, then KOREN router needs do nothing but following the default BGP route selection, i.e. routes with shortest AS path are preferred.

BGP policy for SingAREN In the figure, besides the circuit to TEIN3 SG PoP, SingAREN also has dedicated link to Internet2. Suppose SingAREN wants to access Internet2 through dedicated link and access APAN-JP via Internet2, then what SingAREN router needs to do is just to lower the local-preference of routes learned from TEIN3 SG PoP with community tein-apan-jp tagged.

Case study on Community Signalling Due to the requirement from particular application for high bandwidth between AARNet and GEANT (traffic from GEANT to AARNet), AARNet wants GEANT (AS20965) to take Internet2 (AS11537) as preferred transit network (all 10G links along GEANT-Internet2-AARNet path) for the traffic targeted to specific prefixes of AARNet, so as to avoid possible bottleneck on AARNet SG-Perth 622M link on GEANT-TEIN3-AARNet path So, AARNet exploits community-signalling to have TEIN3 backbone apply 1 time of AS number prepending automatically for specific prefixes of AARNet before advertising to GEANT

Case study for Community Signalling (continued) AARNet tags peer-prep-1-spec community for GEANT (65301:20965) for specific prefixes before announcing to TEIN3 SG PoP SG-PoP> show route receive-protocol bgp 202.179.249.62 114.30.64.0/21 extensive * 114.30.64.0/21 (2 entries, 1 announced) Nexthop: 202.179.249.62 MED: 145 AS path: 7575 I AS path: Recorded Communities: 7575:1000 7575:2203 7575:3003 7575:5021 7660:160 11537:40 65301:20965

Case study on Community Signalling (continued) TEIN3 PoPs apply 1 time of AS number prepending for the specific prefixes tagged with peer-prep-1-spec community (65301:20965) automatically before advertising to GEANT MB-PoP> show route advertising-protocol bgp 202.179.249.86 114.30.64.0/21 extensive * 114.30.64.0/21 (1 entry, 1 announced) BGP group ebgp4-eu-mad type External Nexthop: Self AS path: 24490 [24490] 7575 I Communities: 7575:1000 7575:2203 7575:3003 7575:5021 7660:160 11537:40 24489:65155 24489:65200 65301:20965 BJ-PoP> show route advertising-protocol bgp 202.179.241.42 114.30.64.0/21 extensive BGP group ebgp4-eu-tn type External AS path: 24489 [24489] 24490 7575 I

Case study for Community Signalling (continued) GEANT does not take TEIN3 backbone as preferred transit network for the specific prefixes any more due to longer AS path, and chooses Internet2 instead. GEANT LG output on rt1.mad.es.geant2.net 114.30.64.0/21     *[BGP/170] 5d 04:05:57, MED 3790, localpref 100, from 62.40.114.7                       AS path: 11537 7575 I                     > to 62.40.112.25 via so-2/0/0.0                     [BGP/170] 10:35:36, localpref 100                       AS path: 24490 24490 7575 I                     > to 202.179.249.85 via so-0/2/0.0 GEANT LG output on rt1.cop.dk.geant2.net 114.30.64.0/21     *[BGP/170] 5d 04:38:35, MED 3790, localpref 100, from 62.40.114.7                       AS path: 11537 7575 I                     > to 62.40.112.49 via so-1/1/0.0                     [BGP/170] 1w1d 22:36:38, localpref 100                       AS path: 24489 24489 24490 7575 I                     > to 202.179.241.41 via so-2/1/3.0

Q & A