1. Terena Networking Conference, June 2005, Poznan IPv6 Deployment Challenges on the SEEREN Infrastructure (… or 6PE over CSC VPNs) presented by Andreas Polyrakis National Technical University of Athens Athanassios Liakopoulos, Constantinos Kotsokalis, Jorge-A. Sanchez-P. Greek Research & Technology Network (GRNET) Dimitrios Kalogeras, Andreas Polyrakis National Technical University of Athens (NTUA) Gunter Van de Velde Cisco Systems

2. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 Presentation Outline The SEEREN Network  Connected NRENs  Physical & Network Topology – the CSC solution  IPv6 Deployment Scenarios Introduction to CSC and 6PE  MPLS & MPLS VPNs short introduction  What is CSC?  What is 6PE? 6PE over CSC - The SEEREN Case  The Idea: Combining CSC and 6PE  How it works - Label Distribution and Packet Forwarding  Configuration & Troubleshooting Conclusions

3. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 Presentation Outline The SEEREN Network  Connected NRENs  Physical & Network Topology – the CSC solution  IPv6 Deployment Scenarios Introduction to CSC and 6PE  MPLS & MPLS VPNs short introduction  What is CSC?  What is 6PE? 6PE over CSC - The SEEREN Case  The Idea: Combining CSC and 6PE  How it works - Label Distribution and Packet Forwarding  Configuration & Troubleshooting Conclusions

4. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 The SEEREN Network SE Europe NRENs:  GRNET (Greece)  AMREJ (Serbia & Montenegro)  ISTF (Bulgaria)  MARNET (FYR of Macedonia)  INIMA (Albania)  BIHARNET (Bosnia - Herzegovina) Upstream: GRNET Backup Upstream: RoEduNet (via Athens’ POP ) South Eastern European Research & Educational Network (SEEREN v1 - Jan. ’04)

5. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 The SEEREN Topology Physical Topology: Local connections to an International Service Provider (OTEglobe)

6. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 The SEEREN Topology VPN Connectivity: via a Carrier-Supporting- Carrier (CSC) MPLS VPN  CSC: a special type of an MPLS VPN  CSC is Layer 3 Resulting L3 Topology: Full Mesh 

7. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 IPv6 in SEEREN Native IPv6: Ruled out by the topology  Carrier network was not dual stack   Even if it was, IPv6 is not supported at CSC VPNs  IPv6 over IPv4 tunnels: Last resort solution  Simple, tested and it does work!  Full mesh of tunnels are required, 2 n complexity  (but n=5 )  Some features are not supported, eg. IPv6 QoS  6PE: An MPLS-based technology that allows customers to exchange IPv6 traffic over an ipv4-only MPLS core network  Transparent to the carrier (no upgrades/configuration changes)  QoS would be possible through the MPLS EXP bits  Minor upgrades and configuration changes, small disruption to the production ipv4 network  6PE is indented for inter-domain use!   6PE over CSC???  Will the label exchange protocols work over different domains? Will label swapping and packet forwarding work?  Well, it does work! But it was not easy, lab setup (@cisco) was required to ensure that it works

8. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 Presentation Outline The SEEREN Network  Connected NRENs  Physical & Network Topology – the CSC solution  IPv6 Deployment Scenarios Introduction to CSC and 6PE  MPLS & MPLS VPNs short introduction  What is CSC?  What is 6PE? 6PE over CSC - The SEEREN Case  The Idea: Combining CSC and 6PE  How it works - Label Distribution and Packet Forwarding  Configuration & Troubleshooting Conclusions

9. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 X MPLS Cloud RaPE RbP RcP RdPE C2CE C1CE P Labels LDP Labels LDP Labels LDP MPLS Simplified Terminology  CE: Customer Edge Router (C1,C2)  PE: Provider Edge Router (Ra, Rd)  P: Provider Core Router (Rb, Rc) MPLS: Packet Forwarding based on a Label between L2 and L3 header Labels are created for all ipv4 IGP routes Labels are exchanged (with a Label Distribution Protocol – LDP or BGP) Packet Forwarding  One label is imposed on ingress based on destination IP  Packets are forwarded based on that label  Labels are swapped while packet is forwarded  Penultimate Hop (in most cases) performs Penultimate Hop Popping (PHP); in this case the last hop receives an IP packet  Last hop forwards the packet to the appropriate egress interface IP LrIPLt IP

10. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 MPLS VPNs Simplified VPN definition  Interfaces are declared to belong to the same VPN Label Distribution for VPNs  VPN Routes + VPN Label are exchanged between PEs  Protocol: MP-BGP (Multi-Protocol BGP) Packet Forwarding based on two labels imposed at the ingress point of the MPLS network  Inner: The label of the VPN route  Outer: The label towards the egress router Forwarding: Swap exterior label  Penultimate Hop Popping (PHP) Last Hop: Packet received with interior label only which identifies VPN & egress interface The last label is popped, the IP packet is forwarded to the CE router X MPLS Cloud RaPE RbP RcP RdPE C2CE C1CE P Labels LDP Labels LDP Labels LDP IP LvLrIPLvIPLvLtIPLv VPN Labels - MP-BGP

11. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 What is CsC? How it Works? CSC = Carrier-Supporting-Carrier Implements a L3 MPLS VPN Designed for ISPs that are VPN customers of other (larger MPLS) ISP.  A VPN with very small virtual routing table (VRF) MPLS between CsC-CE and CsC-PE  CsC-CEs exchange limited labels with CsC-PE How?  The CSC-CE receives routes+labels for reaching all the other CSC-CEs  A label is imposed between CsC-CE and CsC-PE  This label is swapped with the two VPN labels by the CSC-PE  The carrier only needs to maintain routing & label information about the CSC-CEs X MPLS Cloud RaPE RbP RcP RdPE C2CE C1CE P Labels LDP Labels LDP Labels LDP IP LvLrIPLvIPLvLtIPLv VPN Labels - MP-BGP CSC Labels Via LDP or BGP IPLc

12. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 What is 6PE? How it Works? An MPLS-based method that allows an MPLS- based ISPs to offer IPv6 interconnection services to their customers without upgrading the entire network to dual stack. 6PE is similar to MPLS VPNs model in terms of technical implementation and complexity.  An outer label for forwarding  An inner label that corresponds to the “IPv6 Routing Table” (instead of the “VPN routing table”) IPv6 Labels (+Routes) are exchanged again through MP-BGP  PEs are reffered to as 6PEs Differences with MPLS VPNs  Refers to v6, not v4  The IPv6 global table is exchanged – not a virtual or a private one (This is not an ipv6 Virtual Private Network!!!) Technical detail: IPv4 addresses are mapped to IPv6 for BGP next- hop X MPLS Cloud RaPE RbP RcP RdPE C2CE C1CE P Labels LDP Labels LDP Labels LDP IPv6 IPLvLrIPv6L6IPv6L6LtIPv6L6 IPv6 Labels - MP-BGP 6PE

13. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 Presentation Outline The SEEREN Network  Connected NRENs  Physical & Network Topology – the CSC solution  IPv6 Deployment Scenarios Introduction to CSC and 6PE  MPLS & MPLS VPNs short introduction  What is CSC?  What is 6PE? 6PE over CSC - The SEEREN Case  The Idea: Combining CSC and 6PE  How it works - Label Distribution and Packet Forwarding  Configuration & Troubleshooting Conclusions

14. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 The Idea + X MPLS Cloud RaPE RbP RcP RdPE C2CE C1CE P Labels LDP Labels LDP Labels LDP IPv6 IPLvLrIPv6L6IPv6L6LtIPv6L6 IPv6 Labels - MP-BGP X MPLS Cloud RaPE RbP RcP RdPE C2CE C1CE P Labels LDP Labels LDP Labels LDP IP LvLrIPLvIPLvLtIPLv VPN Labels - MP-BGP CSC Labels Via LDP or BGP IPLc CSC Labels Via LDP or BGP 6PE

15. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 The Idea = IPv6L6 IPv6 Labels - MP-BGP X MPLS Cloud RaPE RbP RcP RdPE C2CE C1CE P Labels LDP Labels LDP Labels LDP LvLcLv Lb Lv VPN Labels - MP-BGP CSC Labels Via LDP or BGP Lc CSC Labels Via LDP or BGP IPv6L6IPv6L6IPv6L6IPv6L6 !!! 6PE

16. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 6PE over CsC Integration of two techniques:  6PE functionality is installed on the CEs instead of the PEs!  6PE peers belong to different administrative domains.  Three Label Stack !!!  Feasible, because CE- PE connection uses MPLS. IPv6L6 IPv6 Labels - MP-BGP X MPLS Cloud RaPE RbP RcP RdPE C2CE C1CE P Labels LDP Labels LDP Labels LDP LvLcLv Lb Lv VPN Labels - MP-BGP CSC Labels Via LDP or BGP Lc CSC Labels Via LDP or BGP IPv6L6IPv6L6IPv6L6IPv6L6 6PE

17. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 SEEREN 6PE Configuration #1 hostname athens-2 ! ipv6 unicast-routing ipv6 cef ! interface Loopback0 ip address 194.177.210.40 255.255.255.255 ! interface ATM1/0/0.1 point-to-point description SEEREN via OteGlobe VPN ip address 62.75.33.246 255.255.255.252 ! interface GigabitEthernet3/0/0 description Athens2 - Athens3 ipv6 address 2001:648:2FFF:106::2/126 ipv6 router isis ! router isis … Enable IPv6 IPv4 Loopback: Necessary for multihop BGP IPv4 Link with OteGlobe IPv6 GRNET Network IPv6 Routing Protocol. Populates the IPv6 Routing Table

18. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 SEEREN 6PE Configuration #2 router bgp 5408 neighbor 62.75.33.245 remote-as 12713 neighbor 62.75.33.245 description OTEGLOBE PE neighbor 147.91.0.112 remote-as 13092 neighbor 147.91.0.112 description AMREJ-YUGOSLAVIA neighbor 147.91.0.112 ebgp-multihop 5 neighbor 147.91.0.112 update-source Loopback0 ! address-family ipv4 neighbor 62.75.33.245 activate neighbor 62.75.33.245 send-community neighbor 62.75.33.245 remove-private-as neighbor 62.75.33.245 soft-reconfiguration inbound neighbor 62.75.33.245 send-label neighbor 147.91.0.112 activate neighbor 147.91.0.112 send-community neighbor 147.91.0.112 remove-private-as neighbor 147.91.0.112 soft-reconfiguration inbound ! address-family ipv6 neighbor 147.91.0.112 activate neighbor 147.91.0.112 send-community neighbor 147.91.0.112 remove-private-as neighbor 147.91.0.112 soft-reconfiguration inbound neighbor 147.91.0.112 send-label exit-address-family MP-BGP 6PE: Send Labels for IPv6 routes (AMREJ) CSC: Send Labels for IPv4 routes. (OteGlobe)

19. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 6PE verification & troubleshooting athens-2#sh ip bgp nei 147.91.0.112 BGP neighbor is 147.91.0.112, remote AS 13092, external link Description: AMREJ-YUGOSLAVIA BGP version 4, remote router ID 147.91.0.112 BGP state = Established, up for 2d00h Last read 00:00:16, hold time is 90, keepalive interval is 30 seconds Configured hold time is 90, keepalive interval is 30 seconds Neighbor capabilities: Route refresh: advertised and received(new) Address family IPv4 Unicast: advertised and received Address family IPv6 Unicast: advertised and received ipv6 MPLS Label capability: advertised and received athens-2#sh ipv6 cef 2001:4170::/32 internal 2001:4170::/32 path list pointer 4355B5E0 1 path - Nexthop path_pointer 43558B80 traffic share 1 path_list pointer 4355B5E0 nexthop ::FFFF:147.91.0.112 next_hop_len 0 adjacency pointer 4351A6B8 refcount 2 no loadinfo fast tag rewrite with AT1/0/0.1, point2point, tags imposed: {17 30}

20. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 6PE verification & troubleshooting athens-2#sh ip cef 147.91.0.112 147.91.0.112/32, version 48051, epoch 0, cached adjacency to ATM1/0/0.1 0 packets, 0 bytes Flow: AS 12713, mask 32 tag information set, shared, all rewrites owned local tag: BGP route head fast tag rewrite with AT1/0/0.1, point2point, tags imposed: {18} via 62.75.33.245, 4 dependencies, recursive next hop 62.75.33.245, ATM1/0/0.1 via 62.75.33.245/32 valid cached adjacency tag rewrite with AT1/0/0.1, point2point, tags imposed: {17} athens-2#sh bgp ipv6 u labels | b 2001:4170:: 2001:4170::/32 ::FFFF:147.91.0.112 nolabel/30

21. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 6PE verification & troubleshooting athens-2#show bgp ipv6 unicast 2001:4170::/32 BGP routing table entry for 2001:4170::/32, version 2313 Paths: (1 available, best #1, table Global-IPv6-Table) Advertised to update-groups: 2 13092, (received & used) ::FFFF:147.91.0.112 from 147.91.0.112 (147.91.0.112) Origin IGP, metric 0, localpref 100, valid, external, best athens-2#sh ipv6 route 2001:4170:: IPv6 Routing Table - 474 entries Codes: C - Connected, L - Local, S - Static, R - RIP, B - BGP I1 - ISIS L1, I2 - ISIS L2, IA - ISIS interarea, IS - ISIS summary O - OSPF intra, OI - OSPF inter, OE1 - OSPF ext 1, OE2 - OSPF ext 2 B 2001:4170::/32 [20/0] via ::FFFF:147.91.0.112, IPv6-mpls

22. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 Presentation Outline The SEEREN Network  Connected NRENs  Physical & Network Topology – the CSC solution  IPv6 Deployment Scenarios Introduction to CSC and 6PE  MPLS & MPLS VPNs short introduction  What is CSC?  What is 6PE? 6PE over CSC - The SEEREN Case  The Idea: Combining CSC and 6PE  How it works - Label Distribution and Packet Forwarding  Configuration & Troubleshooting Conclusions

23. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 Conclusions Fact: CSC was chosen for SEEREN  For the Carrier ISP: Small virtual routing table  For the NRENs: Scalability and Flexibility IPv6: Two deployment alternatives: 6PE or IPv4 tunnels 6PE was more elegant, but it was never deployed before over CSC  Several difficulties, lab setup first to ensure that it works  Was only possible due to the MPLS existence among the 6PE peers (NRN border routers) 6PE is easy (after all) to deploy over CSC and it is flexible (eg, QoS)

24. GRNET / NTUA TNC 2005 Poznan, 6.6.2005 Questions? Thank You! Andreas Polyrakis A.Polyrakis@noc.ntua.gr Contact the Authors: Athanassios Liakopoulos, Constantinos Kotsokalis, Jorge-A. Sanchez-P., GRNET, {aliako, ckotso, sanchez}@grnet.gr Dimitrios Kalogeras, Andreas Polyrakis, NTUA, {D.Kalogeras, A.Polyrakis}@noc.ntua.gr Gunter Van de Velde, Cisco Systems, gvandeve@cisco.com