Experiences with Implementing MPLS/VPN Services Philip Bridge Nextra (Schweiz) AG 18.10.2000

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 Some Paraphrases ‘For the first time in the history of the Internet the transmission people are giving the network people more bandwidth than they know what to do with’ Peter Lothberg ‘If you aren’t scared, you don’t understand’ Mike O’Dell 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 MPLS-based VPN IP Backbone ‘Private’ Internet Layer-2 Switching IP Routing MPLS/VPN technology creates a kind of ‘Private Internet’ for each customer inside the NextraNet backbone. This ‘Private Internet’ behaves in the same way as the public Internet. All IP-based applications work in exactly the same way. What works across the public Internet will work inside an MPLS VPN 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 TEST

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 MPLS Label Switch Router (LSR) Provider Edge Router (PE) 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 MPLS IP Routing creates consistent, network-wide Routing Tables LSR IP Routing Protocol (OSPF, IS-IS) PE 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 MPLS Label Distribution Protocol runs in parallel to IP routing LSRs use LDP to swap IP Route-to-Label bindings Creates Label Forwarding Tables Label Distribution Protocol 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 MPLS LDP & IP Routing create Label Switch Paths between PE routers Local label Remote label IP prefix o/p interface Local label Remote label IP prefix o/p interface 3 5 3.3.3.3 C 5 X 3.3.3.3 A X 3 10.0/16 A 2) I reach 3.3.3.3 via int C, label 5 LSP Local label Remote label IP prefix o/p interface X 3 3.3.3.3 F X 3 10.0/16 A C 3.3.3.3 A F 4) I reach 3.3.3.3 via int F, label 3 3) To reach 3.3.3.3 via me use label 3 1) To reach 3.3.3.3 via me use label 5 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 MPLS PE routers ‘encapsulate’ IP packet with Label header Works an any layer-2 (Ethernet, WAN link, ATM…) Local label Remote label IP prefix o/p interface Local label Remote label IP prefix o/p interface 3 5 3.3.3.3 C 5 X 3.3.3.3 A X 3 10.0/16 A Local label Remote label IP prefix o/p interface 5 3.3.3.3 X 3 3.3.3.3 F X 3 10.0/16 A C 3.3.3.3 A F Label IP DA IP Data 3 3.3.3.3 IP Packet 3.3.3.3 IP Packet 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 MPLS Normal IP ‘connectionless’ routing builds layer-2 LSP ‘circuits’! LSPs take same path that would be taken by IP-based forwarding 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

MPLS & Network Scalability LDP & OSPF IP routing build LSPs between PE routers 2) To reach 3.3.3.3 via me use label 3 1) To reach 3.3.3.3 via me use label 5 3) I reach 3.3.3.3 via int F, label 3 C A F 3.3.3.3 2.2.2.2 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

MPLS & Network Scalability PE-PE LSPs used to build BGP session between PE routers Core LSRs do not have any BGP routing LSP between BGP endpoints C A F 3.3.3.3 BGP Session 2.2.2.2 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

MPLS & Network Scalability BGP tells each PE which remote PE for each external route Recursive lookup in the routing table to find a route to the remote PE Route to the remote PE is actually a LSP LSP between BGP endpoints 5) BGP neighbor 3.3.3.3 has a route to 10.0/16... C 10.0/16 A F 3.3.3.3 BGP Session 2.2.2.2 6) …so to reach 10.0/16 I use int F, label 3 4) BGP: I have a route to 10.0/16 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

MPLS & Network Scalability BGP route exchange between PEs, no BGP in Core LSPs only established between BGP session endpoints A few LSPs carry 1000’s of Edge routes between PEs Complex routing can be ‘pushed out’ of the Core to the Edges 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 MPLS & VPNs BGP Edge Routes are hidden from the Core IP addresses of data packets are hidden from the Core Overlapping ‘address families’ can share the Core…VPNs! 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 Virtual Routers PE-PE BGP hides Customer routes from Core MPLS hides IP addresses of packets from Core But routes and and addresses are still visible in PE routing table! 10.0/16 B C F 3.3.3.3 10.0/16 BGP Session A 2.2.2.2 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 Virtual Routers Solution is to assign a ‘Virtual Router to each Customer port Routing Tables in Virtual Routers are invisible to each other Cisco name: Virtual Routing and Forwarding Instance (VRF) 10.0/16 B C F 3.3.3.3 A 10.0/16 BGP Session 2.2.2.2 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 Layer 2 Labels Extend BGP to carry L2 labels and routes PE uses 2nd Level Label (L2) to distinguish between VPNs 1) BGP:To reach 10.0/16 via me use L2 10 2) BGP: To reach 10.0/16 via me use L2 12 3) To reach 10.0/16 via 3.3.3.3 I use L2 10 4) To reach Next Hop 3.3.3.3 I use int F, L1 label 3 10.0/16 B C F 3.3.3.3 A 10.0/16 BGP Session 2.2.2.2 5) …so to reach 10.0/16 I use int F, L1 label 3, L2 label 10 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 Layer 2 Labels 1st Level Label gets packets to correct destination PE Destination PE strips 1st Level Label (L1) from incoming packets Destination PE uses 2nd Level Label (L2) to forward incoming packets to correct VRF 1) Packets arriving with L2 10 are for red VRF 2) Packets arriving with L2 12 are for blue VRF 10.0/16 B C F 3.3.3.3 A 10.0/16 BGP Session 2.2.2.2 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 VPN Forwarding Packet arrives on interface E = red VRF PE looks up route to 10.0.0.1 in VRF Routing Table BGP route from 3.3.3.3 gives L2 = 10 Recursive lookup on Next Hop 3.3.3.3 gives L1 = 3 Packet label switched through Core to 3.3.3.3 L1 removed L2 tells PE router to treat packet according to red VRF L2 removed, packet forwarded out of interface A 3 10 10.0.0.1 IP Packet C 10 10.0.0.1 IP Packet 3.3.3.3 L1 L2 IP DA IP Data A F 3 10 10.0.0.1 IP Packet 10.0.0.1 IP Packet E 10 10.0.0.1 IP Packet 10.0.0.1 IP Packet 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 MPLS & VPNs MPLS VPNs are like ‘Private Internets’ Internet protocols work within the VPN Easy to understand - similar to Frame-Relay Compliments Tunnel-based (IPSec) VPNs 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 IP Backbone Sharing ‘Private Internet’ IP Backbone ‘Private’ Internet ‘Private’ Internet Many Customer VPNs share the same NextraNet backbone The VPNs are totally separate…they are invisible to each other. Traffic cannot move from one VPN to another…unless explicitly configured to do so as part of a Customer solution. Totally private and secure. 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 TEST

Frame Relay Comparison The NextraNet VPN solution is similar to frame-relay: the set of ‘circuits’ belonging to each customer share a common frame-relay network, but they are totally invisible to each other. The advantage of a VPN is that it is connectionless, so the solutions are far more scalable and much easier to implement. Lower operating costs and more flexibility. Quicker, cheaper services for the Customer. 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 TEST

Frame Relay Comparison ‘Private’ Internet The NextraNet VPN solution is similar to frame-relay: the set of ‘circuits’ belonging to each customer share a common frame-relay network, but they are totally invisible to each other. The advantage of a VPN is that it is connectionless, so the solutions are far more scalable and much easier to implement. Lower operating costs and more flexibility. Quicker, cheaper services for the Customer. 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 TEST

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 MPLS & VPNs Customers A and B want their own VPNs, and an ExtraNet Customer A wants to connect his VPN to the Internet Customer B does not trust security of Customer A... Internet Service VPN-B Extra Net VPN-A 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Experience with MPLS VPNs Customer VPNs in service for >1 year. Several dozen VPNs Sizes between 2 and dozens of sites Average size of ca. 10 sites. 50:50 mixture of managed/unmanaged CPE Many VPNs have fixed and Dial-Up Access 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Experience with MPLS VPNs Many VPNs are combined with Internet Access Service, and a large proportion of these with managed Firewall services. From the Customer perspective, an MPLS VPN ‘looks’ different from a classical IP network. This has to be explained. strange traceroutes discontiguous routing domains 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Experience with MPLS VPNs MPLS labels increase the length of a packet. Can be a problem with Ethernet equipment from some vendors. Was a big problem for us at the beginning. Is still a problem for IPsec VPNs Equipment vendor MPLS/VPN implementations are reliable. Still some small bugs and missing features, but nothing that can’t be worked around 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Experience with MPLS VPNs MPLS/VPN is a very powerful paradigm for building an infrastructure that can deliver rich set of Services very flexibly and very rapidly It is a simple concept, but there are so many implementation possibilities that complexity can (very) easily can get out of control 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Experience with MPLS VPNs Tools to properly manage VPNs are still lagging way behind the network functionality Available tools restrict the ability of a Service Provider to innovate and develop solutions that are unique GUIs for provisioning are OK, but the real problem is fault resolution…about 5-10 times more difficult to resolve problems than normal IP-based ISP networks 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Experience with MPLS VPNs It is crucial to impose a structured, modular Service model onto the VPN architecture from the beginning Helps Salesmen and Customers to understand what can and cannot be done Helps implementation team to configure the solution Helps NOC to trouble-shoot Reduces load on 3rd level pre-sales & support 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999 Next Challenges Integration of MPLS/VPN and DiffServ QoS Multi-provider/Multi-AS extension of MPLS/VPN based Services Traffic Engineering 5/23/2019 Doc ID: TE-Q-1.01.06-E-17 / Version 1.0 / Release: 28.10.1999

Thank You. http://www. swinog. ch/swinog1/presentations Thank You! http://www.swinog.ch/swinog1/presentations.html pbridge@nextra.ch