Evolution Network SEAMLESS NETWORK EVOLUTION AT COLT APRICOT 2018 Amit Dhamija APRICOT 2018 Kathmandu, Nepal Amit Dhamija Network Engineering amit.dhamija@colt.net | @AmitDhamijain

Agenda Colt’s Legacy Network Colt’s VISION for One global converged packet network Network design evolution towards NextGen technologies Experiences and Key Takeaways

IP Network Ethernet Network COLT’S LEGACY NETWORK IGP - ISIS MPLS-LDP BGP – AS 8220 IGP- OSPF MPLS – RSVP/TE T-LDP – PW IP Network Ethernet Network Services: Internet access MPLS L3 VPN’s (unicast and multicast) Services: E P2P Etree ELAN

Colt’s VISION: One global converged packet network The integration of the network from the service, technology, system and process point of view to offer innovative services, reduce provisioning time, automate end-to-end orchestration and offer truly combined network services. Converged Network Service flexibility Simplified Design & Architecture Resiliency Scalability virtualisation

Network design evolution towards NextGen technologies

Infrastructure Protocols New Network At a Glance E2E Converged Network (Ethernet / IP / Multicast Services) Infrastructure Protocols Services ISIS – IGP MPLS-SR RSVP BFD etc. EVPN – P2P/E-LAN IP Services – L3VPN/Internet access M-VPN – NG multicast VPN’s

NextGen MPLS Evolution

Legacy Network – MPLS Design MPLS-LDP IP Network MPLS-RSVP Ethernet Network PE-X PE-Y PE-X PE-Y MPLS-LDP is used as the control plane protocol for IP Network. MPLS applications – L3 VPN’s/6PE/6VPE. Shortest IGP Path cost. Fast convergence. ECMP Paths. MPLS-RSVP is used as control plane protocol for Ethernet Network. MPLS application – VPWS/VPLS Traffic engineering with explicit paths. Disjoint paths. Fast convergence.

Challenges with existing MPLS Design MPLS LDP MPLS RSVP LDP-FRR coverage issue. LDP generates lots of needless labels and is inefficient. RSVP traffic engineering LSP’s scalability issues. RSVP traffic engineering ECMP issues.

New Network – MPLS Design (Segment Routing) https://datatracker.ietf.org/doc/draft-ietf-spring-segment-routing/ Why We Chose Segment Routing Simplicity – Single MPLS design as standard. MPLS SR supports both modes under one network – (SR-SPT and SR-TE). Solves the ECMP for traffic engineering by using (Node SID & Adj-SID) algorithm. Simplifies the control plane for COLT’s new network – solves the RSVP scaling issues/no dedicated MPLS protocols. Convergence – 100% protection with any topology. Interoperability with existing MPLS Protocols. Supports all the existing services. .

Segment Routing Control Plane COLT MPLS Segment Routed Network PE-W PE-N PE-S PE-E Deployed using IPv4 transport with MPLS data plane. Label Allocation – Only loopbacks with Index ID. SRGB – 100K. Range – 65K label space per node. router isis COLT net 49.0001.1120.6409.4142.00 interface Loopback0 passive circuit-type level-2-only address-family ipv4 unicast prefix-sid index 4142 explicit-null

Segment Routing – TI LFA Coverage Prefixes reachable in L2 Critical High Medium Low Total Priority Priority Priority Priority All paths protected 0 0 735 1295 2030 Some paths protected 0 0 0 0 0 Unprotected 0 0 0 0 0 Protection coverage 0.00% 0.00% 100.00% 100.00% 100.00%

Segment Routing Integration with IP/LDP Network AdvertiseSR labels for legacy LDP PE Nodes ISIS / MPLS-SR ISIS / MPLS-LDP PE LDP Mapping Servers PE-N PE-W PE-E IP Network PE-S New-Network Mapping Severs – In path for our deployment. Features working with this deployment: L3 VPN’s/Internet access/Unicast-v4/v6/QOS etc. Our use case for Integration: LDP over SR / LDP-SR & SR-LDP.

Segment Routing Traffic Engineering – SR-TE https://tools.ietf.org/html/draft-filsfils-spring-segment-routing-policy-04 PCEP for tunnel req & label imposition BGP-LS for topology info LSP Setup/ Tear down(On-Demand) MPLS SR PE-1 PE-2 Centralized statefull PCE PCEP SR-TED PCE LSPDB Synch TED Synch NSO Ongoing evaluation in our engineering lab. SR-TE features - Disjoint paths with Node/Link and SRLG with static and dynamic paths.

NextGen Services Evolution

Legacy Network – Services Design PE-Z PE-X PE-Y IRR VRR T-LDP PE-X PE-Y BGP/MP-BGP IP Network L2-PE L2-PE G.8032/ERP G.8032/ERP L2-PE L2-PE BGP is used as the control plane protocol for IP-Services. Internet access – IPv4/IPv6(6PE) Layer 3 VPN’s – VPNv4 and VPNv6(6VPE) Deployed using RR’s. ERP in the access rings and T-LDP sessions in the core ring for Ethernet Services. E-P2P – MPLS-VPWS (Psuedowires) ELAN – MPLS-VPLS (Mesh of Pseudowires) E-TREE (Psuedowires based)

Challenges with existing Service Design Different implementation methods – Layer 3 & Layer 2 VPN’s. Complicated provisioning and troubleshooting. Layer 2 VPN’s control and data plane are mixed/flooding issues. Full mesh of pesudowires required in core network/scalability issues. Multihomed customers no support of active-active redundancy.

New Network – Services Design (BGP) Why We Chose EVPN Technology IRR Simplicity - Single BGP protocol as the standard for all the services in network. IP and MAC distribution by using “BGP”. No use of pseudowires for L2 VPN’s. Faster convergence for all BGP based services. Ease of configuration, operations and manageability. PR’s BGP Control Plane ERR PE-2 PE-3 ERR BGP- Control Plane BGP- Control Plane AR-1 AR-2 AR-1 AR-2

Ethernet Services (EVPN) P2P Services – MP-BGP https://tools.ietf.org/html/rfc8214 COLT EVPN VPWS AR-2 AR-1 Single BGP Ethernet AD Route containing [RD, RT, ESI, Label(X),Eth-Tag ID(AC)] EVPN NLRI AFI=25/SAFI=70 Show bgp l2vpn evpn Route Distinguisher: 1111:1111 (default for vrf VPWS:1111) *>i[1][0000.0000.0000.0000.0000][111]/120 10.91.126.111 100 0 I *> [1][0000.0000.0000.0000.0000][222]/120 0.0.0.0 0 i Next Gen solution for Ethernet VPN’s. EVPN VPWS – Data plane uses MPLS-SR & Control plane – BGP based. Simple to deploy and scalable solution.

Ethernet Services (EVPN) E-LAN Services – MP-BGP https://tools.ietf.org/html/rfc7432 COLT EVPN MPLS AR-3 AR-1 Four Routes - ELAN Ethernet AD Ethernet MAC ESI Route Inclusive multicast route AR-2 EVPN NLRI AFI=25/SAFI=70 Show bgp l2vpn evpn Route Distinguisher: 10.91.126.115:1 *>i[1][10.91.126.115:1][0000.0100.acce.5500.1401][4294967295]/184 10.91.126.115 100 0 i *>i[1][000a.5b7e.7300.0000.0000][0]/120 *>i[3][0][32][10.91.126.115]/80 10.91.126.115 100 0 I *>i[2][0][48][0000.0200.0001][0]/104 EVPN-MPLS is similar to L3VPN MPLS. Known Unicast – MPLS-SR as transport & BUM – Ingress Replication with MPLS-SR. Rapid convergence – non zero ESI for single homed customers.

IP Services using PW-HE Feature – MP-BGP PE-2 AR-1 PR’s PE-3 AR-2 EVPN VPWS MP-BGP L3-VPN/GRT PW-HE binds the L2 EVPN VPWS to the GRT or L3-VPN’s PW-HE - VRF xconnect group xxxxx p2p xxxxx interface PW-Ether1 neighbor evpn evi 787 target 200 source 100 PW-HE using MPLS-SR as transport – Unicast PW-HE using MPLS-RSVP as transport - Multicast Logical Interface(PW-HE)

NFV – Network function Virtualization

Colt’s Focus towards Network Function Virtualization Virtualization various network infrastructure segments RR / PCE Controller/ DDOS controllers etc. We deploy on commodity x86 hardware & dedicated VM’s for each network function. We run each instance as an independent island. Using KVM Hypervisor for our deployment. VNF VNF VNF KVM-Hypervisor x86 HW

Example - Route Reflector Virtualization (vRR) Next Gen “BGP-RR” deployed for all the services by virtualization of control plane function. Virtualized all the RR’s on KVM Hypervisor. Dedicated VM’s for each RR per Server. Centralized architecture design, each PE forms sessions with all three RR’s. IRR-1 IRR-2 IRR-3 VPN-RR-1 VPN-RR-2 VPN-RR-3 E-RR-1 E-RR-2 E-RR-3 MPLS Layer 3 VPN’s RR Ethernet RR Server 1 Server 2 Server 3 Internet RR

Experiences and Key takeaways

Operations: Experiences from our Deployment All MPLS Services are fully supported by Segment Routing. L2 VPN’s / VPLS / EVPN / L3 VPN’s / IPv6. LDP-SR Inter working full multi-vendor support (Cisco/Juniper & Nokia, in our case). ISIS doesn’t advertise the merge flags when destination prefix looses the last backup path – Patch! Inconsistency in forwarding entries in LC and RP due to TI-LFA bug – Patch! No labels assignment for connected prefixes other then loopbacks on MPLS-SR ( SR feature Limitation). Native multicast solution for SR not yet available (Tree-SID, BIER etc.) Max SID Depth is a problem with SR-TE due to nature of source routing – Binding SID! No Interop issue in EVPN between Cisco/Juniper & Nokia. EVPN Flow based load balancing no support at this stage. Hierarchical EVPN for large networks is still a challenge.

Key Takeaways Single Infrastructure for all services.. Simplified the transport and service architecture. Reduce the cost – opex and capex. Faster Service delivery. Better Orchestrator. It’s good to be an early investor in any new technology. Benefits Realized with Next Gen MPLS technologies – simplified design, deployment & operations. Next Gen MPLS technologies proved easy to deploy, maintain and use. Questions?

For your time Thank you amit.dhamija@colt.net