1. Evolution Network SEAMLESS NETWORK EVOLUTION AT COLT APRICOT 2018
Amit Dhamija
APRICOT 2018
Kathmandu, Nepal
Amit Dhamija
Network Engineering

2. Agenda Colt’s Legacy Network
Colt’s VISION for One global converged packet network
Network design evolution towards NextGen technologies
Experiences and Key Takeaways

3. IP Network Ethernet Network
COLT’S LEGACY NETWORK
IGP - ISIS
MPLS-LDP
BGP – AS 8220
IGP- OSPF
MPLS – RSVP/TE
T-LDP – PW
IP Network
Ethernet Network
Services:
Internet access
MPLS L3 VPN’s (unicast and multicast)
Services:
E P2P
Etree
ELAN

4. Colt’s VISION: One global converged packet network
The integration of the network from the service, technology, system and process point of view to offer innovative services, reduce provisioning time, automate end-to-end orchestration and offer truly combined network services.
Converged Network
Service flexibility
Simplified Design & Architecture
Resiliency
Scalability
virtualisation

5. Network design evolution towards NextGen technologies

6. Infrastructure Protocols
New Network At a Glance
E2E Converged Network
(Ethernet / IP / Multicast Services)
Infrastructure Protocols
Services
ISIS – IGP
MPLS-SR
RSVP
BFD etc.
EVPN – P2P/E-LAN
IP Services – L3VPN/Internet access
M-VPN – NG multicast VPN’s

7. NextGen MPLS Evolution

8. Legacy Network – MPLS Design
MPLS-LDP
IP Network
MPLS-RSVP
Ethernet
Network
PE-X
PE-Y
PE-X
PE-Y
MPLS-LDP is used as the control plane protocol for IP Network.
MPLS applications – L3 VPN’s/6PE/6VPE.
Shortest IGP Path cost.
Fast convergence.
ECMP Paths.
MPLS-RSVP is used as control plane protocol for Ethernet Network.
MPLS application – VPWS/VPLS
Traffic engineering with explicit paths.
Disjoint paths.
Fast convergence.

9. Challenges with existing MPLS Design
MPLS LDP
MPLS RSVP
LDP-FRR coverage issue.
LDP generates lots of needless labels and is inefficient.
RSVP traffic engineering LSP’s scalability issues.
RSVP traffic engineering ECMP issues.

10. New Network – MPLS Design (Segment Routing)
Why We Chose Segment Routing
Simplicity – Single MPLS design as standard.
MPLS SR supports both modes under one network – (SR-SPT and SR-TE).
Solves the ECMP for traffic engineering by using (Node SID & Adj-SID) algorithm.
Simplifies the control plane for COLT’s new network – solves the RSVP scaling issues/no dedicated MPLS protocols.
Convergence – 100% protection with any topology.
Interoperability with existing MPLS Protocols.
Supports all the existing services. .

11. Segment Routing Control Plane
COLT
MPLS Segment Routed Network
PE-W
PE-N
PE-S
PE-E
Deployed using IPv4 transport with MPLS data plane.
Label Allocation – Only loopbacks with Index ID.
SRGB – 100K.
Range – 65K label space per node.
router isis COLT
net
interface Loopback0
passive
circuit-type level-2-only
address-family ipv4 unicast
prefix-sid index 4142 explicit-null

12. Segment Routing – TI LFA Coverage
Prefixes reachable in L2 Critical High Medium Low Total
Priority Priority Priority Priority
All paths protected
Some paths protected
Unprotected
Protection coverage % % % % %

13. Segment Routing Integration with IP/LDP Network
AdvertiseSR labels for legacy LDP PE Nodes
ISIS / MPLS-SR
ISIS /
MPLS-LDP PE
LDP
Mapping
Servers
PE-N
PE-W
PE-E
IP Network
PE-S
New-Network
Mapping Severs – In path for our deployment.
Features working with this deployment: L3 VPN’s/Internet access/Unicast-v4/v6/QOS etc.
Our use case for Integration: LDP over SR / LDP-SR & SR-LDP.

14. Segment Routing Traffic Engineering – SR-TE
PCEP for
tunnel req & label imposition
BGP-LS for topology info
LSP Setup/ Tear down(On-Demand)
MPLS SR
PE-1
PE-2
Centralized statefull PCE
PCEP
SR-TED
PCE
LSPDB Synch
TED Synch
NSO
Ongoing evaluation in our engineering lab.
SR-TE features - Disjoint paths with Node/Link and SRLG with static and dynamic paths.

15. NextGen Services Evolution

16. Legacy Network – Services Design
PE-Z
PE-X
PE-Y
IRR
VRR
T-LDP
PE-X
PE-Y
BGP/MP-BGP
IP Network
L2-PE
L2-PE
G.8032/ERP
G.8032/ERP
L2-PE
L2-PE
BGP is used as the control plane protocol for IP-Services.
Internet access – IPv4/IPv6(6PE)
Layer 3 VPN’s – VPNv4 and VPNv6(6VPE)
Deployed using RR’s.
ERP in the access rings and T-LDP sessions in the core ring for Ethernet Services.
E-P2P – MPLS-VPWS (Psuedowires)
ELAN – MPLS-VPLS (Mesh of Pseudowires)
E-TREE (Psuedowires based)

17. Challenges with existing Service Design
Different implementation methods – Layer 3 & Layer 2 VPN’s.
Complicated provisioning and troubleshooting.
Layer 2 VPN’s control and data plane are mixed/flooding issues.
Full mesh of pesudowires required in core network/scalability issues.
Multihomed customers no support of active-active redundancy.

18. New Network – Services Design (BGP)
Why We Chose EVPN Technology
IRR
Simplicity - Single BGP protocol as the standard for all the services in network.
IP and MAC distribution by using “BGP”.
No use of pseudowires for L2 VPN’s.
Faster convergence for all BGP based services.
Ease of configuration, operations and manageability.
PR’s
BGP Control Plane
ERR
PE-2
PE-3
ERR
BGP- Control Plane
BGP- Control Plane
AR-1
AR-2
AR-1
AR-2

19. Ethernet Services (EVPN) P2P Services – MP-BGP
COLT
EVPN VPWS
AR-2
AR-1
Single BGP Ethernet AD Route containing [RD, RT, ESI, Label(X),Eth-Tag ID(AC)]
EVPN NLRI AFI=25/SAFI=70
Show bgp l2vpn evpn
Route Distinguisher: 1111:1111 (default for vrf VPWS:1111)
*>i[1][ ][111]/120 I
*> [1][ ][222]/120 i
Next Gen solution for Ethernet VPN’s.
EVPN VPWS – Data plane uses MPLS-SR & Control plane – BGP based.
Simple to deploy and scalable solution.

20. Ethernet Services (EVPN) E-LAN Services – MP-BGP
COLT
EVPN MPLS
AR-3
AR-1
Four Routes - ELAN
Ethernet AD
Ethernet MAC
ESI Route
Inclusive multicast route
AR-2
EVPN NLRI AFI=25/SAFI=70
Show bgp l2vpn evpn
Route Distinguisher: :1
*>i[1][ :1][ acce ][ ]/184 i
*>i[1][000a.5b7e ][0]/120
*>i[3][0][32][ ]/80 I
*>i[2][0][48][ ][0]/104
EVPN-MPLS is similar to L3VPN MPLS.
Known Unicast – MPLS-SR as transport & BUM – Ingress Replication with MPLS-SR.
Rapid convergence – non zero ESI for single homed customers.

21. IP Services using PW-HE Feature – MP-BGP
PE-2
AR-1
PR’s
PE-3
AR-2
EVPN VPWS
MP-BGP
L3-VPN/GRT
PW-HE binds the L2 EVPN VPWS to the GRT or L3-VPN’s
PW-HE - VRF
xconnect group xxxxx
p2p xxxxx
interface PW-Ether1
neighbor evpn evi 787 target 200 source 100
PW-HE using MPLS-SR as transport – Unicast
PW-HE using MPLS-RSVP as transport - Multicast
Logical Interface(PW-HE)

22. NFV – Network function Virtualization

23. Colt’s Focus towards Network Function Virtualization
Virtualization various network infrastructure segments
RR / PCE Controller/ DDOS controllers etc.
We deploy on commodity x86 hardware & dedicated VM’s for each network function.
We run each instance as an independent island.
Using KVM Hypervisor for our deployment.
VNF
VNF
VNF
KVM-Hypervisor
x86 HW

24. Example - Route Reflector Virtualization (vRR)
Next Gen “BGP-RR” deployed for all the services by virtualization of control plane function.
Virtualized all the RR’s on KVM Hypervisor.
Dedicated VM’s for each RR per Server.
Centralized architecture design, each PE forms sessions with all three RR’s.
IRR-1
IRR-2
IRR-3
VPN-RR-1
VPN-RR-2
VPN-RR-3
E-RR-1
E-RR-2
E-RR-3
MPLS Layer 3
VPN’s RR
Ethernet RR
Server 1
Server 2
Server 3
Internet RR

25. Experiences and Key takeaways

26. Operations: Experiences from our Deployment
All MPLS Services are fully supported by Segment Routing.
L2 VPN’s / VPLS / EVPN / L3 VPN’s / IPv6.
LDP-SR Inter working full multi-vendor support (Cisco/Juniper & Nokia, in our case).
ISIS doesn’t advertise the merge flags when destination prefix looses the last backup path – Patch!
Inconsistency in forwarding entries in LC and RP due to TI-LFA bug – Patch!
No labels assignment for connected prefixes other then loopbacks on MPLS-SR ( SR feature Limitation).
Native multicast solution for SR not yet available (Tree-SID, BIER etc.)
Max SID Depth is a problem with SR-TE due to nature of source routing – Binding SID!
No Interop issue in EVPN between Cisco/Juniper & Nokia.
EVPN Flow based load balancing no support at this stage.
Hierarchical EVPN for large networks is still a challenge.

27. Key Takeaways Single Infrastructure for all services..
Simplified the transport and service architecture.
Reduce the cost – opex and capex.
Faster Service delivery.
Better Orchestrator.
It’s good to be an early investor in any new technology.
Benefits Realized with Next Gen MPLS technologies – simplified design, deployment & operations.
Next Gen MPLS technologies proved easy to deploy, maintain and use.
Questions?

28. For your time
Thank you