Networks ∙ Services ∙ People Guy Roberts Transport Network Architect, GÉANT TNC16 13 th June 2016 GÉANT Network, Infrastructure and Services Evolution

Networks ∙ Services ∙ People Why the need for change? Network Architecture SDN in Backbone 2 Agenda

Networks ∙ Services ∙ People Exponential Bandwidth Increase Highly unpredictable traffic No application awareness in the network hence no application-based path selection Integration with other e-Infrastrcture Improve network visibility and reduce operational costs Programmability, offer ability to use API to link applications to the network Support for virtual networks Cost Reduction 3 The need for change.

Networks ∙ Services ∙ People LHCONE Traffic increase over last 12 months 4 Bandwidth Increase

Networks ∙ Services ∙ People Current GÉANT Core network is utilizing fully functional routers For IP services, peering, packet forwarding, MPLS services, etc Deployed in sparse mesh (minimal router off-load) Backbone trunks follow fibre path Many packets transit through intermediate router hops Each transit packet consumes portion of the optical interface and forwarding capabilities Does this add any value? Transit through a backbone router incurs unnecessary costs Increases latency Increases utilisation of interfaces at IP layer 5 Core Network Architecture

Networks ∙ Services ∙ People IPoWDM (Thin Core) Router manages optical link Moves the cost of transponder/optics from optical layer equipment to the router Still a sparse mesh topology – unnecessary transit packets Hollow Core Mesh routers through OTN core GÉANT uses single routers in a PoP which performs functionality of both core and edge router In a 100G backbone full mesh would result in both increased complexity and costs 6 Evaluation of Network Architectures

Networks ∙ Services ∙ People IP over OTN Interconnect router with appropriate capacity trunk link Increase meshing between core routers Trunk links can be multiplexed by OTN layer into 500G super channel Transit traffic is kept at OTN layer instead of transiting through multiple routers Maximizes slot capacity at both IP and Optical layer Enables GÉANT to scale and converge the two networks Testing of Infinera PXM (OTN) Cards completed Will be rolled out in Eastern part of the network 7 Evaluation of Network Architectures

Networks ∙ Services ∙ People Current Architecture Follows the physical fibre Significant amount of transit traffic Requires two 100G interfaces per trunk Over provisioned Proposed Architecture IP Trunks based on traffic requirements between the two PoPs Significantly reduces the amount of transit traffic through routers Multiple trunk links can be setup on a single 100GE interface Reduces the amount of 100GE interfaces required on both IP and DWDM layer 8 Current and Proposed Architecture

Networks ∙ Services ∙ People Fully functional core routers require significant CAPEX and OPEX Incur significant operational, support and maintenance cost Router or PoP centric view instead of a network centric view Solving problem of traffic growth with overprovisioning Ensure network reliability and delivery of traffic despite failure No traffic differentiation Use protocol metrics to route based on shortest path (no consideration of link utilisation) 9 Why SDN in Backbone?

Networks ∙ Services ∙ People In GÉANT network the control plane and data plane separation somewhat already exists JunOS Space to Control and Manage Juniper equipment Infinera NMS to Control and Manage Infinera equipment Hundreds of features in these management systems Most of them not used by GÉANT Some features that GÉANT need don’t exists GÉANT can’t develop any new features for those management system Expensive and takes long time Vendor specific controller/management system If GÉANT equipment is changed the management system will also have to be changed 10 Control Plane and Data Plane

Networks ∙ Services ∙ People What we need? A high performance network which can handle exponential growth in traffic and large flows. Reduce the cost of the network while making it more efficient and fault tolerant without too much over provisioning Separation of hardware and software Buy hardware based on the feature we need Choose software based on the services and protocol we need to support 11 SDN in Backbone

Networks ∙ Services ∙ People SA1 and JRA2 have been working together on several use cases to transform the backbone: Programmable Substrate Vendor Agnostic Rapid Application Development Environment which support and encourages innovation 12 SDN in Backbone How are we planning on getting there? Network Operating System/Controller BoD App BGP App PCE App Forwarding Devices

Networks ∙ Services ∙ People 13 Requirements for SDN

Networks ∙ Services ∙ People Most of the scalability requirements aren’t supported by white-box solution based on ToR switch and single table OF implantation In a single table implementation for a 500-port switch to support 100k mac addresses 50 million entries will be required While a two table model would require only 100,500 entries Map the pipeline on the fly doesn’t work 14 Multi-Table Pipeline

Networks ∙ Services ∙ People Working with CORSA to evaluate their switches as replacement for routers Working with On.Lab to develop new applications like BoD and fixing SDN-IP Working with Infinera to develop the REST plugin for ONOS 15 Vendor Collaboration

Networks ∙ Services ∙ People Infinera is the current transmission equipment provider for GÉANT backbone Working closely with us to develop a solution for managing OTN (PXM) cards using an OpenFlow controller Paves the way to multi-layer interaction Use standard open APIs Enables us to bring more intelligence to the core network Reduce overprovisioning Reduce number of transit packets going through intermediate routers In future dynamically modify capacity on OTN layer to cater for peaky traffic on Layer 2 and 3 16 Why Infinera?

Networks ∙ Services ∙ People CORSA Support for 10/40/100GE Transceiver Agnostic Enhanced QoS (Deep buffers) OF 1.3 compliant and support multi-table pipeline Eager to work with us in developing the pipeline that support our use cases Working very closely to help us develop and enhance our use cases We have also helped by giving service provider requirements and help develop some features ON.Lab Supported far more features/requirements compared to other controller framework Showed eagerness to work with us 17 Why Corsa and ON.Lab?

Networks ∙ Services ∙ People Transport SDN Current network architecture GÉANT WDM/OTN NREN GÉANT Router NREN GÉANT Router GÉANT Router GÉANT Router ODU4/2e links

Networks ∙ Services ∙ People Transport SDN Revised architecture with PXM cards NREN GÉANT Router 100GbE NREN GÉANT Router 100GbE GÉANT Router GÉANT Router GÉANT WDM/OTN IP trunks. ODUflex used to manage bandwidth between PXM cards. These are static (not OTS controlled) PXM cards support termination of EVPL circuits on vlans Legacy ODU4/2e links to routers via TIM ports 100GbE Trunks follow traffic rather than fibre

Networks ∙ Services ∙ People GÉANT WDM/OTN NREN GÉANT Router 100GbE NREN GÉANT Router 100GbE GÉANT Router GÉANT Router L2 SDN switch N x 10GbE SDN Phase 2 B/W management with transport SDN ODUflex and OTSv used to dynamically insatiate these links. 100GbE N x 10GbE SDN controller OTSc OTSv N x 10GbE WAN-optimised switch operates as full layer 3 device and replaces the GÉANT router Layer 2 service supported on the same box WAN-optimised switches connected to the WDM/OTN cloud via a 100GE port on a PXM card Both IP trunks and p2p L2 circuits all are provisioned in the same way – ODU 4/2e or ODUflex circuits terminating on PXM or TIM All circuits may be provisioned statically or dynamically via the OTSv Trunks now controlled by OTS

Networks ∙ Services ∙ People summary 21 Summary

Networks ∙ Services ∙ People Thank you and any questions Networks ∙ Services ∙ People © GEANT Limited on behalf of the GN4 Phase 1 project. The research leading to these results has received funding from the European Union’s Horizon 2020 research and innovation programme under Grant Agreement No (GN4-1). 22