Networks ∙ Services ∙ People Mian Usman TNC15, Porto GÉANT IP Layer 17 th June 2015 IP Network Architect, GÉANT

Networks ∙ Services ∙ People 2 Contents Service Architecture Challenges SDN Approach White Box Switches Services and IP Trunks over Infinera Packet TIM

Networks ∙ Services ∙ People 3 Service Architecture Dark Fibre Infrastructure IP/MPLS Network Network DWDM Network Services IP, L2VPN, L3VPN, MDVPN, Multicast, etc Operations

Networks ∙ Services ∙ People 4 Challenges Diverse Service Portfolio Support of Multicast and MDVPN type services Drive for early adoption of JUNOS version Demanding Big Science Users High performing and stable yet innovative network Deliver services at competitive cost Committed to long term contracts and relationship with supplier High switching cost (Dark fibre)

Networks ∙ Services ∙ People Synectics model of “Cycling Worlds” 5 Innovation in GÉANT Project Service Activity World Joint Research Activity World

Networks ∙ Services ∙ People SA1 will carry out two studies and deploy a pilot implementation based on the results: Layer 1 infrastructure study Multi-domain NREN Transit network Run technology trials to assess new technologies and new network solutions to enhance the GÉANT network and improve cost efficiency. Create a GÉANT network evolution plan 6 SA1 and JRA2 Work JRA2 will look at how the existing service and platform could be evolved and what new SDN based connectivity service be offered to NREN community: SDN based connectivity services (BoD) POC of SDX and SDN-IP using white label switches SDN at Optical Layer using Infinera Packet TIM INaaS (Infrastructure and Network as a Service) NetIC (Network in Campus)

Networks ∙ Services ∙ People GÉANT Network Topology

Networks ∙ Services ∙ People GÉANT Network Topology GÉANT leveraging on the existing NREN infrastructure to build GÉANT IP network Layer 1 study will look at which fibre routes could be potentially replaced with CBFs or Alien Waves Multi-domain transit network study will look at how NRENs can use the existing NREN and GÉANT optical infrastructure to peer with and offer transit to each other

Networks ∙ Services ∙ People Separate elements of control plane into individual controllers Reduce reliance on the vendor OS Reduce service interruptions due to OS issues Reduce the time it takes to test the OS version prior to upgrades Multicast service managed by a separate controller No Multicast on Juniper Routers 9 SDN Approach

Networks ∙ Services ∙ People GÉANT Network Topology

Networks ∙ Services ∙ People Juniper Support Contract up for renewal in Aug-16 Opportunity to replace Juniper MX routers in smaller PoPs with White Box Switches Reduce the maintenance and support cost Offer additional functionality Innovative solutions in GÉANT production network 11 White Box Switches

Networks ∙ Services ∙ People GÉANT running out of slots on Juniper MX in bigger PoPs e.g. Frankfurt, Amsterdam and Geneva G Aggregation Switches

Networks ∙ Services ∙ People GÉANT Network Topology

Networks ∙ Services ∙ People PXM GÉANT IP Trunk Over Packet Tim Use Case Current GÉANT Eastern Ring Topology IP Trunks follow the physical fibre path Started the trunk link optimization based on traffic flows Majority of the traffic flows from Western Ring to Eastern Ring Majority of the traffic goes through AT or HU router Majority of the traffic on CZ and SK router is a pass through traffic

Networks ∙ Services ∙ People PXM GÉANT IP Trunk Over Packet Tim Use Case Trunk Link Optimization using pass through links No through traffic via smaller PoPs e.g. GR, SI, SK, HR

Networks ∙ Services ∙ People PXM GÉANT IP Trunks Over Packet Tim

Networks ∙ Services ∙ People PXM GÉANT IP Trunk Over Packet Tim Use Case Reduce the number of 10GE interfaces Reduce the flow limitations based on 10GE interfaces Makes capacity planning easier Reduce the number of slots needed on the Juniper MX router Packet TIM can be used to upgrade Western Ring hence reducing the number of 100G required on Juniper

Networks ∙ Services ∙ People GÉANT Network Topology

Networks ∙ Services ∙ People PXM GÉANT Services Over Packet Tim The Packet TIM can also be to used deliver deterministic network services to the NRENs and end users Bandwidth on-demand and GÉANT Plus circuits can also be delivered over optical layer The big science users overlay networks like LHCONE and LHCOPN can also benefit from using packet TIM E-LAN service which connects NRENs at Optical Layer and enable them to peer directly with each other or using a route server

Networks ∙ Services ∙ People PXM OTS enabled Open Transport Switch (OTS) provides an Open Flow interface to the DTN-X OTS adds transport extensions to the Open Flow interface Initially REST based interface First availability expected in Q however, early versions available now for evaluation. OTS would make a useful API to allow big science users to request EVP-LINE and EVP-LAN services via PXM

Networks ∙ Services ∙ People PXM Evaluation and trial Field trial in Evaluation and service development Field trial integration of PXM and OTS. Evaluate OTS and PXM together.

Networks ∙ Services ∙ People Cambridge Lab 3 x Juniper MX x Juniper MX x Juniper M120 2 x PICA8 Switches 2 x Dell Switches (Cumulus Linux) Servers for VMs Working with vendors to test new products (Infinera Packet TIM) 22 Cambridge Lab

Networks ∙ Services ∙ People 23 Summary Optimise the Layer 3 network by using direct pass through trunks and removing routers from some PoPs Move services and traffic down a layer to reduce the cost of the network Make use of SDN and advance path computation to build intelligence in the network

Networks ∙ Services ∙ People Thank you and any questions Networks ∙ Services ∙ People 24

Networks ∙ Services ∙ People PXM Use case – LHCOPN/LHCONE LHCOPN: High capacity TDM, but not flexible LHCONE: flexibility of L3 VPN, but no reserved bandwidth Can we give users the best of both solutions?

Networks ∙ Services ∙ People NREN GÉANT 500G fiber cloud NREN Tier 1 site Wigner Data Center User service requests Tier 2 Campus CERN PXM EPL & EVPL Services over Layer 1 VPN MEF type EVP-LAN or EVP-Line pool of provisionable OTN B/W OTS REST/OF API to allow experimenter’s applications to manage connectivity OF/REST

Networks ∙ Services ∙ People LHCOPN NRENs are providing LHC with point-to-point Layer2 circuits LHC centers built a virtual routed network out of the circuits LHC centers are providing Network Services to each other: CERN is providing un-restricted transit Some centers are providing limited transit Some LHC centers are peering NRENs support individual link operations & management LHC Sites are responsible for network management (layer 3 configurations) including operations, monitoring, troubleshooting, capacity planning, security management, AUP enforcement, etc.

Networks ∙ Services ∙ People LHCONE – as it exists at the moment LHCONE is currently setup as an overlay VRF on existing NREN infrastructure which are interconnected via regional networks and Open Exchanges NRENs provide the network including core links and routers as a virtual overlay on their regular infrastructure NRENs have a peering or transit relationship with each other LHC centers are strictly users of the services Restrictions apply to the advertised IP Space LHCONE infrastructure includes dedicated access, Trans-Atlantic and some backbone links

Networks ∙ Services ∙ People LHC Workshop Recap Rather than maintaining distinct networks, the LHC community should aim to unify its network infrastructure Traffic aggregation on few links Concerns If the T1s and T2s upgrade to 100G, then the global infrastructure needs to follow LHCONE Evolution Currently LHCONE exists side-by-side with general R&E infrastructure Traffic is segregated but what's the real benefit?