Delivering a production service Alien waves in GÉANT – Delivering a production service Guy Roberts Senior Transport Network Architect, GÉANT TNC 16, Prague May 14 2016

Agenda Motivation Amsterdam-Hamburg performance Operational experience

Motivation 1 Traffic growth LHCONE Monthly average Gbps Year on year growth Q1 2015 -> Q1 2016 Growth ROUTING only Q1 2015 -> Q1 2016 Growth Including Lambdas 47.70% 26.40%

Motivation 2 Maximize utilization The key to reducing the cost of R&E networks is to achieve high levels of infrastructure utilization. A big part of the cost of transmission is fixed costs: fibre & common equipment. Utilization rates are low in some parts of network, in particular fibre through Spain and Croatia/Slovenia. Value-add for GÉANT of the Infinera equipment is: The flexibility of the OTN layer GMPLS control plane Fast turn up on new services Owning the fibre and amplifiers ourselves adds very little value to our service portfolio but makes services uneconomic where utilization is low.

14,000km of lit dark fibre €8M initial investment and then €2.5M per year.

Amsterdam – Hamburg First GÉANT AW implementation: 5 x 100G The R&E community previously had two parallel fibres between Amsterdam and Hamburg. Yellow fibre is SURFnet, Green is GÉANT. Duplication of infrastructure dilutes utilization. Solution: alien waves Fibre now shared between SURFnet, NORDUnet and GÉANT using AWs GÉANT’s fixed fibre costs are reduced by 2/3 compared to GÉANT only fibre. Eliminating redundant fibre infrastructure

System performance Excellent receive margin Receive sensitivity measured on two wavelengths in each direction on Ams-Ham system Better than 10 dB receive margin in both directions – i.e power into Rx can fall by 10dB before errors are seen 10dB 12dB

Robustness testing Tolerant to input power variation Power injection is reduced at two points: Where the AW signal is injected into the fibre Where the AW signal is injected into the Ciena equipment In both cases the reduction in power did not result in a significant change in the pre-FEC BER. Demonstrates that system is stable under power level changes. Technical contributions from Rob Smets from SURFnet and the GÉANT Operations team

Long term Performance Q-factor Worst Q-factor during the day from 15 minute bins Blue dots direction towards Hamburg, orange in direction towards Amsterdam. The red line shows the threshold at which post FEC errors could start to occur. System is running error free post-FEC in both directions since March.

Long term Performance pre-FEC BER Daily maximum pre-FEC bit error rate from 15 minute bins Blue dots direction towards Hamburg, orange in direction towards Amsterdam. System is running error free post-FEC in both directions since March.

Drawing lines of responsibility Demarcation points Infinera Tx Infinera Rx Ciena ROADM SURFnet owns ROADM, fibre and amplifiers GÉANT owns transponders on DTN-X Demarcation is fibre pair carrying 5 x 100G Amplifier Tx Rx Optical fiber Service Demarcation GÉANT transponders Surfnet: fibre, amplifiers and muxing GÉANT transponders

Alarm management Infinera alarms allow debugging of alien wave Alarms and notification aid debugging: four types signal deterioration can be distinguished If there is a loss of fidelity of the signal, this can be detected using pre-FEC threshold crossing notification If the alien wave becomes so degraded that the Q-factor drops below 10, Infinera will raise a pre-FEC Q out of range alarm A cut in the SURFnet fibre results in a Loss of Frame (LOF) alarm, some signal from the optical amplifiers leaks through, so there will not be a Loss of Signal (LOS) alarm. If the local patching between GÉANT and SURFnet sites is broken, then a Loss of Signal (LOS) alarm is raised. Infinera pre-FEC Q out of range alarm Infinera Loss of Frame alarm

History of outages One planned maintenance and two unscheduled outages Three outages since the system was commissioned: 17 Sept 2015: Unscheduled outage because of new fibre being provisioned by SURFnet where engineer disconnected GÉANT fibre. 24 Sept 2015: Planned maintenance announced by SURFnet 08 Oct 2015: Power failure in one of hut site in SURFnet’s network Information comes from out ticket system There have been no further outages since then. Our Operations team is happy that the level of response from SURFnet to fibre outages matches the level we experience from commercial providers.

Uncertainties What we don’t know yet Licencing model for AWs: works for Ams-Ham, but will this model carry over to other vendors? How well will AWs work when fibre is multi-domain (concatenated between NRENs)? How will vendors respond in the case of performance deterioration (risk of finger pointing)? Can we integrate management systems between vendors? What is the standardization future of AWs – do ‘Black Links’ have a future?

Summary Amsterdam-Hamburg alien wave is saving money for the community by increasing utilization on shared fibre. GÉANT has found the system to be robust and easy to manage. We would like to increase AWs in our network in the future. More experience is necessary with AWs on multi-domain fibre.

Any Questions? Guy.Roberts@geant.org

AW brings savings cf. leased waves with only 2 x 100Gbps Economics of alien waves Actual costs for Ams-Ham AW brings savings cf. leased waves with only 2 x 100Gbps Annualized cost over 3 years in Euros

JRA T1 Next targets for AWs GÉANT fibre leases end in 2020/21 – replacement planning starting now. In SGA1, SA1 identified routes that could benefit from infrastructure sharing: Milan – Geneva on GARR fibre Madrid – Lisbon on RENATER RedIRIS fibre Paris – Lisbon on RENATER, RedIRIS, FCCN, comm. fibre Marseille – Milan on RENATER, GARR, commercial fibre Marseille – Madrid on RENATER, RedIRIS commercial fibre Budapest – Zagreb on NIIF, commercial fibre In SGA2, JRA1 will work with local NRENs to design solutions. Commercial options will also be evaluated. Commercial fibre: Between Bilbao & Bordeaux Between Turin & Grenoble Between Barcelona & Montpellier Between Zagreb & Nagykanzsa Slide: Alexander van der Hil