1. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 NetherLight day-to-day experience APAN lambda networking BOF Erik Radius Manager Network Services SURFnet bv

2. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 SURFnetSURFnet Provides the Dutch National Research Network Not for profit company, 50 employees 150 connected organizations, 500,000 users Turnover (2002): 30 million Euro Infrastructure services: –innovation paid for by government –cost effective exploitation for higher education and research

3. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 SURFnet5SURFnet5 Partners BT and Cisco 15 PoPs connected by thirty 10 Gbit/s lambdas Dual stack IPv4 and IPv6 since 2001 More than 100 institutes connected at Gbit/s level 500,000 users

4. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 NetherLightNetherLight Optical Internet exchange point in Amsterdam Built and operated by SURFnet Experiments with light path provisioning in a multi domain environment www.netherlight.net

5. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 NetherLight Network 2001 2.5Gbit/s lambda between StarLight, Chicago, USA and NetherLight, Amsterdam, NL Lambda terminated on Cisco ONS15454 muxes, –WAN side: SONET framed: OC48c –LAN side: GbE interfaces to computer clusters StarLight NetherLight 2.5G lambda GbE

6. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 NetherLight Network at iGrid2002 2.5 Gbit/s SURFnet 2.5 Gbit/s SURFnet 10 Gbit/s Level3 10 Gbit/s IEEAF 2.5 Gbit/s CERN DWDM SURFnet Chicago StarLight Chicago StarLight New York Geneva CERN Geneva CERN Dwingeloo Astron/Jive Dwingeloo Astron/Jive Amsterdam NetherLight Amsterdam NetherLight

7. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 NetherLight equipment in A’dam 10GBase-LR(SM fiber, = 1310nm) 1000BaseT (copper) 1000BaseSX(MM fiber,  = 850nm) 1000BaseLX(SM fiber,  = 1310nm) DWDM line terminal (Cisco ONS15252) Ethernet switch (Cisco 6509) optical cross connect (Calient PXC) L1 (SONET) multiplexer (Cisco ONS15454) Lambdas to Chicago, Geneva, Prague etc DWDM over dark fiber to JIVE Computer clusters UvA/NIKHEF

8. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 International lightpath network 1Q2004 DWDM SURFnet lambda service path IP service path 10 Gbit/s SURFnet 10 Gbit/s SURFnet 10 Gbit/s IEEAF 10 Gbit/s Dwingeloo ASTRON/JIVE Dwingeloo ASTRON/JIVE Prague CzechLight Prague CzechLight 2.5 Gbit/s NSF 10 Gbit/s London UKLight London UKLight Stockholm NorthernLight Stockholm NorthernLight 2.5 Gbit/s New York MANLAN New York MANLAN Tokyo WIDE Tokyo WIDE 10 Gbit/s 10 Gbit/s 10 Gbit/s 2x10 Gbit/s IEEAF 10 Gbit/s 2x10 Gbit/s 10 Gbit/s 2.5 Gbit/s Tokyo APAN Tokyo APAN Geneva CERN Geneva CERN Chicago Amsterdam

9. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 Day-to-day lightpath use Part of lambda capacity statically configured for IP production traffic –Router-to-router SONET circuits between Amsterdam and Chicago for SURFnet and others (e.g. NSF link) –Link & interface state monitored by SURFnet NOC Part of lambda capacity for lightpath research community –Changed on weekly/monthly basis upon request –Has a ‘test’ status; is hardly monitored by NOC

10. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 NSF funded 10G lambda US - EU

11. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 Application for a lightpath Researchers can apply for a lightpath by an e-mail to netmaster-optical@surfnet.nlnetmaster-optical@surfnet.nl Information in the application should include: –Test purpose –End points A and Z [CERN, StarLight, Stockholm] –Interface type [1GbE: SX or LX, 10GbE WAN PHY] –Number of lightpaths needed –Lightpath capacity in multiples of SONET STS-6 (311Mb/s): [STS-6 / STS-12 / STS-24 / STS-48 / STS-192] * Full GbE capacity: STS-24 * –VLAN ID (if applicable) –Start date, end date –Contact details (e-mail, phone) * Bandwidth granularity depends on transport equipment, e.g. Cisco ONS15454

12. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 Day-to-day operations (1) SURFnet NOC manages production network plus NetherLight –Part of NetherLight has production status –NetherLight test circuits get lower priority than SURFnet5 IP network Some learning curve to get NOC engineers familiar with transport (SONET) troubleshooting –ONS15454 network operations: craft terminals only DWDM link to Dwingeloo (ASTRON): very stable, no pro-active management of optical system yet –DWDM (ONS15252) craft terminal is not very user-friendly

13. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 Day-to-day operations (2) International lambda network monitoring –SNMP monitoring by NOC for SONET alarms and port errors –BER level monitoring on ONS15454 SONET ports BER parameter is part of SLA with carriers –GbE ports: errors mostly attributable to LAN troubles –No IP traffic monitoring possible on SONET ports ONS15454 cannot look inside SONET circuits, only provide overall quality information (BER, errored-seconds etc) No trouble with clocking observed yet –SONET network should have one master clock –ONS15454 internal Stratum III clock is sufficient, no external clock needed

14. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 Day-to-day operations (3) Layer0: optical cross connect –Calient Photonic Cross Connect, automated patch panel –Research status only, managed by University of Amsterdam –Topology discovery difficult Layer1: SONET layer –Very flexible GbE path setups on +- biweekly basis –10G end-to-end also possible, for couple of months at a time –Multi-domain setup works very well by building half-circuits –ONS15454 networks from Canet and SURFnet pre-wired at StarLight

15. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 What did we learn / need to work on Point to point lambdas are a powerful service –High-performance research groups are pulling at us after getting a taste of 10GE all-the-way… Straightforward to build a hybrid network architecture on top of optical: –Some lambdas to interconnect your IP prod. routers –Some lambdas for new high speed few-to-few usage Current optical-electrical-optical equipment can allocate sub- lambdas to individual applications –Ethernet-over-SONET based, this will get more advanced soon (we need to learn what LCAS can do here) Management is still cumbersome, learning curve for all of us –How to make this dynamic? Or self-provisionable…?

16. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 Light Path Provisioning in SURFnet6 SURFnet6 Layer 2 / Layer 1 network Light Path Provisioning via Control Plane Layer 3 flow data collection and analysis Operations Management OGSA Applications Web service Broker SURFnet6 Routers Light Path

17. Techs in Paradise 2004, Honolulu / Lambda Networking BOF / Jan 27 Erik.Radius@SURFnet.nl www.netherlight.net www.surfnet.nl