1. Research Department www.ces.net Towards a Nation-wide Fibre Footprint in Research and Education Networking Lada Altmannova Stanislav Sima Rhodes June 9th, 2004

2. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 2 Situation in REN design l Network design is on the top of design levels hierarchy: ICs, cards/units, computers, networks, and is relatively new. WAN/MAN design is still less developed (mainly in consequence of previous long-term monopolies). l Traditional building elements (procured) are data transmission services, routers, terminals, etc. l Traditional network function is to provide best effort IP connectivity l New building elements: dark fibres, optical transmission equipment, switches l New network function: provide E2E circuits

3. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 3 Multi-vendor network design l Multi-vendor networking is necessary for feasibility in multiple domain l Multi-vendor networking is necessary in general case for cost effectiveness also in single domain l Standards, Multi-Source Agreements (MSA), interoperability, etc. gives you independence on one vendor (in design, reparation, upgrade, etc.) and allows you to buy best equipment in each category

4. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 4 Fibre is strategic asset l Fibre infrastructure has been recognised as strategic asset for research and education networking –In some countries, states or regions in Canada, Europe and USA since 2000 –In USA (national scale) since 2003 –in EU (union scale) since 2004 (GN2) l Customer Empowered Fibre (CEF) networks are emerging technology, that gradually changes network topology, architecture and services from bottom of hierarchy to top

5. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 5 Lengths of dark fibres in RENs l More than 20 000 km in Europe (plus some fibres for GN2 and GN2 testbed) l About 33 920 km in USA (including National LambdaRail) l About 6 700 km in Canada l NFF contracted in Australia ….

6. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 6 CEF Networks and industry support l Government support of network research should prepare investment opportunity and should support national/federal/union industry development l Traditionally this means support of carriers and router vendors l For CEF Networks this means rather support of fibre cable vendors, companies laying fibre cables with maintenance, transmission equipment vendors etc. l Optronics vendors now are becoming aware of the non-carrier (enterprise, RENs) market for their gear l = CEF Networks brings new relation between network research and vendors

7. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 7 Economics decides l Dark fibres brings progress to REN design and subsequently to WAN design generally (and than in competition winner takes nearly all) l Strategic importance of getting fibres has been recognised l Comparison of temporary dark fibre and lambdas prices is usable for very partial view only and cannot solve problem in general. Economics decides, but not naïve economics. l Some reasons: uncertain number of lambdas, period of using, future upgrade of the transmission rate, topology changes, uncertain future economical conditions, unexpected new possibilities, … l Actual result: some managers build CEF Networks and others study the problem…

8. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 8 Why dark fibre for RENs ? l Freedom in REN design l Getting fibre means controlling the network l Independence on carriers and on router vendors l Fixed costs of long-term use l Transmission capacity up to Tbps l Cost-effective ways of fibre sharing (WDM, TDM, …) l Expensive dark fibres can be shared by non-profit partners l Multiple wavelengths possible (e.g. 1 – 256) l Multiple fibres in cable, multiple ducts in way… l Equipment selection, moving, design, …..

9. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 9 How to go from telco services to dark fibres l Procure dark fibre independently and before lambdas procurement l Prefer agreements on long term lease with discounts instead of IRU (disadvantages of IRU are prepayment, taxes and risk of bankruptcy) l Do not relay on one provider only, maintain concurrency (for price and for first mile acquiring) l If price is not excellent, use annual contracts and re-procurement l If dark fibre line is not offered but exists, use annual contract for lambda(s) and re-procure dark fibre

10. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 10 Dark fibre deployment in RENs l Increasing and world-wide deployment of dark fibre (acquisition of dark fibres instead of SDH or lambda services) – without moving back l Increasing number of customer premises connected by fibre: to achieve this is more difficult problem than intercity fibre acquisition l Lease of fibres: good prices are 0.4 Euro/m/pair/year for G.652 or 0.5 Euro/m/pair/year for G.655, including maintenance l Building first mile 12 fibre lines: about 30 Euro/m in ground (6 month for delivery), and about 10 Euro/m in air (3 month for delivery)

11. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 11 CEF network architecture will be new l Prefer network architecture with NIL (single span) lines and without overlaying (concurrent) fibres l KIS rule for fibres: Keep It Short, including connections crossing state borders l Realization of lambdas by own equipment and fibres for long distance data transmission l Search actively for new fibre providers (railways, roads and highways, electric power, fuel, gas and heat distribution companies, cable TV companies, etc.) l Search actively for regional authorities and local boards cooperation on laying first mile fibre cables

12. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 12 Photonic Air Links (PAL) l Gigabit transmission rates are possible in free space l Customer Empowered Photonic (CEP) Networks are expected: fibre+PAL l PAL is used usually between buildings (fan-out of photonic transmission to end users in some area) l Transmission rate: from 10 Mbps up to 2.5 Gbps, distance 100 m – 4000 m, costs 1500 – 30 000 Euro l DIY solution of PAL 10 Mbps (for example for students to home – costs 150 Euro) l Microwave backup (probably up to 622 Mbps) of PAL (on L1 level) is possible – e. g. for foggy weather

13. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 13

14. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 14

15. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 15 Dark fibre lighting register: see before new line design

16. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 16 NFF in European NRENs l Switzerland –SWITCH: NFF operational, 1150 km, transmission system with own development (NIL on single fibre) l The Czech Republic –CESNET2: NFF operational, 2800 km, transmission system with own development (long distance NIL) –CzechLight: 445 km DF with experimental traffic l Poland –PIONIER: NFF 2648 km plus 1426 km own and leased fibres operational, plus 2300 km in negotiation –10GE DWDM transmission system operational

17. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 17 NFF in European NRENs II. l Ireland –HEAnet : 1300 km fibres available, operational part, support of optical networking abroad l The Netherlands –SURFnet6: 4000 km fibres available, operational part with DWDM transmission system, new approach on NREN architecture (central routers only) l Slovakia –SANET: NFF operational, 1660 km own and leased fibres, GE transmission system, partially CWDM 4 x GE l Serbia –RCUB: 115 km operational, contracted 1400 km with telco operator (with government support), GE transmission system, CWDM testing

18. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 18 NFF in European NRENs III. l Portugal –FCCN : Tender for 400 km dark fibre l Denmark –UNI-C : 525 km dark fiber for testing, 10 GE l Norway –using or sharing of nation-wide dark fibre available (by agreement with telco operator) l See reference to CEF Networks workshop for more details l Metropolitan dark fibre RENs in Netherlands, Poland, Czech Rep., Greece, Hungary, Ireland, …..

19. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 19

20. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 20

21. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 21 Cheaper International Fibre Footprint (IFF) l KIS: Connect dark fibres used by NREN by a short cross-border fibre lines (called also Near over Border - NoB) l Use WDM equipment on national and cross- border fibres for implementation of international lambdas between end users l This KIS approach avoids some double payments of fibres or lambdas (parallel lines are paid by NREN and DANTE), in the biggest item of budgets l Such change of networking strategy is difficult, needs research and testbeds

22. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 22 KIS for cheaper architecture l Connection of NRENs by short dark fibre (Keep It Short) between NREN PoPs or Users Near over Border (NoB) l NoB dark fibres (lenght, status): –Bratislava (Slovakia)–Vienna (Austria): 104 km, operational –Bratislava (Slovakia)–Brno (Czech Rep.): 182 km, operational –Ostrava (Czech Rep.)-Bialsko-Biela (Poland): 136 km, available –Subotica (Serbia)-Szeged (Hungary): 40 km, available –Kista in Stockholm (Sweden)-Ventspils (Latvia): 405 km, experimental university connection, undersea part l Other lines are prepared

23. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 23 Examples (see GÉANT topology) l Short dark fibres Brno – Bratislava – Vienna and Ostrava –Bialsko-Biala gives possibility to realize lambdas Prague – Bratislava, Prague – Vienna, Poznan – Vienna, Prague – Poznan etc. l Lambda services London – Stockholm and Poznan – Stockholm could be realized by means of connections London – Brussels – Amsterdam – Copenhagen – Stockholm and Poznan - Copenhagen

24. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 24 PC Light l Design Kit for wide area fibre networks, based on racked PCs with build-in optical amplifier modules l Multi-vendor (multi-source) building elements l Preliminary comparison of OA (EDFA) costs: –OA from „ISP vendor“: 40 000 USD –OA from „optical vendor“: 18 000 USD –OA from „module vendor“: 2 500 USD (two OA in module) –OA from Integrated Circuit vendor: ??? (forthcoming) l OA costs are the biggest item in long distance transmission systems costs (e.g. 60%) => PC Light helps to radically change WAN/MAN costs

25. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 25 EDFA 2in1: PC Light equipment example l PC with build-in OA EDFA booster and preamp l Successfully used for NIL GE connection from Plzen to CzechLight in Prague by dark fibre pair (159.4 km, 36.7 dB) since May 2004 l One Side Amplification (OSA NIL approach): this equipment is placed in Prague only. No OA equipment is placed in Plzen, so we expect lower service costs and higher availability (we have 24/7 staff in Prague only) l Advantages: low cost, low size (1U), low power consumption, Linux, SNMP, reach up to 225 km, possibility of development, ….

26. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 26 EDFA 2in1: EDFA Module in Standard 1U PC with Linux Alarm LEDs BO IN IDE +5V DC RS 232 PA OUT BO OUT FE RS232C USB* Network boot is also possible PA IN MicroATX MainBoard with fanless CPU Power Supply Flash disc* Low NF preamplifier Booster

27. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 27 PC Light forthcoming l EDFA 2in1 works on 10 GE (tested in laboratory) l GE interface card on 1550 nm l GE interface card for bidirectional transmission on single fibre l etc.

28. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 28 Lighting of dark fibre: NIL approach l Production lines G.652, 1550 nm, EDFA 189 km Praha – Pardubice GE since May 2002 235 km Brno – Ostrava GE since June 2003 l Experimental lines G.652, 1550 nm, EDFA 235 km Brno – Ostrava OC-48 June 2003 286 km Praha – Brno GE May 2004 (+Raman) l More wavelengths possible

29. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 29 NIL tests in laboratory l G.652 fibre spools, EDFA + Raman: 325 km GE 250 km OC-48 252 km 2x 10 GE 290 km 10 G DWDM l G.655 fibre spools (recently delivered):  testing  lower cost of dispersion compensation on 10 Gbps

30. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 30 Prepared CzechLight PoP in Brno Fibre Praha - Brno 298.3 km, including 257.3 km of G.655 fibre. DWDM 10 Gbps cards for Cisco 15454 l NIL transmission, if possible l June – July 2004

31. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 31 Global Lambda Integrated Facility (GLIF) Environment for co-operation: networking, infrastructure, network engineering, system integration, middleware, applications GLIF was established by invited participants at the 3rd Global Lambda Grid Workshop, held August 27, 2003 in Reykjavik, Island, www.glif.iswww.glif.is Dark fibres are often used for Gigabit or 10 Gigabit access to cities and university premises

32. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 32 CzechLight May 2004

33. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 33 Prepared CzechLight connections

34. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 34 References l Presentations from CEF Networks Workshop, Prague, May 2004 http://www.ces.net/project/optsit/CEFNetworks/ http://www.ces.net/project/optsit/CEFNetworks/ l Dark fibre Sweden-Latvia www.balticopen.netwww.balticopen.net

35. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 35 Example from BalticOpen Ventspils Ventspils Nafta Fårösund Utö Stockholm

36. Research Department TERENA Networking Conference www.ces.net Rhodes June 9th, 2004Towards a Nation-wide Fibre Footprint in RENs 36 Acknowledgement l Jan Gruntorad for support and GLIF entry l Miroslav Karasek and Jan Radil for lighting of CESNET2 and CzechLight fibres l Josef Vojtech for EDFA 2in1 development l Comment: presented ideas and opinions are result of our ongoing R&D activities and are opened to improvement