Update on Dark Fiber Techniques in the US Steve Cotter, Internet2 CEF Workshop May 17, 2005

Background Director, Network Services at Internet2 Responsibilities include: Abilene Network Manhattan Landing Exchange Point (MAN LAN) FiberCo Also working on: Next-generation Abilene HOPI Project (Hybrid Optical Packet Infrastructure) Previously had responsibility for managing National LambdaRail’s fiber acquisition and facilities / optronics buildout

Networking in the US The model of a facilities-based infrastructure built with owned assets has gained momentum on both the regional and national scales What’s driving this? Desire to move away from the traditional GigaPoP model of provisioned, high-capacity services A distressed telecom market has created a contrarian opportunity to own what is viewed by many as a strategic asset – dark fiber – but the window is closing! Increasing capabilities / expertise in deploying and managing networks within the community Research needs - Grid computing wants to view the network as a schedulable resource On par with CPU, memory, storage, and visualization capabilities All at a time when carriers have not made major capital expenditures in new optical infrastructure

Current Landscape The result: U.S. research universities have been aggressively acquiring raw assets & ancillary services from wholesale telecom providers National LambdaRail launched to provide a national scale infrastructure for research and experimentation in networking technologies and applications Upwards of 30,000 route-miles of inter-city dark fiber are now held by this community < 40% held by NLR More than 25 RONs have emerged mostly state-based many with strong gubernatorial support often with concurrent economic development objectives Active examination of new network service models is underway Moving beyond the common bearer service view of best-effort IP

State and Regional Optical Networks Alabama* Arizona (CENIC) Arkansas* California (CALREN) Colorado (FRGP/BRAN) Connecticut (Conn. Education Network) Florida (Florida LambdaRail) Georgia (Southern Light Rail) Great Plains Network* Indiana (I-LIGHT) Illinois (I-WIRE) Louisiana* (LONI) Maryland, D.C. & northern Virginia (MAX) Michigan (MiLR) Minnesota New England region (NEREN) New Mexico* (NMSU, UNM) New York (NYSERNet, Cornell) North Carolina (NC LambdaRail) Ohio (Third Frontier Network) Oklahoma (OneNet) Oregon Pacific Northwest (Lariat – NIH BRIN, PNNL) Rhode Island (OSHEAN) SRON* (southeastern U.S.) Tennessee* (OneTN) Texas (LEARN) Virginia (MATP) Wisconsin (WiscNet) Wyoming (*RONs with RFx’s issued or in process of acquiring fiber) (RONs in red have made dark fiber acquisitions through FiberCo)

States with Regional Optical Networks States with a RON

FiberCo Overview Tool designed to support optical initiatives Spun off from NLR governance discussions Internet2 took responsibility for forming the organization Patterned on the success of Quilt commodity Internet project Operates on behalf of U.S. higher education and affiliates – Internet2 membership Not an operating entity Will not light the fiber – only a holding company Functions Market maker Assignment vehicle for both national & regional optical initiatives

FiberCo Overview Fiber partner National-scale IRU and O&M pricing available through March, 2006 on Level(3)’s traditional footprint Exploring compatible agreements with other providers - WilTel, Qwest, etc. Level(3) chosen because of their overall wholesale approach Open fiber interconnection policy Co-location space availability Homogenous fiber type - G.655 (Corning LEAF) No AUP/CoU restrictions Ease of interconnection with NLR 20 yr IRU: $310/km (~245 €/km) O&M: $62 pair/km/yr (~49 €/km/yr)

FiberCo Overview Bifurcated contracts designed to survive a carrier bankruptcy event 20 yr IRU – can be assigned to other non-profit entities O&M Contract – renewable on 5 yr terms FiberCo assigns both the fiber IRU asset and recurring O&M agreements through an assignment agreement 3-way among FiberCo, Level(3) & assignee Transaction fee (1/yr) charged to recover costs Ongoing bilateral relationship directly between Level(3) and assignee Assignees become responsible for any recurring IRU tax liabilities and potential exemptions

Current Product Portfolio Dark Fiber - ~7500 route miles assigned to date 20-year IRU agreements 5-year renewable agreement for fiber O&M and equipment co-location & power Pre-negotiated contracts customized for the R&E community Collaborative effort with Level(3) on interconnects and lateral builds Builds at cost plus 15% Metro Fiber National-scale pricing for metro fiber with Level(3) Volume discounts on metro fiber 50% paid for temporary lit services are credited toward cost of metro IRU Access to the Level(3) footprint in Europe at FiberCo prices

Work in Progress Closer collaboration with NLR and its regionals to help facilitate more deployments Formalized agreements with additional fiber providers so FiberCo can cover more geography, continue beyond March ‘06 Professional services agreement taking advantage of aggregate demand for: Network engineering & design Installation, test & turn up services Shared NOC services Remote hands & eyes

National LambdaRail Fiber Map SEA 8 8 POR 8 8 BOI SYR 8 4 4 4 STAR 4 NYC OGD 4 8 4 PIT 4 8 8 8 DEN KAN 8 CHI CLE CLE 8 SVL 8 4 8 8 8 SLC 8 4 4 8 8 8 4 WDC 4 8 LAX RAT RAL 4 8 LAX 8 4 8 8 PHO 4 ALB TUL 4 4 8 8 8 SAN 4 8 ATL 4 DAL 4 4 ELP 4 4 PEN 4 JAC 8 4 Level3 fiber WilTel fiber Other fiber 4 4 4 4 4 4 4 SAA BAT 4 4 HOU Cisco 15808 terminal Cisco 15808 OADM Cisco 15454 terminal Cisco 15454 OADM 8 4

NLR Layer 2 Network Cisco 6509 switch 10GE wave SEA CHI NYC PIT DEN SVL KAN CLE WDC RAL LAX PHO ALB TUL ATL ELP JAC BAT HOU Cisco 6509 switch 10GE wave

NLR Layer 3 Network Cisco CRS-1 router 10GE wave SEA CHI NYC DEN PIT WDC LAX RAL PHO TUL ALB ATL BAT JAC HOU Cisco CRS-1 router 10GE wave

NLR Equipment Layout 15808 15454 West East 6509 CRS-1 NLR demarc 15500 Metro and Regional Optical Networks 15500 15500 DWDM 1GE 10GE or OC192

Lessons Learned No single carrier has all routes Build ongoing relationships with multiple carriers The last mile (including the campus) is still the most difficult Dark fiber is real estate Transactions can be time consuming (up to 6 months) Definite advantage to pre-negotiated contracts Owning the fiber is only ~10% of the total cost of ownership Fiber plant design (hut spacing, topology), power, optical gear maintenance have great impact on recurring costs Know the capabilities of the fiber you are acquiring Fiber type, characteristics, and method of deployment must allow you to reach your networking goals (i.e. high channel count DWDM, 40G) Because of the uncertainty with funding, regionals in the US need to develop a sustainable business model You are now a service provider Need to remain aware of the capabilities of the equipment and new feature releases in order to deliver advanced services

Lessons Learned Need a mechanism to facilitate research collaboration on a regional basis between universities and the private/public sectors Creative solutions are sometimes best Combine lit services with deploying your own equipment to expand reach and minimize costs (i.e. adding commodity Internet, wave services for backhaul to the deal) Consider owning a pool of cards that can be deployed and redeployed on carrier gear where and when you choose Anchor tenant approach – carrier owns the chassis, you own up to X% channel cards within that chassis Lease fiber with option to buy Rent conduit, pull your own fiber

Observations The window for acquiring fiber at current pricing levels in the US is almost over – the economy is recovering and market prices going up! Potentially disruptive technologies are on the horizon Consolidation of the telecom industry in the US does not bode well for customer-owned networks

For more information Internet2 FiberCo National LambdaRail http://www/internet2.edu FiberCo http://www.fiberco.org National LambdaRail http://www.nlr.net