1. Update on Dark Fiber Techniques in the US
Steve Cotter, Internet2
CEF Workshop
May 17, 2005

2. Background Director, Network Services at Internet2
Responsibilities include:
Abilene Network
Manhattan Landing Exchange Point (MAN LAN)
FiberCo
Also working on:
Next-generation Abilene
HOPI Project (Hybrid Optical Packet Infrastructure)
Previously had responsibility for managing National LambdaRail’s fiber acquisition and facilities / optronics buildout

3. Networking in the US
The model of a facilities-based infrastructure built with owned assets has gained momentum on both the regional and national scales
What’s driving this?
Desire to move away from the traditional GigaPoP model of provisioned, high-capacity services
A distressed telecom market has created a contrarian opportunity to own what is viewed by many as a strategic asset – dark fiber – but the window is closing!
Increasing capabilities / expertise in deploying and managing networks within the community
Research needs - Grid computing wants to view the network as a schedulable resource
On par with CPU, memory, storage, and visualization capabilities
All at a time when carriers have not made major capital expenditures in new optical infrastructure

4. Current Landscape The result:
U.S. research universities have been aggressively acquiring raw assets & ancillary services from wholesale telecom providers
National LambdaRail launched to provide a national scale infrastructure for research and experimentation in networking technologies and applications
Upwards of 30,000 route-miles of inter-city dark fiber are now held by this community
< 40% held by NLR
More than 25 RONs have emerged
mostly state-based
many with strong gubernatorial support
often with concurrent economic development objectives
Active examination of new network service models is underway
Moving beyond the common bearer service view of best-effort IP

5. State and Regional Optical Networks
Alabama*
Arizona (CENIC)
Arkansas*
California (CALREN)
Colorado (FRGP/BRAN)
Connecticut (Conn. Education Network)
Florida (Florida LambdaRail)
Georgia (Southern Light Rail)
Great Plains Network*
Indiana (I-LIGHT)
Illinois (I-WIRE)
Louisiana* (LONI)
Maryland, D.C. & northern Virginia (MAX)
Michigan (MiLR)
Minnesota
New England region (NEREN)
New Mexico* (NMSU, UNM)
New York (NYSERNet, Cornell)
North Carolina (NC LambdaRail)
Ohio (Third Frontier Network)
Oklahoma (OneNet)
Oregon
Pacific Northwest (Lariat – NIH BRIN, PNNL)
Rhode Island (OSHEAN)
SRON* (southeastern U.S.)
Tennessee* (OneTN)
Texas (LEARN)
Virginia (MATP)
Wisconsin (WiscNet)
Wyoming
(*RONs with RFx’s issued or in process of acquiring fiber)
(RONs in red have made dark fiber acquisitions through FiberCo)

6. States with Regional Optical Networks
States with a RON

7. FiberCo Overview Tool designed to support optical initiatives
Spun off from NLR governance discussions
Internet2 took responsibility for forming the organization
Patterned on the success of Quilt commodity Internet project
Operates on behalf of U.S. higher education and affiliates – Internet2 membership
Not an operating entity
Will not light the fiber – only a holding company
Functions
Market maker
Assignment vehicle for both national & regional optical initiatives

8. FiberCo Overview Fiber partner
National-scale IRU and O&M pricing available through March, 2006 on Level(3)’s traditional footprint
Exploring compatible agreements with other providers - WilTel, Qwest, etc.
Level(3) chosen because of their overall wholesale approach
Open fiber interconnection policy
Co-location space availability
Homogenous fiber type - G.655 (Corning LEAF)
No AUP/CoU restrictions
Ease of interconnection with NLR
20 yr IRU: $310/km (~245 €/km)
O&M: $62 pair/km/yr (~49 €/km/yr)

9. FiberCo Overview
Bifurcated contracts designed to survive a carrier bankruptcy event
20 yr IRU – can be assigned to other non-profit entities
O&M Contract – renewable on 5 yr terms
FiberCo assigns both the fiber IRU asset and recurring O&M agreements through an assignment agreement
3-way among FiberCo, Level(3) & assignee
Transaction fee (1/yr) charged to recover costs
Ongoing bilateral relationship directly between Level(3) and assignee
Assignees become responsible for any recurring IRU tax liabilities and potential exemptions

10. Current Product Portfolio
Dark Fiber - ~7500 route miles assigned to date
20-year IRU agreements
5-year renewable agreement for fiber O&M and equipment co-location & power
Pre-negotiated contracts customized for the R&E community
Collaborative effort with Level(3) on interconnects and lateral builds
Builds at cost plus 15%
Metro Fiber
National-scale pricing for metro fiber with Level(3)
Volume discounts on metro fiber
50% paid for temporary lit services are credited toward cost of metro IRU
Access to the Level(3) footprint in Europe at FiberCo prices

11. Work in Progress
Closer collaboration with NLR and its regionals to help facilitate more deployments
Formalized agreements with additional fiber providers so FiberCo can cover more geography, continue beyond March ‘06
Professional services agreement taking advantage of aggregate demand for:
Network engineering & design
Installation, test & turn up services
Shared NOC services
Remote hands & eyes

12. National LambdaRail Fiber Map
SEA 8 8
POR 8 8
BOI
SYR 8 4 4 4
STAR 4
NYC
OGD 4 8 4
PIT 4 8 8 8
DEN
KAN 8
CHI
CLE
CLE 8
SVL 8 4 8 8 8
SLC 8 4 4 8 8 8 4
WDC 4 8
LAX
RAT
RAL 4 8
LAX 8 4 8 8
PHO 4
ALB
TUL 4 4 8 8 8
SAN 4 8
ATL 4
DAL 4 4
ELP 4 4
PEN 4
JAC 8 4
Level3 fiber
WilTel fiber
Other fiber 4 4 4 4 4 4 4
SAA
BAT 4 4
HOU
Cisco terminal
Cisco OADM
Cisco terminal
Cisco OADM 8 4

13. NLR Layer 2 Network Cisco 6509 switch 10GE wave SEA CHI NYC PIT DEN
SVL
KAN
CLE
WDC
RAL
LAX
PHO
ALB
TUL
ATL
ELP
JAC
BAT
HOU
Cisco 6509 switch
10GE wave

14. NLR Layer 3 Network Cisco CRS-1 router 10GE wave SEA CHI NYC DEN PIT
WDC
LAX
RAL
PHO
TUL
ALB
ATL
BAT
JAC
HOU
Cisco CRS-1 router
10GE wave

15. NLR Equipment Layout 15808 15454 West East 6509 CRS-1 NLR demarc 15500
Metro and Regional Optical Networks
15500
15500
DWDM
1GE
10GE or OC192

16. Lessons Learned No single carrier has all routes
Build ongoing relationships with multiple carriers
The last mile (including the campus) is still the most difficult
Dark fiber is real estate
Transactions can be time consuming (up to 6 months)
Definite advantage to pre-negotiated contracts
Owning the fiber is only ~10% of the total cost of ownership
Fiber plant design (hut spacing, topology), power, optical gear maintenance have great impact on recurring costs
Know the capabilities of the fiber you are acquiring
Fiber type, characteristics, and method of deployment must allow you to reach your networking goals (i.e. high channel count DWDM, 40G)
Because of the uncertainty with funding, regionals in the US need to develop a sustainable business model
You are now a service provider
Need to remain aware of the capabilities of the equipment and new feature releases in order to deliver advanced services

17. Lessons Learned
Need a mechanism to facilitate research collaboration on a regional basis between universities and the private/public sectors
Creative solutions are sometimes best
Combine lit services with deploying your own equipment to expand reach and minimize costs (i.e. adding commodity Internet, wave services for backhaul to the deal)
Consider owning a pool of cards that can be deployed and redeployed on carrier gear where and when you choose
Anchor tenant approach – carrier owns the chassis, you own up to X% channel cards within that chassis
Lease fiber with option to buy
Rent conduit, pull your own fiber

18. Observations
The window for acquiring fiber at current pricing levels in the US is almost over – the economy is recovering and market prices going up!
Potentially disruptive technologies are on the horizon
Consolidation of the telecom industry in the US does not bode well for customer-owned networks

19. For more information Internet2 FiberCo National LambdaRail
FiberCo
National LambdaRail