1. CANARIE The Customer Empowered Networking Revolution http://www.canarie.ca http://www.canet3.net Background Papers on Gigabit to The Home and Optical Internet Architecture Design Available Optical Internet News list: Send e-mail to Bill@Canarie.ca Bill.St.Arnaud@canarie.ca

2. Outline The Message CANARIE CA*net 3 CA*net 4 Gigabit Internet to the School and Home

3. The Message In mid 1990s the prevailing wisdom was that commercial sector would drive design of Internet infrastructure R&E networks would focus on applications or specialized services As a result in North America R&E networks were commercialized or discontinued e.g NSFnet & CA*net However new network technologies and most importantly dark fiber is allowing R&E networks to once again redefine telecommunications not only for themselves but also for businesses and most importantly the last mile to the home R&E networks may become the cornerstone of municipal fiber to the home networks Over time the current hierarchical connection oriented telecom environment will look more like the Internet which is made up of autonomous peering networks. These new concepts in customer empowered networking are starting in the same place as the Internet started – the university and research community.

4. Mission: To facilitate the development of Canadas communications infrastructure and stimulate next generation products, applications and services Canadian equivalent to Internet 2 and NGI private-sector led, not-for-profit consortium consortium formed 1993 federal funding of $300m (1993-99) total project costs estimated over $600 M currently over 140 members; 21 Board members CANARIE Inc

5. GigaPOP CA*net 3 National Optical Internet Vancouver Calgary Regina Winnipeg Ottawa Montreal Toronto Halifax St. Johns Fredericton Charlottetown ORAN BCnet Netera SRnet MRnet ONet RISQ ACORN Chicago STAR TAP CA*net 3 Primary Route Seattle New York Los Angeles CA*net 3 Diverse Route Deploying a 4 channel CWDM Gigabit Ethernet network – 400 km Deploying a 4 channel Gigabit Ethernet transparent optical DWDM– 1500 km Multiple Customer Owned Dark Fiber Networks connecting universities and schools 16 channel DWDM -8 wavelengths @OC-192 reserved for CANARIE -8 wavelengths for carrier and other customers Consortium Partners: Bell Nexxia Nortel Cisco JDS Uniphase Newbridge Condo Dark Fiber Networks connecting universities and schools Condo Fiber Network linking all universities and hospital

6. Customer Empowered Networks School boards and municipalities throughout North America are deploying their own dark fiber networks in partnership with next generation carrier Individual institutions – the customers – own and control their own strands of fiber Fiber are configured in point to point private networks; or Connect to local ISP or carrier hotel Low cost LAN architectures and optics are used to light the fiber Control and management of the optics and wavelengths is under the domain of the LAN customer at the edge, as opposed to the traditional carrier in the center These new concepts in customer empowered networking are starting in the same place as the Internet started – the university and research community. Customers will start with dark fiber but will eventually extend further outwards with customer control and ownership of wavelengths Extending the Internet model of autonomous peering networks to the telecom world

7. Examples Customer Empowered Networks Universities in Quebec are building their own 3500km condominium fiber network in partnership with 6 next gen carriers- $US 2million Will deploy and manage their own optics and long haul transmission gear Universities in Alberta are deploying their own 400 km 4xGbe dark fiber network - $US 200K Deploy and manage their own optics and long haul transmission gear City of Montreal is second most fibered city in the world because of municipal owned open access conduit In Ottawa is deploying a 85km- 144 strand condominium network connecting 26 institutions – cost $1m US Peel County – Missassuaga & Brampton has built a 200km public sector fiber network - $US 5m Many other cities including Ashland OR, Halifax, Toronto are looking at similar initiatives

8. Market Drivers First - low cost Up to 1000% reduction over current telecom prices. 6-12 month payback Second - LAN invades the WAN – no complex SONET or ATM required in network Network Restoral & Protection can be done by customer using a variety of techniques such as wireless backup, or relocating servers to a multi-homed site, etc Third - Enables new applications and services not possible with traditional telecom service providers Relocation of servers and extending LAN to central site Out sourcing LAN and web servers to a 3 rd party because no performance impact IP telephony in the wide area (Spokane) HDTV video Fourth – Allows access to new competitive low cost telecom and IT companies at carrier neutral meet me points Much easier to out source servers, e-commerce etc to a 3 rd party at a carrier neutral collocation facility

9. À venir Bande passante louée Projet démarré Construit Observatoire Mont-Mégantic Val dOr/Rouyn MAN de Montréal MAN de Québec MAN de Sherbrooke MAN dOttawa/Hull Quebec University Condo Network

10. St-Laurent/Vanier Lanaudière Maisonneuve Marie-Victorin Champlain Rosemont Sorel-Tracy Montmorency Édouard-Montpetit Vieux-Montréal Bois-de-Boulogne Ahuntsic Lionel-Groulx Vers Québec Gérald-Godin John-Abbott André-Laurendeau Dawson À venir Bande passante louée Projet démarré Construit Montreal Public Sector Condominium Networks

11. Schoolboard Condominium Builds

12. Typical Capital Costs Fixed One Time Capital Costs Include Management, engineering and construction costs Negotiating support structure agreements Fiber optic cables Fusing of fibers OTDR sweeps, Premise termination, etc. Average total cost between $7 and $15 per meter as follows: Engineering and Design: $1 - $3 per meter for engineering, design, supervision, splicing Plus Installation: $7 to $10 per meter for install in existing conduit; or $3 to $6 per meter for install on existing poles Plus Premise termination: Average $5k each Plus cost of fiber: 15¢ per strand per meter for 36 strands or less 12¢ per strand per meter for 96 strands or less 10¢ per strand per meter 192 strands or less 5¢ per strand per meter over 192 strands

13. Examples of Dark Fiber costs University network Urban Fiber Builds Varennes: 50 km - $406K (maintenance $26K/year) Montreal East: 14 km - $120K (maintenance $9K/year) Laval: 33km - $213K (maintenance $15K/year) University network Rural Fiber Builds Sorel: 54km - $266K (maintenance $19K/year) Megantic: 40km -$273K (maintenance $14K/year) Schoolboards Victoriaville school board -Average price for fiber(s) $2 - $7 per meter Spokane School District - $US 800/mo for first 5 years then $US 400/mo Over 50 schools Stockholm - $1200/mo – over 100 schools Las Vegas School district – 240 schools – Telcordia (Bellcore) prime contractor Many, many others in the works Companies like Telcordia (Bellcore), IBM, etc are now leading development of dark fiber networks for schools

14. Condo Fiber Build Examples Des affluents: Total cost $1,500,00 ($750,00 for schools) 70 schools 12 municipal buildings 204 km fiber $1,500,000 total cost average cost per building - $18,000 per building Mille-Isles: Total cost $2,100,000 ($1,500,000 for schools) 80 schools 18 municipal buildings 223km $21,428 per building Laval: Total cost $1,800,000 ($1,000,000 for schools) 111 schools 45 municipal buildings 165 km $11,500 per building Peel county: Total cost $5m – 100 buildings Cost per building $50,000

15. Ottawa Fiber Condominium Consortium consists of 16 members from various sectors including businesses, hospitals, schools, universities, research institutes 26 sites Point-to-point topology 144 fibre pairs Route diversity requirement for one member 85 km run $11k - $50K per site Total project cost $CDN 1.25 million Cost per strand less than $.50 per strand per meter 80% aerial Due to overwhelming response to first build – planning for second build under way

17. Typical Payback for school (Real example – des affluents – north of Montreal) DSL to 100 schools - $400 per month per school Over 3 years total expenditure of $1,440,000 for DSL service Total cost of dark fiber network for 100 schools $1,350,000 Additional condominium participants were brought in to lower cost to school board to $750,000 School board can now centralize routers and network servers at each school Estimated savings in travel and software upgrades $800,000 Payback typically 8 –16 months Independent Study by Group Secor available upon request

18. Before After fiber fiber Antennas780 Novell Servers821 SQL Servers133 Lotus Notes Servers 21 Tape Backup Servers124 Ethernet switches/hubs1098 Routers1083 Cache/proxy (Linux)120 Fire walls (Linux) 11 Reduction in the number of servers

19. CA*net 4 Vancouver Calgary Regina Winnipeg Ottawa Montreal Toronto Halifax St. Johns Fredericton Charlottetown Chicago Seattle New York Europe Dedicated Wavelength or SONET channel OBGP switches Optional Layer 3 aggregation service Large channel WDM system

20. Overall Objective To deploy a novel new optical network that allows GigaPOPs at the edge of the network (and ultimately their participating institutions) to setup and manage their own wavelengths across the network and thus allow direct peering between GigaPOPs on dedicated wavelengths and optical cross connects that they control and manage To allow the establishment of wavelengths by the GigaPOPs and their participating institutions in support of QoS and grid applications To allow connected regional and community networks to setup transit wavelength peering relationships with similar like minded networks to reduce the cost of Internet transit To offer an optional layer 3 aggregation service for those networks that require or want such a facility

21. O-BGP (Optical BGP) Control of optical routing and switches across an optical cloud is by the customer – not the carrier Use BGP peering at network configuration stage for process to establish light path cross connects Customers control of portions of OXC which becomes part of their AS Optical cross connects look like BGP speaking peers All customer requires from carrier is dark fiber, dim wavelengths, dark spaces and dumb switches Building carrier free networks Traditional BGP gives no indication of route congestion or QoS, but with DWDM wave lengths edge router will have a simple QoS path of guaranteed bandwidth May allow smaller ISPs and R&E networks to route around large ISPs that dominate the Internet by massive direct peerings with like minded networks Wavelengths will become new instrument for settlement and exchange eventually leading to futures market in wavelengths

22. The biggest challenge of all… To foster and accelerate broadband Internet to the home

23. The basic assumptions The good, the bad and the ugly.. Monopolies are bad Duopolies are ugly Facilities based competition is good The private sector, in an open competitive market, is far more effective at responding to consumers needs and introducing new services at lower prices than any kind of government regulation But government has a responsibility to foster competition and ensure a level playing field Where a natural monopoly exists government has a responsibility to regulate that monopoly, but only as a last resort First it should make every attempt to develop mechanisms for introducing private sector competition rather than depending on legislative fiat Regulation should be seen as a last resort

24. Facilities based competition in the residential neighborhood? Facilities based competition is alive and well in downtown core The biggest challenge for governments is manage and coordinate the digging up of streets Outside of downtown in big cities Usually only a monopoly telecom provider At best a duopoly How do we introduce facilities based competition into this market (or at least come as close as possible to true facilities based competition)? As well how can we assure scalable high speed Internet services to the home that eventually will support Gigabit speeds or higher?

25. Critical role for governments and universities Municipal dark fiber networks increases facilities based competition, levels the playing field and provides greater choice to the consumer Governments can play a critical role in paying for dark fiber to all public sector buildings Private sector can extend the fiber to businesses and homes ( via wireless, fiber, DSL, etc) Universities can play critical role in organizing municipal condominium fiber builds in their community and serve as the anchor tenant Governments and universities can also encourage building carrier neutral collocation facilities In downtown cores will likely be done by private sector In suburbs will probably have to be public facility like school board office, university, etc

26. Networked Nation CA*net 4 Provincial research and education network Usually one GigaPOP per province Usually one access facility in every major town and city School board office City Hall University School Hospital Library School Colo Option B: Home owners are aggregated at node by service provider of their choice Option A: Home owners and businesses have fused connections all the way to service provider at supernode SuperNodes Nodes Colo Splice Box Homes Splice Box Commercial Internet Commercial Internet

27. Benefits to Industry For cablecos and telcos it help them accelerate the deployment of high speed internet services into the community Currently deployment of DSL and cable modem deployment is hampered by high cost of deploying fiber into the neighbourhoods Cable companies need fiber to every 250 homes for cable modem service, but currently only have fiber on average to every 5000 homes Telephone companies need to get fiber to every 250 homes to support VDSL or FSAN technologies Wireless companies need to get fiber to every 250 homes for new high bandwidth wireless services and mobile Internet It will provide opportunities for small innovative service providers to offer service to public institutions as well as homes For e-commerce and web hosting companies it will generate new business in out sourcing and web hosting For Canadian optical manufacturing companies it will provide new opportunities for sales of optical technology and components

28. CANARIE's 6th Advanced Networks Workshop "The Networked Nation" November 28 and 29, 2000 Palais des Congrès Montreal, Quebec - Canada "The Networked Nation", will focus on application architectures ("grids") made up of customer owned dark fiber and next generation Internet networks like CA*net 3 that will ultimately lead to the development of the networked nation where eventually every school, home and business will have high bandwidth connection to the Internet. Three tracks: Customer owned dark fiber for schools, hospitals, businesses and homes. Next generation optical Internet architectures that will be a natural and seamless extension of the customer owned dark fiber networks being built for schools, homes and businesses. "application grids", which are a seamless integration of dark fiber and optical networks to support specific collaborative research and education applications.