CANARIE “Critical Role Universities Will Play in the Future of the Internet” The Customer Empowered Networking Revolution Tel:

Outline  The Message  CANARIE  CA*net 3  Customer Empowered Networking  CA*net 4 and GigaPOP  eScience

The Message  In mid 1990s the prevailing wisdom was that commercial sector would drive design of Internet infrastructure  R&E networks would focus on applications or specialized services  As a result in North America R&E networks were commercialized or discontinued  e.g NSFnet & CA*net  However new network architectures and most importantly dark fiber is allowing R&E networks to once again redefine telecommunications and the future of the Internet  LAN architectures, technologies and most importantly LAN economics are invading the WAN  Carrier neutral IXs (in essence GigaPOPs) are the essence of this network  Control and management of the optics and wavelengths will increasingly be under the domain of the LAN customer at the GigaPOP, as opposed to the traditional carrier in the center  These new concepts in customer empowered networking are starting in the same place as the Internet started – the university and research community.  Internet 2, SURFnet5 and CA*net 3 & 4

 Mission: To facilitate the development of Canada’s communications infrastructure and stimulate next generation products, applications and services  Canadian equivalent to Internet 2 and NGI  private-sector led, not-for-profit consortium  consortium formed 1993  federal funding of $300m ( )  total project costs estimated over $600 M  currently over 140 members; 21 Board members CANARIE Inc

GigaPOP CA*net 3 National Optical Internet Vancouver Calgary Regina Winnipeg Ottawa Montreal Toronto Halifax St. John’s Fredericton Charlottetown ORAN BCnet Netera SRnet MRnet ONet RISQ ACORN Chicago STAR TAP CA*net 3 Primary Route Seattle New York CA*net 3 Diverse Route Deploying a 4 channel CWDM Gigabit Ethernet network – 400 km Deploying a 4 channel Gigabit Ethernet transparent optical DWDM– 1500 km Multiple Customer Owned Dark Fiber Networks connecting universities and schools 16 channel DWDM -8 reserved for CANARIE -8 wavelengths for carrier and other customers Consortium Partners: Bell Nexxia Nortel Cisco JDS Uniphase Newbridge Condo Dark Fiber Networks connecting universities and schools Condo Fiber Network linking all universities and hospital

Customer Empowered Networks  Universities in Quebec are building their own 3500km fiber condo network in partnership with number of carriers  Universities in Alberta are deploying their own 700 km 4xGbe dark fiber network  Soon will be extended to all communities in Alberta  School boards and municipalities throughout North America are deploying their condo open access, dark fiber networks  Illinois Iwire, Indiana Gwire, California DCP, Georgia SURAnet, Stockholm condo fiber builds  Chicago CivicNET, Alberta SuperNET, Canadian NBTF  Carrier are selling “dim wavelengths” managed by customer to interconnect dark fiber networks  Williams, Level 3, Hermes  Typical cost is one time $20K US per school for a 20 year IRU for condominium fiber connection

What is condominium fiber?  A number of organizations such as schools, hospitals, businesses and universities get together to fund and build a fiber network  Carrier partners are also invited to be part of condominium project  Several next generation carriers and fiber brokers are now arranging condominium fiber builds  IMS, QuebecTel, Videotron, Cogeco, Dixon Cable, GT Telecom, etc etc  Fiber is installed, owned and maintained by 3 rd party professional fiber contractors – usually the same contractors used by the carriers for their fiber builds  Each institution gets its own set of fibers, at cost, on a 20 year IRU (Indefeasible Right of Use)  One time up front cost, plus annual maintenance and right of way cost approx 5% of the capital cost  Institution lights up their own strands with whatever technology they want – Gigabit Ethernet, ATM, PBX, etc  New long range laser will reach 120 km  Ideal solution for point to point links for large fixed institutions  Payback is usually less than 18 months

Why Condo Fiber?  First - low cost  Up to 1000% reduction over current telecom prices month payback  Second - LAN invades the WAN – no complex SONET or ATM required in network  Network Restoral & Protection can be done by customer using a variety of techniques such as wireless backup, or relocating servers to a multi-homed site, etc  Third - Enables new applications and services not possible with traditional telecom service providers  Relocation of servers and extending LAN to central site  Out sourcing LAN and web servers to a 3 rd party because no performance impact  IP telephony in the wide area (Spokane)  HDTV video  Fourth – Allows access to new competitive low cost telecom and IT companies at carrier neutral meet me points  Much easier to out source servers, e-commerce etc to a 3 rd party at a carrier neutral collocation facility

À venir Bande passante louée Projet démarré Construit Observatoire Mont-Mégantic Val d’Or/Rouyn MAN de Montréal MAN de Québec MAN de Sherbrooke MAN d’Ottawa/Hull Quebec University Condo Network

St-Laurent/Vanier Lanaudière Maisonneuve Marie-Victorin Champlain Rosemont Sorel-Tracy Montmorency Édouard-Montpetit Vieux-Montréal Bois-de-Boulogne Ahuntsic Lionel-Groulx Vers Québec Gérald-Godin John-Abbott André-Laurendeau Dawson À venir Bande passante louée Projet démarré Construit Montreal Public Sector Condominium Networks

List of Schoolboard Fiber Builds

Province wide network of condominium fiber to 420 communities in Alberta Guaranteed cost of bandwidth to all public sector institutions $500/mo for 10 Mbps, $700/mo for 100 Mbps Network a mix of fibre builds and existing supplier infrastructure (swap/buy/lease) Condominium approach: All suppliers can Buy (or swap) a share of the fibre (during build or after) Lease bandwidth at competitive rates GOA has perpetual right to use (IRU) Ownership will be held at arms length GOA/stakeholder rates are costs to run divided over users Because of fibre capacity, bandwidth can be made available to businesses at urban competitive rate Total cost $193m Bell Intrigna prime contractor Alberta SUPERnet

National Broadband Task Force  Mandate:To map out a strategy and advise the Government on best approaches to make high-speed broadband Internet services available to businesses and residents in all Canadian communities by the year  To ensure Canada’s competitiveness in a global economy  To address the Digital Divide  To create opportunities for all Canadians  35 members including carriers, educators, librarians, communities, equipment manufacturers, etc  Chair – David Johnston

Chicago CivicNet CivicNet - A City-Wide Condominium Fiber Project Connecting up 1600 public sector institutions City will offer existing telecom budget of $25m for 10 years plus access to city ducts, sewers, subway tunnels, etc Private sector in turn must build condominium fiber to all public sector buildings – 1600 Any competitive service provider can purchase fiber and offer service to schools, businesses and homes in vicinity of public sector building Foundation = Existing City Fiber Builds Promises to reduce City Telecom budget 20%

Fredericton Fiber Build  Started as Economic Development tool  MUSH, Govt., Research - ISP, carriers invited to participate  Build partners emerged quickly, $50,000 “donated” by three firms  Contracting now for 8 km phase 1, $110,000, complete Sept 2001  48 fiber min.  Unique experiment to extend off campus Internet access through wireless

Ottawa Fiber Condominium  Consortium consists of 16 members from various sectors including businesses, hospitals, schools, universities, research institutes  26 sites  Point-to-point topology  144 fibre pairs  Route diversity requirement for one member  85 km run  $11k - $50K per site  Total project cost $CDN 1.25 million  Cost per strand less than $.50 per strand per meter  80% aerial  Due to overwhelming response to first build – planning for second build under way

Condo Fiber Costs - Examples  Des affluents: Total cost $1,500,00 ($750,00 for schools)  70 schools  12 municipal buildings  204 km fiber  $1,500,000 total cost  average cost per building - $18,000 per building  Mille-Isles: Total cost $2,100,000 ($1,500,000 for schools)  80 schools  18 municipal buildings  223km  $21,428 per building  Laval: Total cost $1,800,000 ($1,000,000 for schools)  111 schools  45 municipal buildings  165 km  $11,500 per building  Peel county: Total cost $5m – 100 buildings  Cost per building $50,000

Typical Payback for school (Real example – des affluents – north of Montreal)  Over 3 years total expenditure of $1,440,000 for DSL service  Total cost of dark fiber network for 75 schools $1,350,000  Additional condominium participants were brought in to lower cost to school board to $750,000  School board can now centralize routers and network servers at each school  Estimated savings in travel and software upgrades $800,000  Payback typically 8 –16 months  Independent Study by Group Secor available upon request

Before After fiber fiber Antennas780 Novell Servers821 SQL Servers133 Lotus Notes Servers 21 Tape Backup Servers124 Ethernet switches/hubs1098 Routers1083 Cache/proxy (Linux)120 Fire walls (Linux) 11 Reduction in the number of servers

Condo fiber for Business  Significant reduction in price for local loop costs  No increase in local loop costs as bandwidth demands increase  Ability to outsource LAN and web servers to distant location as LAN speeds and performance can be maintained over dark fiber  Access to lower cost competitive service providers at carrier neutral hotels  New entrants cannot afford high cost of building out their own fiber networks  Even small businesses with less than 20 employees can realize significant savings and benefits  Examples:  Colgate-Palmolive build in Cincinnati  Nortel, Cisco, Gov’t depts in Ottawa

Community Fiber Architecture  A community consortia would put together a plan to fiber up all public sector buildings in their community  A community can be a province, a municipality, village, etc  A fiber splice box that terminates the fiber at the street side nearby each public sector building such as school, hospital, library is called a “Node”  Community should must insure that potential facilities exist near the for private sector equipment to connect up future home owners – colo facility  Colo facility allows private sector to extend wireless, VDSL or HFC services to the neighbourhood around the school  Public sector buildings will have dedicated fiber strands that connect to a “Supernode” which is a fiber splice box on the street beside outside of major public sector central facility such as school board office, city hall, university, etc  Community should insure that facilities exist nearby the Supernode for the private sector to install equipment to service home owners and businesses – colo facility  Additional fibers are made available from the Supernode to all Nodes such that competitive service providers can purchase fiber to the node at some future date

Possible architecture for large town School School board office School Telco Central Office Central Office For Wireless Company VDSL, HFC or Fiber Provisioned by service provider Condominium Fiber with separate strands owned by school and by service providers Carrier Owned Fiber Cable head end Average Fiber Penetration to homes Colo Facility b

Benefits to Industry  For cablecos and telcos it help them accelerate the deployment of high speed internet services into the community  Currently deployment of DSL and cable modem deployment is hampered by high cost of deploying fiber into the neighbourhoods  Cable companies need fiber to every 250 homes for cable modem service, but currently only have fiber on average to every 5000 homes  Telephone companies need to get fiber to every 250 homes to support VDSL or FSAN technologies  Wireless companies need to get fiber to every 250 homes for new high bandwidth wireless services and mobile Internet  It will provide opportunities for small innovative service providers to offer service to public institutions as well as homes  For e-commerce and web hosting companies it will generate new business in out sourcing and web hosting  For Canadian optical manufacturing companies it will provide new opportunities for sales of optical technology and components

CA*net 4 Overall Objective  To deploy a novel new optical network that gives GigaPOPs at the edge of the network (and ultimately their participating institutions) to setup and manage their own wavelengths across the network and thus allow direct peering between GigaPOPs on dedicated wavelengths and optical cross connects that they control and manage  To allow the establishment of wavelengths by the GigaPOPs and their participating institutions in support of QoS and eScience applications  To allow connected regional and community networks to setup transit wavelength peering relationships with similar like minded networks to reduce the cost of Internet transit  To offer an “optional” layer 3 aggregation service for those networks that require or want such a facility

CA*net 4 Possible Architecture Vancouver Calgary Regina Winnipeg Ottawa Montreal Toronto Halifax St. John’s Fredericton Charlottetown Chicago Seattle New York Europe Customer controlled optical switches Layer 3 aggregation service Optional Service Available to any GigaPOP Large channel WDM system

eScience  The ultimate goal of e-science is to allow students and eventually members of the general public to be full participants in basic research.  We have seen in other fields like bird census, comet watching, public are interested in participating in basic  Using advanced high speed networks like CA*net 4 and novel new concepts in distributed peer to peer computing, called “Grids” many research experiments that used to require high end super computers can now use the computer capabilities of thousands of PCs located at our schools and in our homes.  High performance computers that are part of C3.ca can be seamlessly integrated with eScience distributed computers using CANARIE Wavelength Disk Drive over CA*net 4  Allows researcher access to the significant computational capabilities of all these distributed computers at our schools and homes  Will also allow students and individuals to be a full participant in the analysis and basic research.  With e-science it might be possible that the next big scientific discovery could be by a student at your local school.

Demonstrated that PC Internet Computing Could Grow to Megacomputers  Running on 500,000 PCs, ~1000 CPU Years per Day  Over Half a Million CPU Years so far!  22 Teraflops sustained 24x7  Sophisticated Data & Signal Processing Analysis  Distributes Datasets from Arecibo Radio Telescope Next Step- Allen Telescope Array Arecibo Radio Telescope

Forest Grid– on CA*Net 4 BC AlbertaSask.Man. Ont. Quebec. Maritimes ORAN Univ. Labs/Ministries. PFC UVic UBC U of A CFS Northern York U. Ottawa Waterloo EDC Chicago STARTAP. CCRS CFS GLFC CFS-HQ CFS Laurentian UNB CFS Atlantic Cornerbrook Min. of For. Quebec. European NSF VBNS UCAID Abilene JPL Seattle DREN ESnet CFS New York NISN NREN CSARSI

Neptune eScience Grid  Joint US-Canadian project to build large undersea dark fiber network off west coast of USA and Canada  Undersea network will connect instrumentation devices, robotic submarines, sensors, under sea cameras, etc  All devices available to students and researchers connected to CA*net 4 and Internet 2 networks  Neptune will be used to gather research data in a variety of fields – seismology, sea vents, fish migrations and population, deep sea aquatic life, etc  Distributed computing and data storage devices on CA*net 4 and Internet 2 will be used to analyze and store data

Neptune – Undersea Grid

Neptune eScience

Wavelength Disk Drives Vancouver Calgary Regina Winnipeg Ottawa Montreal Toronto Halifax St. John’s Fredericton Charlottetown CA*net 3/4 WDD Node

Wavelength Disk Drives  CA*net 3 and CA*net will be “nation wide” virtual disk drive for grid applications  Big challenges with grids or distributed computers is performance of sending data over the Internet  TCP performance problems  Congestion  Rather than networks being used for “communications” they will be a temporary storage device

Conclusion  Many governments have recognized the importance of access to low cost dark fiber as fundamental economic enabler  It will be the 21 st century equivalent to the roads and railways that were built in the 20 th century