Giving users control Designing the Future 2005 Sydney, 6 April 2005 Tel:

CANARIE Inc.- Overview Federal leadership: Concept born in 1990 out of Industry Canada discussions Founding: Incorporated in 1993 by industry and academia Funding: From Industry Canada: For networks and research applications from Canadian Heritage, HRDC, Health Canada Mission: To facilitate development and use of Canada’s advanced communications infrastructure Primary stakeholders: universities, Government Departments, provincial research networks, broader research community, colleges, carriers, IT sector, SMEs, broader education sector, broader health sector, provinces

Dark fiber: Businesses see the light > ght/ _ html?tag=sas. > According to publisher of USA Today, if you are spending more than $7000/mth on telecom, then dark fiber is for you > Lighting up fiber used to be technically difficult, but CWDM has made it a no brainer > Next generation CWDM will allow up to 80 Gbps for less than $5k with 10Gbps wavelength

Customer owned fiber for businesses > Significant reduction in price for local loop costs > Ability to outsource LAN and web servers to distant location as LAN speeds and performance can be maintained over dark fiber > Access to lower cost competitive service providers at carrier neutral hotels – New entrants cannot afford high cost of building out their own fiber networks > Reduce Internet transit costs via remote peering > Examples: – Colgate-Palmolive build in Cincinnati – Bank of America – Bell Canada subsidiary CGI in Montreal – Lehman Brothers in NY – Ford in Detroit

Condominium Fiber Networks > Several next generation carriers and fiber brokers are now arranging condominium fiber builds – Lexent, Urban Networks, America Fiber, Looking Glass, etc > Organizations such as schools, hospitals, businesses, municipalities and universities become anchor tenants in the fiber build > Each institution gets its own set of fibers on a point to point architecture, at cost, on a 20 year IRU > Fiber is installed and maintained by 3 rd party professional fiber contractors > Institution lights up their own strands with whatever technology they want – Gigabit Ethernet, ATM, PBX, etc > Cost – on average $25K US plus $1500 per year for maintenance

Municipal Condo Architecture School School board or City Hall School Telco Central Office Central Office For Wireless Company VDSL, HFC or FTTH Condominium Fiber with separate strands owned by school and by service providers Carrier Owned Fiber Cable head end Average Fiber Penetration to homes Colo Facility b Business

NYC Condo build with Lexent Inc

Canberra ICON project > ICON provides ‘dark fibres’ to all government departments and agencies in Australian capital Canberra > Several thousand fiber strands between sites > ICON does not mandate any speed nor protocol – Gives agencies whatever they want – Gigabit Ethernet is very common > One time cost of $1000 per strand of fiber from anywhere to anywhere – No charges for bandwidth – Each agency has annual maintenance cost of $15000 per annum regardless of number of fibers

Halifax Condo Fiber Build Private sector fibre optic network km $350,000 build $150,000 engineering Links all major universities, hospitals, research centers and some schools Connects to CA*net4 at Nova Scotia GigaPOP

What’s next? > As more and more institutions and users acquire fiber and wavelengths there will be a need to interconnect these islands of fiber > Cost of national wavelengths less than $US 150K per year – Individual researchers and/or institutions can afford their own local fiber and national wavelengths > Two ways to interconnect these networks – Purchase a managed service from telcos; or – Develop a new peer to peer technology that allows direct interconnection

Customer Controlled Lightpaths > CANARIE’s CA*net 4 is world’s first customer controlled network > Rather than building a traditional IP routed network with central management and control, CA*net 4 is made up of a number of separate customer controlled IP networks – Articulated Private Networks (APNs) using UCLP technology > Examples – Large enterprise wide area network for NRC institutes – Discipline specific IP network for high energy physics facilities – Distributed backplane for computational grids > Provides most of the advantages of dark fiber – Customer can control bandwidth, routing, topology, add/drop, etc – Customer can partition network and offer it to third parties – Customer can do their own inter-domain connectivity > Similar concepts in ITU Y.1312/Y.1313

What is UCLP? > User Controlled LightPaths – a configuration and provisioning tool built around web services > A proxy that sits in front of optical switches and SONET cross connects that allows control of a subset of the cross connects to be delegated to a third party > Third party can concatenate cross connects together from various networks to produce a wide are network that is under their control – Articulated Private Network (APN) > Uses Service Oriented Architecture (SOA) and so network can be integrated with other web service applications > APN can also do routing or switching with logical routers or switches represented as web services

UCLP general operation CA*net 4 UCLP Switch Agents Signal Control Plane Agents Standard CLI or TL1 interface Customer A and sub- partition Customer B Customer C DWDM Eastbound DWDM Westbound X X OSPF GMPLS ISIS Customer A signaling plane Subtended Lightpaths to User Customer B signaling plane Grooming agents Customer C signaling plane X X Customer A UCLP Server MonFox TL1 Proxy OXC X Customer C signaling plane Customer C

User Controlled LightPaths (UCLP): Objectives > Wide Area Network for the enterprise – To integrate wavelengths and fiber from different suppliers within institution’s network management domain – offer VPNs to users > Create discipline specific re-configurable IP networks – Multi-homed network which bypasses firewalls with direct connect to servers and routers – Crosses multiple domain and institutions > User controlled traffic engineering for remote peering – Active replacement for Sockeye and Route Science – Alternative to MPLS

UCLP intended for projects like National LambdaRail CAVEwave acquires a separate wavelength between Seattle and Chicago and wants to manage it as part of its network including add/drop, routing, partition etc NLR Condominium lambda network Original CAVEwave

Today’s hierarchical IP network University Region al National or Pan-Nationl IP Network Other national networks Regional A Regional B Regional C Regional D

Remote peering for ISPs and enterprise World University Region al Server World National DWDM Network Regional A Regional B Regional C Regional D Child Lightpaths Child Lightpaths

Creation of specialty IP networks Commodity Internet Bio-informatics Network University CERN University Automobile Parts Network Business Supply Chain Network Dept Research Network

CANARIE provides APN to TRIUMF Amsterdam New York Toronto Vancouver Victoria Edmonton Ottawa Geneva 10G Lightpath WS 1G Interface WS 5G Interface WS 1.Note: An incoming lightpath (STS) can be assigned to an outgoing STS or a specific interface 2.TIUMF UCLP GUI would only see this APN 3.CANARIE UCLP GUI can this APN or underlying network or other APN 1G Lightpath WS Montreal To Fermi To Brookhaven URI:

TRIUMF GUI harvests other APNs from UoVic, UoT, etc UoToronto Physics Tier 2 UoVictoria Physics Tier 2 TRIUMF Tier 1 CERN Tier 0 Amsterdam New York Chicago Toronto Vancouver Victoria FERMI Tier 1 Brookhaven Tier 1 UBC Physics UA Physics UoT Physics Carleton Physics UdM Physics CA*net 4 Edmonton Ottawa Geneav 10G Lightpath WS TRIUMF APN UoT APN UoV APN 1G Interface WS 5G Interface WS External links or APNs Note: Typical View on TRIUMF UCLP GUI Montreal

1 Gbe 5 Gbe GUI display using a workflow tool 2 Gbe Vic-Van 10 Gbe Van-Edm 10 Gbe Edm-Tor 10 Gbe Tor-NYC 1 Gbe Tor-Ott 10 Gbe NYC-Ams 10 Gbe Ams-Gen 1 Gbe NYC-MTl 1 Gbe 1 Gbe 1 Gbe 1 Gbe 5 Gbe 5 Gbe 5 Gbe 1 Gbe TRIUMF FERMI Brookhaven CERN UoVic Tier 2 UoT Tier 2 Harvested APNs Interface web service Lightpath web service External web service ATLAS server TRIUMF VLAN TRIUMF CWDM Tier 2 Server Note: External APN may be represented as a single web service “drag and drop”

1G HEPnet daisy chain routed TRIUMF partitions APN and establishes cross connects with 3 rd parties APNs UoToronto Physics Tier 2 UoVictoria Physics Tier 2 TRIUMF Tier 1 CERN Tier 0 Amsterdam New York Chicago Toronto Vancouver Victoria FERMI Tier 1 Brookhaven Tier 1 UBC Physics UA Physics UoT Physics Carleton Physics UdM Physics CA*net 4 Edmonton Ottawa To other physics users at smaller universities Geneav CWDM 5G Tier 1 data 2G Tier 2 data Optional interfaces Note: Typical View on TRIUMF UCLP GUI

TRIUMF creates child APN for HEPnet Toronto Vancouver Victoria UBC Physics UA Physics UoT Physics Carleton Physics UdM Physics CA*net 4 Edmonton Ottawa UoV APN 1G Interface WS Montreal Note: TRIUMF has created this child APN from elements from the original CANARIE APN and the APNs provided by UoVictoria, TRIUMF, UoT, etc Note: View seen by HEPnet UCLP GUI CERN HEPnet APNcannot see switches in Amsterdam or NY

Resultant HEPnet routed network UBC Physics UA Physics UoT Physics Carleton Physics UdM Physics CA*net 4 UoV APN 1G Interface WS Montreal CERN To smaller physics depts through university router

CANARIE provides APN to NRC New York Chicago Seattle Victoria Vancouver Edmonton Calgary Regina Saskatoon Winnipeg Toronto Ottawa Montreal Fredericton Halifax CA*net 4 router 2G Lightpath WS GbE interface WS

NRC partitions APN New York Chicago Seattle Victoria Vancouver Edmonton Calgary Regina Saskatoon Winnipeg Toronto Ottawa Montreal Fredericton Halifax

NRC logical view of APN New York Chicago Seattle Victoria Vancouver Edmonton Regina Saskatoon Winnipeg Toronto Ottawa Montreal Fredericton Halifax

Integrating network into SOA > All hardware (sensors -wireless and wired), software processes (Data processing and HPC) and network elements (ORAN, CA*net 4) expressed as WSDL web services – Web services may be instantiations of orchestrations > Hardware, software and network web services linked together by science user with BPEL – WSDL and BPEL provide for generic and open control plane > Elimination of network made up of layers – Every layer a web service that can communicate with other WS > Hence all “science” processes use network data recursive architectures – Re use and replication of same modules for software, hardware and network for each science project

Network Workflow graph

Conclusion - 1 > The concept of customer owned networks started with the same people who first brought you the Internet – our universities and research institutions – Most major universities and research centers have acquired their own metro and regional fiber > Customer owned networks are becoming increasingly affordable with the advent of companies specializing in dark fiber installation and availability of low cost optics such as CWDM > Consolidation of telcos is forcing large enterprises and systems integrators to acquire their own networks > Prices of dark fiber are still high and do not come close to reflecting actual costs – These prices are expected to drop in the metro area as utilities focus on providing dark fiber rather than trying to be telcos

Conclusion -2 > The Canberra ICON project demonstrates how cheap fiber should be > The cost of long distance wavelengths is dropping dramatically and is now affordable for most large enterprises and systems integrators > UCLP and Y.1312/1313 provides customers ability to manage their own wide area optical network integrated with their LAN > Allows the network to be integrated with SOA architectures > Security, remote peering, supply chain management are the big drivers for customer owned networks