CAnet 4 Updates Thomas Tam June 11, 2006

Topics >L3 network update >L1 network update >L1 redesign >10GbE LANPHY testing >Connection program (CCP) – Damir

L3 network updates >Current Layer 3 network >Connections to the North >IPv6 multicast deployment >UNB and PEI backup peering >New BGP peers

Current Layer 3 network

Connections to the North – Physical details >Northwestel ATM circuit (over radio) from Whitehorse and Yellowknife to Ft. Nelson, and south to Ft. St. John, then to CAnet Edmonton PoP >Juniper M5 added in Edmonton POP (c4-edm01) – 4 port ATM DS3 Physical Interface card (PIC) – 4 port FastE PIC >c4-edm01 peers with c4-cal01 via STS-1 lightpath >Completion date – Yukon - June, 2005 – NWT - July, 2005

Connections to the North – circuit details

IPv6 Multicast >JUNOS bugs delayed deployment late summer. >Deployed late >Embedded-RP enabled on all routers. >Currently receive IPv6 multicast routes from Abilene, GEANT, AARnet, NYSERnet, BCnet and Fed. GigaPoP. >Not much testing over the last few months, looking for partners to test within Canada.

UNB and PEI backup peering >The idea was discussed for number of times but never found time to do the work. >The idea is that ORANs peer with each other and both announce same set of routes to CAnet 4. >In the normal situation, CAnet 4 routers select the routes from the directly connected ORAN due to the BGP longer AS path decision. >In mid February Blair, Ying and Damir were working together on routing policy to make it happened. >It is the first it’s kind on CAnet 4. >Since then multiple maintenance outages and circuit problems, backup peering maintained connectivity to CAnet 4.

New peers >National Lamba Rail (NLR) - AS19401 – A peer at Starlight, IPv4 and IPv6 peering to c4-win01 and c4- tor01 >LosNettos - AS226 – Los Angeles Regional Network, through CENIC infrastructure to Pacific Wave in Seattle. – IPv4 and IPv6 peering to c4-cal01

L1 network updates >HDXc deployment in Chicago and Seattle >4 th and 5 th waves >LP usage report >10G demos and activities

Current CAnet 4 network VanCal EdmSas RegWin Vic Sea TorOttMonHal Fre VanOttMonCal SeaChi KamKel NYC Chi Tor Win Edm CalSea CISCO ONS 1545Nortel OME 6500Nortel HDXc NYC Thu 1 st and 2 nd waves 3 rd wave 4 th wave 5 th wave

HDXc deployment >HDXc deployment in Chicago and Seattle – Both HDXcs were deployed in early Sept, – Seattle HDXc 8 x 10G circuits connected, 5 x CAnet 4 10G, KREONet2 10G, 10G to IEEAF(PNWGP) ONS and CENIC 10G 2 x 10G are planned in a month, IEEAF 10G and WIDE (U. of Tokyo) 10G – Chicago HDXc 9 x 10G circuits connected, 4 x CAnet 4 10G, 2 x 10G to Amsterdam, 3 x 10G to Starlight Force10. 2 x 10G are planned to connect OMNInet. >Both systems were used heavily during IGRID05 and SC05, were two of the major hubs in North American to provide connectivity to Asia and Europe.

4 th and 5 th waves >Were delivered in October, >But three breakouts were competed in the later day – Kamloops, BC – Dec. 19 th, 2005 – Kelowna, BC – Nov. 30 th, 2005 – Thunder Bay, ON – May 12 th, 2006 >Both Kamloops and Kelowna breakouts allow BCnet to build their internal infrastructure. >The thunder Bay breakout provides a redundant path for the northern segment of the ORION network

Capacity allocation chart # 1

Capacity allocation chart # 2

Capacity allocation chart # 3

10G demos and activities >McGill and UBC – Ultra Hi-Def Video conferencing project has taken up one of the 10G circuits. During SC05 they were putting 5Gbps of traffic from McGill to Seattle. >TRIUMF now connected at 10GbE, will be doing tests with CERN very soon once 10G circuit Amsterdam to CERN is ready. >All 10G circuits were fully utilized during IGRID05 and SC05. >U of Tokyo has been using one of our 10G circuits to participate in IPv4 and IPv6 Land speed record challenge.

Objectives – L1 redesign >meet CCP requirement >meet CAnet 4 scaling requirement >minimize soon-to-be obsolete ONS expenditures >minimize the overall costs

ONS limitations >Maximum of 4 x OC192 cards per chassis >Maximum of 2 full speed GbE per cards, 8 slots chassis. Total of 16 full speed GbE ports >Few hub sites already or almost ran out of slot >No 10GbE LAN PHY support >Proprietary GbE encapsulation LAN Extension Protocol (LEX) to SONET framing, >GFP standard GbE encapsulation and VCAT are supported only on the new type of GE card, require hardware upgrade – new version of timing control cards (TCC) and GbE cards

Why VCAT? >SONET/SDH transport structures was designed to provide the traditional transport service (i.e. OC-3c/12c/48c/192c and more recently OC-24c) by combining multiple consective SONET channels. >VCAT was originally designed to extend the utility of the SONET/SDH transport layer. >VCAT provides a highly granular, flexible, and efficient method of provisioning traditional SONET/SDH transport bandwidth for packet-based data service transport. >VCAT allows for the grouping of non-consecutive SONET/SDH synchronous payload envelopes (SPEs) to create “virtual” concatenation bandwidth groups. >With VCAT, an arbitrary number (X) of virtual containers (STS-1/3c SPEs) are grouped together with the combined payload (STS-n-Xv) used to match the required bandwidth. >The combined payload isn’t limited going through one SONET path, can be through multiple SONET paths. >Here is some information on VCAT

Redesign criteria >Flexibility in routing LPs through different SONET paths >Allow GbEs to be dropped in either ONS and OME in the hub sites >Availability of slots for more 10G circuits if required. >GbE ports requirement for CCP and LPs >Maintain the continuity of manage infrastructure among different optical platforms >Minimize costs

Idea Scenario VanCal EdmSas Reg Win Vic Sea TorOttHal Fre VanOttMonCal SeaChi KamKel NYC Chi TorWin Mon NYC Edm CalSea CISCO ONS 1545Nortel OME 6500Nortel HDXc NYC Thu 1 st and 2 nd waves 3 rd wave 4 th wave 5 th wave

Idea Scenario >Breakout 4 th wave in Winnipeg – breakout cost >Re-termination of OC-192 circuits on OMEs – Vancouver, Winnipeg, Toronto, Montreal – 8 x OC192 cards >Migrate some of the LPs from ONS to OME – 7 x GbE cards >BUT, 8 ONS GbE cards are required for CCP needed

New topology >No more purchase of ONS cards >No breakout in Winnipeg, save on the breakout cost and cost of a OC192 card >Install a new OME in Victoria, Edmonton, and Saskatoon for CCP/LP needs >Redeploy one of the ONS in RISQ PoP, so LPs can be dropped on either ONS and OME. >Re-termination of OC-192 circuits on OMEs – Vancouver, Winnipeg, Montreal

Network Topology VanCal EdmSas RegWin Sea TorOtt Hal Fre VanOttMonCal Sea Chi KamKel NYCChi TorWin Mon #1 NYC Edm CalSea CISCO ONS 1545Nortel OME 6500Nortel HDXc NYC Vic Mon New OME Thu Mon #2 RISQ 1 st and 2 nd waves 3 rd wave 4 th wave 5 th wave

Migration schedule >Will be completed in the week of June 19 – June 19, 20 - OME install in Victoria – June 21, 22 – OME install in Edmonton, and Sasakatoon – June 22, 23 – ONE install in Montreal installation >At the same time, all CCP related work will be completed also.

Change on the connections to the North

10GE LANPHY testing #1 >10GE LANPHY is a new type of the interface card for Nortel OME platform, the general availability is in the first quarter of The card will provide: – VCAT capacity, can be sub-rated down to STS3c – 2 x 10GbE LANPHY ports >The early deployment card is now available. CANARIE will purchase three cards for testing. Nortel will replace them once the general availability of the card is released. >It will support only – one GE LANPHY port and – sub-rated down to STS3c-32V (5G).

10GbE LANPHY testing #2 >VCAT support give us the ability to utilize the SONET circuit efficiently. >For examples – McGill Ultra video conferencing – require only 5G but take up a whole 10G circuit – TRIUMF would have a 5G LP to CERN, multiple 1G to tier 2 site

10GbE LANPHY testing #3

Connection Program (CCP) Damir Pobric