Network Update Steve Cotter Director, Network Services Joint Techs, Fermilab July 16, 2007

Slide 2 Internet2 Network Layer 1: Managed wavelengths from Level(3) Communications Level(3) owns and manages Infinera optical gear: responsible for software upgrades, equipment maintenance, remote hands, sparing, NOC services Internet2 NOC has total provisioning control Layer 2: Internet2 owned and managed Ciena CoreDirectors Using DRAGON GMPLS control plane Layer 3: Internet2 owned and managed Juniper T640s Expanded Observatory Platform for layer 1/3 network performance data collection, collocation, experimentation perfSONAR integration for intra- & inter-network performance analysis International connectivity Layer 1 network extended to international exchange points in Seattle, Chicago and New York City Peering points in Seattle, PAIX, Equinix Chicago, others Infrastructure Overview

Regen DTN Core Node Drop DTN Other Level 3 Denver 1850 Pearl Kansas City GPN 1100 Walnut Seattle 1000 Denny Way Level 3 Portland 1335 NW Northrop Level 3 Sunnyvale 1380 Kifer Pacific Northwest GP th Ave Westin Bldg San Diego Portland Oregon GP 707 SW Washington Qwest San Francisco Sacramento Los Angeles CENIC 600 W 7th Equinix Los Angeles 818 W 7 th Level 3 Salt Lake Inter-Mountain GP Front Range GP 572 S DeLong Albuquerque New Mexico GP 104 Gold Ave SE Tulsa 18 W Archer Houston LEARN 1201 N I-45 Dallas San Antonio Baton Rouge LONI 9987 Burbank Chicago 600 W Chicago Detroit Chicago CIC/MREN BOREAS Internet2 710 N Lakeshore Starlight Atlanta 180 Peachtree St NW Birmingham Charlotte Raleigh 5301 Departure Dr Pittsburgh Pittsburgh GP 143 S 25th Washington MAX 1755 Old Meadow Philadelphia MAGPI 401 N Broad New York NYSERNET 32 A of the A New York th Level 3 Cambridge NOX 300 Bent St Jacksonville FAMU&USF 4814 Phillips Hwy Indianapolis IU 1902 S East St Louisville Univ Louisville 848 S 8 th St via 1005 N B St El Paso 501 W Overland Nashville Univ Memphis 2990 Sidco Dr Cleveland TFN/MERIT 4000 Chester Albany 316 N Pearl Level 3 Ogden Rieth Boise Tionesta Rawlins Omaha St. Louis Eureka Reno Valentine Sanderson Raton Phoenix San Luis Obispo Mobile Tallahassee Syracuse Tucson Buffalo Rancho De La Fe (tentative) Edison Atlanta SLR 345 Courtland

Denver Front Range GP 1850 Pearl Level 3 Kansas City GPN 1100 Walnut Level 3 Seattle 1000 Denny Way Level 3 Portland 1335 NW Northrop Level 3 Sunnyvale CENIC 1380 Kifer Level 3 Pacific Northwest GP th Ave Westin Bldg San Diego Portland Oregon GP 707 SW Washington Qwest San Francisco Sacramento Los Angeles CENIC 600 W 7th Equinix Los Angeles 818 W 7 th Level 3 Salt Lake Inter-Mountain GP 572 S DeLong Level 3 Albuquerque New Mexico GP 104 Gold Ave SE Level 3 Tulsa OneNet 18 W Archer Level 3 Houston LEARN 1201 N I-45 Level 3 Dallas San Antonio Baton Rouge LONI 9987 Burbank Level 3 Chicago 600 W Chicago Level 3 MC Detroit Chicago CIC/MREN MERIT BOREAS Internet2 710 N Lakeshore Starlight Atlanta 180 Peachtree St NW Level 3 MC Birmingham Charlotte Raleigh NCREN 5301 Departure Dr Level 3 Pittsburgh Pittsburgh GP 143 S 25th Level 3 Washington MAX 1755 Old Meadow Level 3 Philadelphia MAGPI 401 N Broad Level 3 New York NYSERNET 32 Ave of the Americas New York th Level 3 Cambridge NOX 300 Bent St Level 3 Jacksonville FLR 4814 Phillips Hwy Level 3 Indianapolis 1902 S East St Level 3 Louisville 848 S 8 th St Level 3 via 1005 N B St via 1075 Triangle Ct Nashville Tennessee GP 2990 Sidco Dr Level 3 Cleveland TFN 4000 Chester Level 3 Albany 316 N Pearl Level 3 Ogden Rieth Boise Tionesta Rawlins Omaha St. Louis Eureka Reno Valentine Sanderson Raton Phoenix San Luis Obispo Mobile Tallahassee Syracuse Tucson Buffalo Rancho De La Fe (tentative) Edison Atlanta SLR 345 Courtland Denver Front Range GP 1850 Pearl Level 3 Kansas City GPN 1100 Walnut Level 3 Seattle 1000 Denny Way Level 3 Portland 1335 NW Northrop Level 3 Sunnyvale CENIC 1380 Kifer Level San Diego Portland Oregon GP 707 SW Washington Qwest San Francisco Sacramento Los Angeles CENIC 600 W 7th Equinix Los Angeles 818 W 7 th Level 3 Salt Lake Inter-Mountain GP 572 S DeLong Level 3 Albuquerque New Mexico GP 104 Gold Ave SE Level 3 Tulsa OneNet 18 W Archer Level 3 Houston LEARN 1201 N I-45 Level 3 Dallas San Antonio Baton Rouge LONI 9987 Burbank Level 3 Chicago 600 W Chicago Level 3 MC Detroit Chicago CIC/MREN MERIT BOREAS Internet2 710 N Lakeshore Atlanta 180 Peachtree St NW Level 3 MC Birmingham Charlotte Raleigh NCREN 5301 Departure Dr Level 3 Pittsburgh Pittsburgh GP 143 S 25th Level 3 Washington MAX 1755 Old Meadow Level 3 Philadelphia MAGPI 401 N Broad Level 3 New York th Level 3 Cambridge NOX 300 Bent St Level 3 Jacksonville FLR 4814 Phillips Hwy Level 3 Indianapolis 1902 S East St Level 3 Louisville 848 S 8 th St Level 3 via 1005 N B St via 1075 Triangle Ct El Paso 501 W Overland Level 3 Nashville Tennessee GP 2990 Sidco Dr Level 3 Cleveland TFN 4000 Chester Level 3 Albany 316 N Pearl Level 3 Ogden Rieth Boise Tionesta Rawlins Omaha St. Louis Eureka Reno Valentine Sanderson Raton Phoenix San Luis Obispo Mobile Tallahassee Syracuse Tucson Buffalo Rancho De La Fe (tentative) Edison Atlanta SLR 345 Courtland Fully Operational Installed – not fully turned up Not Installed yet Install Cancelled Circuit Services Nodes Availability Status July 15, 2007

Transition Progress PHASE 1: New York, Philadelphia, Washington, Pittsburgh, Cleveland, Chicago, Boston  Internet2 and Connector Site Design Discussions Complete  Colocation Planning, Connector Coordination Complete  Colocation Availability and Equipment Installation Complete  Wave Availability and Backbone Turn Up Complete  Connector Cutover and Abilene Node Turn Down Complete PHASE 2: Raleigh, Atlanta, Nashville, Louisville, Indianapolis  Internet2 and Connector Site Design Discussions Complete  Colocation Planning, Connector Coordination Complete  Colocation Availability and Equipment Installation Complete  Wave Availability and Backbone Turn Up Complete  Connector Cutover and Abilene Node Turn Down Complete except USF PHASE 3: Kansas City, Tulsa, Houston, Baton Rouge, Jacksonville, Albuquerque, Denver  Internet2 and Connector Site Design Discussions Complete  Colocation Planning, Connector Coordination Complete  Colocation Availability and Equipment Installation Complete  Wave Availability and Backbone Turn Up Complete  Connector Cutover and Abilene Node Turn Down Complete except HOU, FAMU, BTR PHASE 4: Salt Lake City, Seattle, Portland, Los Angeles  Internet2 and Connector Site Design Discussions Complete  Colocation Planning, Connector Coordination Complete  Colocation Availability and Equipment Installation Complete  Wave Availability and Backbone Turn Up Complete  Connector Cutover and Abilene Node Turn Down In progress; CENIC, SLC, Portland outstanding

Internet2 Network 13,500 long haul route miles Deployed and configured over 300 Infinera Network Elements Day 1 capacity of 100Gbps Built 27 custom collocation suites representing 3,365 sqft of space including: 91 Racks - Internet2, ESnet, third-parties 60 Individual bulk cables with 48 & 96 fiber count Deployed 64 metro fiber route miles Enabled connectivity from Houston-to-El Paso & El Paso-to-Denver for L(3) to Internet2/GRNOC and ESnet NOC access Product Summary: Wavelengths, HSIP, Private Lines, Metro Fiber, Collocation, Cross-Connects, Power Controllers, and GPS timing Backbone alone was 569 individual tasks completed over 261 days! Developed the Virtual Network Operations Center – Provisioning and Troubleshooting Dashboard Network by the Numbers:

Slide 8

Slide 9 Internet2 Network Best-Effort High-Speed IP Service Enables delivery of advanced content, commodity services, etc. Point-to-Point Wavelength Services Circuit Service for static or on-demand bandwidth Point-to-point Ethernet (VLAN) Framed SONET Circuit Point-to-point SONET Circuit Bandwidth provisioning available in 50 Mbps increments Physical Connection 1 or 10 Gigabit Ethernet OC-192 SONET Flexible Infrastructure Supporting e-Science, Network Research & Education

Slide 10 Internet2 Network The growing list of members taking advantage of this service includes: Great Plains Network (GPN) ● Indiana GigaPoP MERIT through CIC ● Northern Crossroads (NOX) OSCnet ● Oregon GigaPoP University of Iowa through CIC ● University of Louisville University of New Mexico ● University of Texas – Austin Currently, the CPS offers over 60,000 routes through peering partnerships with commercial networks in Chicago, Seattle, and Palo Alto. A connection to NYC is underway, which will further improve and diversify our commercial network peering structure. Additional peering connections are being investigated. Commodity Peering Service

Slide 11 Internet2 Network Connection oriented services provide for: Guaranteed bandwidth and predictable latency (repeatable, dependable performance between collaborating sites) Traffic segregation (support specific policy or traffic engineering requirements) Router bypass: Express links created for high- bandwidth, limited duration long-haul traffic reducing the need for mid-path L3 interfaces Cost efficiency: L3 router blades cost > L2 ports > L1 or L0 interfaces Capability tradeoff but could possibly improve performance Wavelength & Circuit Services

Slide 12 Internet2 Network Automated circuit provisioning enable rapid deployment and efficient utilization of capital investment Establishing end-to-end lightpaths is a non-trivial task: it is resource intensive and error prone Automated reservation, allocation, and provisioning enables co-scheduling of network and non-network resources Wavelength & Circuit Services

Slide 13 Multi-Service/Domain/Layer/Vendor Provisioning Regional Network Internet2 Network ESNet Dynamic Ethernet TDM GEANT IP Network (MPLS, L2VPN) Ethernet Router SONET Switch Ctrl Element Domain Controller LSP Data Plane Control Plane Adjacency Multi-Domain Provisioning Interdomain ENNI (Web Service and OIF/GMPLS) Multi-domain, multi-stage path computation process AAA Scheduling TDM Slide from Tom Lehman, ISI-East GUI XML AST

Slide 14 Internet2 Network Create private networks between key collaborating sites within a science community or on a project-by- project basis. Examples: Radio Astronomy community sharing telescopes and correlation facilities High Energy Physics community distributing large data sets Smaller projects may link data repositories and computational resources among key participants – e.g. grid portals Network engineering and operations community can use these capabilities to support traffic engineering objectives We expect applications will use these circuit services to carry high-bandwidth, limited duration flows and will retain use of IP links for generic Internet access. Circuit Services Applications

Slide 15 Collaborations The DICE group - Dante (GEANT2), Internet2, CANARIE, and ESnet Working closely with ESnet on interfacing OSCARS and HOPI - involves AAA work, using OSCARS interface “Stitching” project to describe data layer interconnections between segments of a P2P path Reporting back progress to the GLIF and other organizations For example, Phosphorus, in coordination with the SURFnet and University of Amsterdam participants Also having discussions with JGN2 Coordinating with OGF on various schema - topology, path computation, signaling Working with the appropriate standards bodies - ITU, IETF, and OIF

Slide 16 For more information: Thank you!

Slide 17

Slide 18