Internet2 Network of the Future Steve Corbató Director, Backbone Network Infrastructure CANARIE Advanced Networking Workshop Toronto 27 November 2001 This is a general overview presentation about Internet2. Internet2 is a consortium, led by US universities, which is recreating the partnership among academia, industry and government that fostered today’s Internet in its infancy.

Internet2 Network of the Future Current state of Abilene Evolution of optical networking Next phase of Abilene 5/13/2019

Abilene background & milestones Abilene is a UCAID project in partnership with Qwest Communications (SONET service) Nortel Networks (SONET kit) Cisco Systems (routers) Indiana University (network operations) ITECs in North Carolina and Ohio (test and evaluation) Timeline Apr 1998: Project announced at White House Jan 1999: Production status for network Oct 1999: IP version of HDTV (215 Mbps) over Abilene Apr 2001: First state education network added Jun 2001: Participation reaches all 50 states & D.C. Nov 2001: Raw HDTV/IP (1.5 Gbps) over Abilene Partnerships are the foundation of how the Internet developed and they are also a part of the foundation of Internet2. 5/13/2019

Abilene focus Enabling innovative applications and services not possible over the commercial Internet Advanced service efforts Multicast IPv6 QoS Measurement Security DDoS detection efforts (Arbor Networks & Asta Networks) 5/13/2019

Abilene status – November, 2001 IP-over-SONET (OC-48c) backbone 54 direct connections 3 OC-48c (2.5 Gbps) connections 22 will connect via at least OC-12c (622 Mbps) by year end 200+ primary participants All 50 states, District of Columbia, & now Puerto Rico 15 regional GigaPoPs support ~70% of participants 37 sponsored participants 15 state education networks (SEGPs) Collaboration of sponsoring member universities and Abilene connectors 5/13/2019

5/13/2019

International connectivity Transoceanic R&E bandwidths growing! Key international exchange points facilitated by Internet2 membership and the U.S. scientific community STARTAP & STAR LIGHT – Chicago Pacific Wave – Seattle Abilene now has GigE connectivity to both SL and P/WAVE AMPATH – Miami New York City – GigE/10GigE EP under development CA*NET3: Seattle, Chicago, and New York CUDI: CENIC and Univ. of Texas at El Paso International transit service Collaboration with CA*NET3 and STARTAP 5/13/2019

Measurement and DDoS Traffic characterization (Ohio ITEC) Network utilization by state education networks and AITN Abilene Scavenger Service policing GigaPoP pair hotspot identification Passive measurement Planned for Indianapolis router backbone links Collaboration with SDSC/MOAT group Distributed Denial of Service detection Strong IU Global NOC interest Asta Networks (UCSD/U of Washington roots) Arbor Networks (U of Michigan/Merit roots) Data privacy and anonymity policy 5/13/2019

Abilene & SCxy Emblematic of ongoing support for HPC Annually escalating bandwidth SC99 Portland: OC-12c SONET (622 Mbps) SC2000 Dallas: OC-48c SONET (2.5 Gbps) SC2001 Denver: 2xOC-48c SONET (5 Gbps) SC2002 Baltimore: 10-Gbps  (planned) SCxy transit connectivity offered to domestic & international R&E nets Backbone MTU raised to 9K bytes Traffic engineering for SC2001 End-to-End Performance: GigaTCP testing 5/13/2019

Abilene traffic engineering – SC2002 5/13/2019

Network of the Future: Context for the next backbone Computational science as an emerging interdisciplinary field Bandwidth and distributed sensing capability as the next critical parameters Complement CPU, memory & storage Increasingly distributed data collection and storage NSF Distributed Terascale Facility solicitation Emergence of optical technologies Dense Wave Division Multiplexing (DWDM) Important distinction: optical transport vs. switching Much new transcontinental conduit and fiber in place; a lot of business plans abandoned… Glut of fiber & conduit – but not bandwidth 5/13/2019

Current state of optical networking Dense Wave Division Multiplexing (DWDM) Current systems can support >160 10-Gbps ’s (1.6 Tbps!) Optical growth can overwhelm Moore’s Law (routers) Costs scale dramatically with distance Three possible scenarios for the future Enhanced IP transport (higher BW and circuit multiplicity) Fine-grained traffic engineering p2p links between campuses, HPC centers, & Gigapops Physical manifestation of switched circuits (a la ATM SVCs) Evolution of optical switching will be critical Leading international efforts in R&E exploration The Netherlands, Canada, STAR LIGHT (Chicago) 5/13/2019

National optical networking options 1 - Incremental wavelengths Provision 10-Gbps ’s from provider(s) in the same way that SONET circuits are done for Abilene now Exploit smaller incremental cost of additional ’s 2 - Dim Fiber Partnership with a facilities-based provider Acquisition of fiber IRU and subsequent O&M agreement for inter-PoP services (amplifiers, regenerators, DWD MUXes?) BSA’s metro condominium fiber concept on the national scale National footprint of 1-2 fiber pairs IRU for the fiber alone could cost $10-20M Most likely awaits the availability of lower-cost optical transmission equipment 5/13/2019

Future of Abilene Original UCAID/Qwest MoU amended on October 1, 2001 Extension of Qwest’s original commitment to Abilene for another 5 years – 10/01/2006 Originally expired March, 2003 Upgrade of Abilene backbone to optical transport capability - ’s x4 increase in the core backbone bandwidth OC-48c SONET (2.5 Gbps) to 10-Gbps DWDM Capability for flexible provisioning of ’s to support future point-to-point experimentation & other projects 5/13/2019

Key aspects of next backbone - I Native IPv6 Motivations Resolving IPv4 address exhaustion issues Preservation of the original End-to-End Architecture model p2p collaboration tools, reverse trend to CO-centrism International collaboration Router and host OS capabilities Run natively - concurrent with IPv4 Replicate multicast deployment strategy Close collaboration with Internet2 IPv6 Working Group on regional and campus v6 rollout Addressing architecture 5/13/2019

Key aspects of the backbone - II Network resiliency Abilene ’s will not be protected a la SONET Increasing use of videoconferencing/VoIP impose tighter restoration requirements (<100 ms) Options: Currently: MPLS/TE fast reroute Would prefer IP-based IGP fast convergence Addition of new measurement capabilities Enhance active probing Latency & jitter, loss, TCP throughput Add passive measurement taps Support of Internet2 End-to-End Performance Initiative Intermediate performance beacons 5/13/2019

Next generation network deployment October, 2001: Detailed technical design starts Spring 2002: PoP upgrades start deployment in three phases April, 2002 – Phase 1 October, 2002 – Phase 2 April, 2003 – Phase 3 October 2003 - Completion of 10-Gbps upgrade 5/13/2019

Network design overview Overall next generation topology is expected to be very similar to current design Previous iterations to router locations Washington DC, Chicago, Sunnyvale, Houston A few differences expected due to Qwest DWDM topology Expect same number of backbone routers (12) 5/13/2019

Regional optical fanout Next generation architecture: Regional & state based optical networking projects are critical three-level hierarchy: Backbone, GigaPoPs/ARNs, campuses Leading examples CENIC ONI (California), I-WIRE (Illinois), SURA Crossroads (Southeastern U.S), Indiana, Ohio Collaboration with the Quilt Regional Optical Networking project U.S. carrier DWDM access is now not nearly as widespread as with SONET circa 1998 5/13/2019

The Quilt A UCAID project support regional advanced networking initiatives 15 charter GigaPoPs (2 more have joined since) Connections typically support 4 or more research universities EDUCAUSE and SURA Quilt GigaPoPs support over 70% of Abilene participants Initial projects Commodity Internet Services Regional Optical Networking (ties into Internet2 NoF) Measurement (ties into Internet2 E2E Performance) Led by Wendy Huntoon (Pittsburgh SC/CMU) 5/13/2019

Conclusions Abilene future UCAID’s partnership with Qwest extended through 2006 Backbone to be upgraded to 10-Gbps in three phases starting spring 2002 Capability for flexible  provisioning in support of future experimentation in optical networking Overall approach to the new technical design and business plan is for an incremental, non-disruptive transition Follow-on network most likely will be developed around a national dark fiber facility and will utilize next generation optical transport technology 5/13/2019

For more information Web: www.internet2.edu/abilene E-mail: abilene@internet2.edu 5/13/2019

www.internet2.edu