1. Internet2’s new network Heather Boyles, heather@internet2.eduheather@internet2.edu APAN Singapore Meeting – Network Engineering Session 19 July 2006

2. Heather Boyles2 The New Internet2 Network Announced at the SMM, but details had to be withheld Will be deployed nationally over 13,000 miles of dedicated fiber using Level3 fiber and Infinera optronics across the backbone 100G capacity to start (10x10G waves) Sustainable model with clear path forward to 40G & 100G Network capable of supporting production and experimental services across the same infrastructure Internet2 will have complete control of the optical layer and highly granular lightpath services that can be provisioned dynamically The network will provide short-term and long-term waves, as well as on-demand or advanced reservation “lightpath” scheduling Waves come with Service Level guarantees The IP network, corresponding to the current Abilene footprint, will be built on the optical network Overview

3. Rick Summerhill3 Layer 1 Topology

4. Rick Summerhill4 Community Design Workshop Held 15-16 June in Indianapolis Provided details of the new network and received input from the community Attended by more than 120 Internet2 member technical and executive staff Breakout sessions regional, connector and site specific details were discussed Internet2 received valuable input on architecture, services and transition which is being integrated into the new network design and policies Additional information is available at http://networks.internet2.edu/ New Network Activities

5. Heather Boyles5 Why a new network? New requirements from the community Current contract with Qwest Communications coming to an end October 2007 Could have extended for 1 additional year with decision by 31 March 2006 - declined Background

6. Heather Boyles6 Where did these new requirements come from? Group A report from joint engineering committee from Internet2 and NLR - describing a wide ranging set of network attributes Abilene Technical Advisory Committee (TAC) Report outlining a need to provide for new experimental services along with production services like IP MORPHnet document (1997) - Multi-Modal Organizational Research and Production Heterogeneous Network Requirements

7. Heather Boyles7 Requirements, cont’d New capabilities of regional and state network (facilities-based) – Quilt group input Research examining a wide variety of different types of capabilities DRAGON and CHEETAH - NSF funded projects examining dynamic provisioning Science related projects such as UltraLight and eVLBI OptIPuter and TeraGrid HOPI Testbed Input from the International community GEANT2 providing hybrid types of capabilities GLIF and lightpath capabilities for scientific work Requirements

8. Heather Boyles8 Architecture Design Goals Develop an innovative optical system on a national footprint to serve the broad research and education community The goal is to support all Internet2 members and to enhance the ability to serve a wider community Develop a hybrid network capable of providing point- to-point services together with an IP network Every connector (RON) connects to the backbone across a metro location not requiring extensive backhaul The community retains complete control of the layer 1 optical system including provisioning and switching of wavelengths Architecture

9. Heather Boyles9 Architecture Design Goals, cont’d Internet2 does not have to concentrate on reliability and sparing - Level 3 is responsible for an SLA. The community focus is on networking, not on managing devices like amplifiers The system is capable of supporting network research in wide variety ways Minimal Conditions of Use (CoU), allowing full participation from the entire community in providing new services and capabilities Architecture

10. Heather Boyles10 The System System uses dedicated fiber from the carrier Built on Infinera platform providing innovative optical technology Simple and convenient add/drop technology Simple and convenient wave setup, requiring only additional endpoint interfaces Demonstrated high reliability in initial period of operation on the Level3 network Economics of Infinera system are disruptive in the market place Infinera gear is dedicated to Internet2; controlled by Internet2 Architecture

11. Heather Boyles11 The System, cont’d Architecture has maximum flexibility. Every RON can access every wave on the system if needed System includes grooming capabilities - lightpaths can be built over Ethernet or SONET Can take advantage of advanced SONET capabilities like GFP, VCAT, and LCAS Capable of lightpath provisioning to the campus Lightpath capabilities illustrated in the following diagram Architecture

12. Heather Boyles12 System, cont’d RON connectivity expected to evolve to 2 x 10 Gbps connections 10 Gbps IP connection 10 Gbps point-to-point connection (capable of STS- 1 granularity lightpaths provisioned in seconds), most likely provisioned using Ethernet (GFP based) Hybrid capabilities Expect 20 - 24 connectors Simple and consistent connection scheme Promotes aggregation Architecture

13. Rick Summerhill13Rick Summerhill Optical Nodes

14. Rick Summerhill14 Node Architecture Infinera Optical Gear - Provisioned by NOC management Grooming capabilities in ADM to provide sub channels HOPI types of activities on the grooming boxes Simplified and standardized interface to connectors, exchange points, and other global research and education networks 2 x 10 Gbps interfaces Example displayed is the Atlanta optical node, showing connections required by the collocated IP router node Measurement and control servers will support the node

15. Rick Summerhill15Rick Summerhill IP Network

16. Heather Boyles16 IP Network IP network built on top of optical system High reliability - architecture provides a variety of protection options Commodity service will be available depending on connector requirements Current plan is to continue to use highly reliable Juniper routers, but open to new technologies Will likely use fewer routers, emphasizing point-to- point capabilities and hybrid networking Dynamic inclusion of circuits for capacity or backup Potential near term option of 40 Gbps Services

17. Heather Boyles17 Circuit Services Short Term Dynamically Configured STS-1 Granularity: deterministic circuits using two waves on the full footprint, with framing either SONET or GFP mapped Ethernet. There is no additional cost to use these capabilities. They are included in the base connectivity costs. There may be blocking in the use of these capabilities. Long Term Circuits: Configured as above There is an extra fee for these circuits, proportional to bandwidth, and charged on a day-by-day basis. The shortest term is one day, and there is no call blocking for these capabilities. Long Term Full Waves: configured and provisioned on request based on advance planning Services

18. Heather Boyles18 Key Projects & Capabilities Application Specific Topologies vLBI, TeraGrid, LHC, NEES, NEON, NEPTUNE, national file system Dynamic services Creating lightpaths across multiple administrative domains in seconds TCP data flows File transfers over long distance segments not requiring congestion control

19. Heather Boyles19 Engineering and Operations Evolved from the current Abilene Engineering/NOC at Indiana University and HOPI Testbed Support Center 3 key areas: Control plane development, deployment, and support – MAX through DRAGON project Application support – NCREN (North Carolina) Advanced engineering, monitoring, and management – Indiana University Global NOC Engineering

20. Heather Boyles20 Network Research Flexibility is key feature of the new network Smaller projects at lower bandwidths can be supported on variable footprints Projects can also be supported for variable lengths of time Projects like VINI and GENI can be supported by the new Internet2 network Network Research

21. Heather Boyles21 Network Research, cont’d The Abilene observatory will be expanded to include the new capabilities of the network Data collection at all layers of the network, with datasets made available to network researchers Support for collocation of equipment in optical nodes Internet2 has supported individual network research projects whenever possible. Examples include The router clamp MPLS tunnels for researchers - these can now be supported at layer 1 on the new Internet2 network The 100x100 project Network Research

22. Heather Boyles22 Implications for International Peers Physical Interconnection Some moves, but…. Remain at US “coastal” exchange points: PacificWave (north and south), StarLight, MAN LAN, Atlantic Wave

23. Heather Boyles23 Implications for International Peers, cont’d Policies (aka “AUP” or “COU”) Who can connect What traffic can be carried International Transit Internet2 network participation in International Transit Network (ITN) remains

24. Heather Boyles24 Implications for Int’l Peers, cont’d New Services Beyond just Layer 3 peering….. Provision circuit services as described above, with those with similar services Service Trial underway with GEANT2 Interested in similar trials with others

25. Heather Boyles25 Questions? Comments? heather@internet2.edu http://networks.internet2.edu http://www.internet2.edu/files/Internet2- New-Network-Tech-v0.9.pdf

26. Heather Boyles26