1. The newly completed Internet2 network upgrade: capabilities, services and how to connect with them Heather Boyles Director, International Relations, Internet2 heather@internet2.edu APAN Network Engineering Workshop APAN 24 th Meeting, Xi’An China 27-31 August 2007

2. Outline of Talk Brief Recap of Overall Internet2 infrastructure Physical Plant Networks running on the Internet2 infrastructure ESnet Internet2 –Internet2 IP Network –Internet2 Dynamic Circuit Network (DC Network) Other capabilities that can be provisioned over the Internet2 infrastructure Connections between the Internet2 infrastructure and non-US networks Connectivity details specific to APAN and APAN member networks

3. Internet2 Infrastructure Dedicated fiber pair on national footprint of Level3 Communications 13,500 route miles Infinera optical gear Owned by Level3, *controlled* by Internet2 NOC Have lit 10 waves Additional waves can be lit on same equipment

4. 4 Internet2 Infrastructure Internet2 Network Optical Switching Node Level3 Regen Site Internet2 Redundant Drop/Add Site ESnet Drop/Add Site

5. Networks running on the Internet2 infrastructure ESnet ESnet4 – US Dept. of Energy 2 of the (10)10Gbps waves lit on Internet2 infrastructure ESnet IP Core and ESnet Science Data Network (SDN) Internet2 networks Internet2 IP Network Internet2 Dynamic Circuit Network

6. Internet2 IP Network Replaces the old “Abilene” network Same AS number 11537 Same routing policies for non-US peers Currently uses 1 of the (10)10G wavelengths Highly reliable, high-performance IP network Best-effort packet-based Native IPv4, IPv6 and multicast for both

8. *Commercial Peering Service New (sort of) to Internet2 IP Network Old: peering with commercial IPv6 and multicast networks (this continues) New: settlement-free peering with commercial IPv4 networks and content providers Optional for Internet2 access networks (domestic connectors) Implemented via MPLS VPN, uses separate VRF No change in BGP peering with International Partners *Available to domestic (US) paying connectors only

9. Dynamic Circuit Network (DC Network) A new Internet2 network Although predecessor is the HOPI testbed “network” because once connected, can set up circuit to any other connector “circuit” because circuit-switched (not packet-switched) “dynamic” because operates under a control plane that supports dynamic setup of circuits

10. DC Network Provisioned on Internet2 infrastructure with Ciena CoreDirectors Allow circuits of bandwidth 50mpbs up to 10Gbps Characteristics: Subject to blocking based on available capacity Short in duration, no longer than 2 weeks Connectors must utilize control plane to set up circuits Built on DRAGON control plane (NSF-funded project at U Maryland and USC-ISI-East)

11. Internet2 Dynamic Circuit Network

12. Recap: Internet2 networks 2 networks with which a non-US network can connect Internet2 IP Network Dynamic Circuit Network Putting the 2 different networks back together: the hybrid model Ultimately, transparent to user – application will decide on packet or circuit-switched Much investigation yet to do over next several years Continue to use HOPI testbed (now provisioned over Internet2 infrastructure) to try out these ideas

13. Other capabilities of the Internet2 infrastructure Transport Services (Static Circuit Services) 2 fixed endpoints across the Internet2 infrastructure 50Mbps to full 10Gbps or multiple 10Gbps waves Provisioned over Ciena Core Directors or directly over Infinera Set up manually by Internet2 NOC Can be reserved up to a year in advance Can be ordered as protected Duration is weeks to years Fee is based on distance, bandwidth, duration and protection scheme

14. International connections with the Internet2 infrastructure and its networks Internet2 IP Network Internet2 Dynamic Circuit Network Ordering and utilizing Transport Services over the Internet2 infrastructure

15. Internet2 IP Network Very little change from “Abilene” network Most existing connections and peerings have transitioned seamlessly to new Internet2 IP Network 25 direct* non-US peers *including via multi-hop BGP peering

16. How to interconnect and peer with Internet2 IP Network In US, prefer to interconnect via international exchange points PacificWave (north and south) StarLight NGIX-East MAN LAN Atlantic Wave (via NGIX-East and MAN LAN) Outside US, Internet2 has no IP network infrastructure Transit via established peers E.g. TEIN2 transits to/from Internet2 via TransPAC2 E.g. India’s ERNET transits to/from Internet2 via GEANT2 For direct peering, contact heather@internet2.edu

17. Internet2 IP Network International Peering MAN LAN 32 Ave. of the Americas New York, NY 10GE NGIX-East College Park, MD 10GE StarLight 710 North LakeShore Drive Chicago, IL 10GE PacificWave-N 2001 Sixth Ave Seattle, WA 10GE PacificWave-S 624 S. Grand, 600 W. 7 th or 818 W. 7 th 10GE

18. Internet2 IP Network International Peering MAN LAN 32 Ave. of the Americas New York, NY 10GE NGIX-East College Park, MD 10GE StarLight 710 North LakeShore Drive Chicago, IL 10GE PacificWave-N 2001 Sixth Ave Seattle, WA 10GE PacificWave-S 624 S. Grand, 600 W. 7 th or 818 W. 7 th 10GE

19. Internet2 IP Network International Peering MAN LAN 32 Ave. of the Americas New York, NY 10GE NGIX-East College Park, MD 10GE StarLight 710 North LakeShore Drive Chicago, IL 10GE PacificWave-N 2001 Sixth Ave Seattle, WA 10GE PacificWave-S 624 S. Grand, 600 W. 7 th or 818 W. 7 th 10GE AtlanticWave MANLAN NGIX-EAST Southern Crossroads 56 Marietta St Atlanta, GA AMPATH 50 NE 9th St Miami, FL 10GE via MAN LAN 10GE via NGIX-East

20. Internet2 Dynamic Circuit Network New! We’re all in this together figuring out what this means Physical connection Control plane ‘peering’ – inter-domain control of circuits Internet2-GEANT2 service trials underway to learn more about how to do this Interested in other service trials: JGN2, AARNET

21. Physical connection to the Internet2 DC Network In US, again prefer via exchange points where available: MAN LAN: DC Network is connected to Nortel HDXc and OME6500 StarLight: DC Network is connected to Force 10 Ethernet switch PacificWave: DC Network not connected yet in Seattle or Los Angeles Talking with John Silvester about connection to TransLight/PacificWave ONS15454 that serves AARNET southern link When PacificWave decides SONET/TDM/Layer 1 equipment, will connect NGIX-East: DC Network not connected yet: will be to NGIX- East Ciena Core Director

22. MAN LAN 32 Ave. of the Americas New York, NY OC192c to MAN LAN Nortel HDXc NGIX-East College Park, MD None yet StarLight 710 North LakeShore Drive Chicago, IL 10GE to Force10 Ethernet Switch PacificWave-N 2001 Sixth Ave Seattle, WA None Yet PacificWave-S 624 S. Grand, 600 W. 7 th or 818 W. 7 th None yet Internet2 DC Network International Connectivity

23. The Internet2 DC Network control plane Internet2 DC Network adopting control plane architecture based on NSF-funded DRAGON project Jerry Sobieski, MAX and Tom Lehman, USC-ISI DRAGON using GMPLS to build layer 2 circuits More information on DRAGON/Internet2 DC Network control plane at: http://events.internet2.edu/2007/spring- mm/sessionDetails.cfm?session=3165&event= 267 http://events.internet2.edu/2007/spring- mm/sessionDetails.cfm?session=3165&event= 267

24. “Peering” control planes Working jointly with ESnet and DANTE (GEANT2, Autobahn) on inter-domain control plane development Also University of Amsterdam Pursuing jointly developed web services-based solution to tie together heterogeneous control planes Initial release real soon now; official release Fall 2007 For good overview, see presentation from Joint Techs at: http://www.internet2.edu/presentations/jt2007jul/2007 0717-lehman-et-all.ppt http://www.internet2.edu/presentations/jt2007jul/2007 0717-lehman-et-all.ppt

25. Internet2 DC Network summary Stay tuned! Let us know if you have a similar network to our Dynamic Circuit Network What control plane are you using for dynamic circuit setup across your network? Contact heather@internet2.edu to discuss moreheather@internet2.edu

26. Ordering and Utilizing Internet2 Transport Services (Static Circuit Services) Think of this as if you were going to buy a circuit from a vendor on that vendor’s infrastructure In this case, that vendor is Internet2 The circuits available are on the fiber footprint we have available to us Endpoints at any of the yellow dots in the following map

27. Regen DTN Core Node Drop DTN Other Level 3 Denver 1850 Pearl Kansas City GPN 1100 Walnut Seattle 1000 Denny Way Level 3 Portland 1335 NW Northrop Level 3 Sunnyvale 1380 Kifer Pacific Northwest GP 2001 6 th Ave Westin Bldg San Diego Portland Oregon GP 707 SW Washington Qwest San Francisco Sacramento Los Angeles CENIC 600 W 7th Equinix Los Angeles 818 W 7 th Level 3 Salt Lake Inter-Mountain GP Front Range GP 572 S DeLong Albuquerque New Mexico GP 104 Gold Ave SE Tulsa 18 W Archer Houston LEARN 1201 N I-45 Dallas San Antonio Baton Rouge LONI 9987 Burbank Chicago 600 W Chicago Detroit Chicago CIC/MREN BOREAS Internet2 710 N Lakeshore Starlight Atlanta 180 Peachtree St NW Birmingham Charlotte Raleigh 5301 Departure Dr Pittsburgh Pittsburgh GP 143 S 25th Washington MAX 1755 Old Meadow Philadelphia MAGPI 401 N Broad New York NYSERNET 32 A of the A New York 111 8 th Level 3 Cambridge NOX 300 Bent St Jacksonville FAMU&USF 4814 Phillips Hwy Indianapolis IU 1902 S East St Louisville Univ Louisville 848 S 8 th St via 1005 N B St El Paso 501 W Overland Nashville Univ Memphis 2990 Sidco Dr Cleveland TFN/MERIT 4000 Chester Albany 316 N Pearl Level 3 Ogden Rieth Boise Tionesta Rawlins Omaha St. Louis Eureka Reno Valentine Sanderson Raton Phoenix San Luis Obispo Mobile Tallahassee Syracuse Tucson Buffalo Rancho De La Fe (tentative) Edison Atlanta SLR 345 Courtland

28. Using Internet2 Transport Services Example: Los Angeles to Chicago circuit 50Mbps to 10G full wave You arrange for connectivity to both ends We can help arrange ‘tail circuits’ in some cases, depending on destination You pay according to distance, bandwidth, duration and protection scheme Contract with Internet2 will be required Contact heather@internet2.edu if you are interestedheather@internet2.edu

29. Information specifically for APAN Network and APAN Member Networks In Asia: TransPAC2 is the US NSF-funded project linking US R&E networks to APAN network and others in Asia Internet2 is a partner in TransPAC2 Internet2 IP Network: peer with TransPAC2 in Tokyo at the APAN XP 10Gbps capacity across Pacific and with Internet2 IP Network Internet2 DC Network: TransPAC2 looking to implement similar capabilities Connecting to Internet2 DC Network in Los Angeles (hopefully ultimately via PacificWave)

30. Info for APAN Networks, cont’d In the US: PacificWave is the US-based international exchange point on the US west coast Interconnect here for peering with the Internet2 IP Network Future: Interconnect here for connectivity with the Internet2 DC Network –Now: may entertain direct connections on LA Core Director

31. We Have Renamed NOC URL: http://noc.net.internet2.edu/ NOC email: noc@net.internet2.edu Internet2 Network URL http://www.internet2.edu/network/ Internet2 Network contact address: network@internet2.edu

32. Questions? Heather Boyles heather@internet2.edu