1 ESnet Planning for the LHC T0-T1 Networking William E. Johnston ESnet Manager and Senior Scientist Lawrence Berkeley National Laboratory

ESnet Science Data Network (SDN) core TWC SNLL YUCCA MT BECHTEL-NV PNNL LIGO INEEL LANL SNLA Allied Signal PANTEX ARM KCP NOAA OSTI ORAU SRS JLAB PPPL INEEL-DC ORAU-DC LLNL/LANL-DC MIT ANL BNL FNAL AMES 4xLAB-DC NREL LLNL GA DOE-ALB GTN&NNSA International (high speed) 10 Gb/s SDN core 10G/s IP core 2.5 Gb/s IP core MAN rings (≥ 10 G/s) OC12 ATM (622 Mb/s) OC12 / GigEthernet OC3 (155 Mb/s) 45 Mb/s and less Office Of Science Sponsored (22) NNSA Sponsored (12) Joint Sponsored (3) Other Sponsored (NSF LIGO, NOAA) Laboratory Sponsored (6) QWEST ATM 42 end user sites ESnet IP core SINet (Japan) Japan – Russia (BINP) CA*net4France GLORIADKreonet2 MRENNetherlands StarTapTANet2 Taiwan (ASCC) Australia CA*net4 Taiwan (TANet2) Singaren ESnet IP core: Packet over SONET Optical Ring and Hubs ELP HUB ATL HUB DC HUB peering points MAE-E PAIX-PA Equinix, etc. PNWGPoP SEA HUB ESnet High-Speed Physical Connectivity to DOE Facilities and Collaborators, Summer 2005 IP core hubs SNV HUB Abilene high-speed peering points Abilene CERN (LHCnet – part DOE funded) GEANT - Germany, France, Italy, UK, etc NYC HUB Starlight Chi NAP CHI-SL HUB SNV HUB Abilene SNV SDN HUB JGI LBNL SLAC NERSC SND core hubs SDSC HUB Equinix MAN LAN Abilene MAXGPoP SoXGPoP SNV SDN HUB ALB HUB ORNL CHI HUB

3 ESnet The IP core is primarily a layer 3 infrastructure o However, supports layer 2 via MPLS o Directly connects sites o Provides global peering for sites The SDN core is primarily a layer 2 infrastructure o Targeted at providing virtual circuit services

GEANT (Europe) Asia- Pacific New York (AOA) Chicago (CHI) Sunnyvale (SNV) Washington, DC (DC) El Paso (ELP) Primary DOE Labs Existing IP core hubs ESnet Target Architecture: IP Core + Science Data Network + MANs Possible new hubs Atlanta (ATL) CERN Core loops Seattle (SEA) Albuquerque (ALB) New hubs SDN hubs Aus. Production IP core Science Data Network core Metropolitan Area Networks Lab supplied International connections ESnet Science Data Network (2nd Core) ESnet IP Core Metropolitan Area Rings

5 Site gateway router site equip. Site gateway router ESnet production IP core ANL FNAL ESnet MAN Architecture (e.g. Chicago) R&E peerings monitor ESnet management and monitoring ESnet managed λ / circuit services tunneled through the IP backbone monitor site equip. ESnet production IP service ESnet managed λ / circuit services T320 International peerings Site LAN ESnet SDN core T x 10 Gbps channels core router switches managing multiple lambdas core router Starlight Qwest

6 The First ESnet MAN: SF Bay Area Chicago El Paso Seattle and Chicago LA and San Diego SF Bay Area λ 4 future λ 3 future λ 2 SDN/circuits λ 1 production IP SLAC Level 3 hub Qwest / ESnet hub SNLL Joint Genome Institute LBNL NERSC LLNL 2 λs (2 X 10 Gb/s channels) in a ring configuration, and delivered as 10 GigEther circuits Dual site connection (independent “east” and “west” connections) to each site Will be used as a 10 Gb/s production IP ring and 2 X 10 Gb/s paths (for circuit services) to each site Qwest contract signed for two lambdas 2/2005 with options on two more Project completion date is 9/2005. Qwest-ESnet national core ring National Lambda Rail circuits ESnet MAN ring

7 FNAL ESnet site gateway router site equip. Qwest hub (NBC Bld.) Starlight ESnet IP core FNAL Shared w/ IWire ESnet ESnet Near-Term Planning for FNAL FNAL CERN ESnet Switch/RTR ESnet/ Qwest ORNL OC192 ESnet fiber T320 NRL, UltraScienceNet, etc. T320 ESnet SDN core All circuits are 10Gb/s Notes Qwest IWire FNAL

8 ESnet Planning for BNL (Long Island MAN Ring) Engineering Study for LI MAN Other connections

Denver Seattle Sunnyvale LA San Diego Chicago Pitts Wash DC Raleigh Jacksonville Atlanta KC Baton Rouge El Paso - Las Cruces Phoenix Pensacola Dallas San Ant. Houston Albuq. Tulsa New York Clev Proposed ESnet Lambda Infrastructure Based on National Lambda Rail – FY08 NLR wavegear sites NLR regeneration / OADM sites Boise

10 DEN ELP ALB ATL Metropolitan Area Rings ESnet Goal – 2007/2008 Aus. CERN Europe SDG AsiaPac SEA Major DOE Office of Science Sites High-speed cross connects with Internet2/Abilene New ESnet hubs ESnet hubs SNV Europe Japan CHI Science Data Network core Lab supplied Major international Production IP ESnet core DC Japan NYC Aus. Metropolitan Area Rings 10 Gbps enterprise IP traffic Gbps circuit based transport ESnet Science Data Network (2nd Core – Gbps, National Lambda Rail) ESnet IP Core (≥10 Gbps) 10Gb/s 30Gb/s 40Gb/s

11 OSCARS: Guaranteed Bandwidth Service A high-priority R&D topic in “DOE Science Networking Challenge: Roadmap to 2008” The MICS funded OSCARS project is implementing dynamically provisioned circuit switching in ESnet o On-demand Secure Circuits and Advance Reservation System (OSCARS) o End-to-end provisioning will be initially provided by a combination of Ethernet switch management of λ paths in the MAN and Ethernet VLANs and/or MPLS paths in the ESnet core o Provisioning will initially be provided by manual circuit configuration, on-demand in the future o A technology collaboration with Internet2/Abilene

12 OSCARS: Guaranteed Bandwidth Service Progress o Testing OSCARS Label Switched Paths (MPLS based virtual circuits) -A static LSP between BNL and FNAL has been configured for the sites to test their infrastructure -Multiple static LSPs have been configured between GA and NERSC to test local and wide-area QoS o Collaboration – code is being jointly developed with Internet2's Bandwidth Reservation for User Work (BRUW) project

13 OSCARS: Guaranteed Bandwidth Service user system2 user system1 site B resource manager resource manager resource manager policer authorization shaper site A allocation manager policer bandwidth broker resource manager