Scientific Networking: The Cause of and Solution to All Problems April 14 th Workshop on High Performance Applications of Cloud and Grid Tools Jason Zurawski, Research Liaison

Topics so far on the core design and operation of Grid/Cloud infrastructures – Fertile area for work – Lots of advancement – being driven by scientific needs (e.g. Physics, Biology, Climate, etc.) Achilles Heal of Grid/Cloud computing = infrastructure that links the components – Distributed CPU, Disk, and Users – Earlier efforts to improve the overall performance (e.g. Logistical Networking) Role of Networking – “Under the hood”. Should enable science, but stay out of the way – Lots of advancement, highlight 2 efforts today: DYNES – Dynamic Networking to end sites LHCONE – Dedicated resources for data movement 2 – 10/28/2015, © 2011 Internet2 And Now for Something Completely Different

Data movement to support science: – Increasing in size (100s of TBs in the LHC World) – Becoming more frequent (multiple times per day) – Reaching more consumers (VO size stands to increase) – Time sensitivity (data may grow “stale” if not processed immediately) Traditional networking: – R&E or Commodity “IP” connectivity is subject to other users – Supporting large sporadic flows is challenging for the engineers, and frustrating for the scientists 3 – 10/28/2015, © 2011 Internet2 DYNES

Solution – Dedicated bandwidth (over the entire end to end path) to move scientific data – Invoke this “on demand” instead of relying on permanent capacity (cost, complexity) – Exists in harmony with traditional IP networking – Connect to facilities that scientists need to access – Integration with data movement applications Invoke the connectivity when the need it, based on network conditions Prior Work – “Dynamic Circuit” Networking – creation of Layer 2 point to point VLANs – Transit the Campus, Regional, and Backbone R&E networks – Software to manage the scheduling and negotiation of resources 4 – 10/28/2015, © 2011 Internet2 DYNES

NSF Funded “Cyber-Instrument” – Internet2/Caltech/University of Michigan/Vanderbilt University Provide equipment and software to extend the Internet2 ION service into Campus and Regional networks – Build using the OSCARS IDC software (based on work in OGF NSI Working Group) – perfSONAR Monitoring (based on work in the OGF NM, NMC, and NML Working Groups) – FDT (Fast Data Transfer) data movement 5 – 10/28/2015, © 2011 Internet2 DYNES

Deployment Targets: – 25 End Sites – 8 Regional Networks – Collaboration with like minded efforts (DoE ESCPS) Plans to consider provisional applications (send to if you are Supporting all science - e arly focus on Physics (LHC) sites 6 – 10/28/2015, © 2011 Internet2 DYNES

DYNES Infrastructure Overview 7 – 10/28/2015, © 2010 Internet2

Inter-domain Controller (IDC) Server and Software – IDC creates virtual LANs (VLANs) dynamically between the FDT server, local campus, and wide area network – Dell R410 (1U) Server Fast Data Transfer (FDT) server – Fast Data Transfer (FDT) server connects to the disk array via the SAS controller and runs the FDT software – Dell R510 (2U) Server DYNES Ethernet switch options (emerging): – Dell PC6248 (48 1GE ports, 4 10GE capable ports (SFP+, CX4 or optical) – Dell PC8024F (24 10GE SFP+ ports, 4 “combo” ports supporting CX4 or optical) DYNES Standard Equipment 8 – 10/28/2015, © 2010 Internet2

DYNES Data Flow Overview 9 – 10/28/2015, © 2010 Internet2

4 Project Phases – Phase 1: Planning ( Completed in Feb 2011 ) – Phase 2: Initial Deployment ( Feb 2011 through July 2011 ) – Phase 3: Full Deployment ( July 2011 through Sept 2011 ) – Phase 4: Testing and Evaluation ( Oct 2011 through August 2012 ) A draft DYNES Program Plan document is available with additional details on the project plan and schedule: – Questions can be sent to the mailing list: – DYNES Current Status 10 – 10/28/2015, © 2010 Internet2

Campus connectivity is just one part of a solution – Campus has been the traditional bottleneck – Using a traffic engineering solution like DYNES will connect sites on a national level in a point to point fashion – What about transit to non-DYNES sites? What about other countries? Resources on a national and international level – Investment in networking is still strong – Backbone capacity upgrades coupled with availability of new sites (U.S. UCAN)U.S. UCAN 11 – 10/28/2015, © 2011 Internet2 Inductive Step

Scientific networking needs to be pervasive – Availability where the science is, e.g. “everywhere” – Linking the resources that require this capability Clusters and Supercomputers Data stores Scientific Instruments (Telescopes, Colliders). LHC Community: – Pro-active in terms of network preparedness – Designing next generation connectivity options to meet the needs of the VO as a whole – Sensitive to funding, but always wanting the best for the community to support scientific activity for the next 10+ years 12 – 10/28/2015, © 2011 Internet2 Inductive Step

The goal of LHCONE is to provide a collection of access locations that are effectively entry points into a network that is private to the LHC It is anticipated that LHCONE access locations will be provided in countries / regions in a number and location so as to best address the issue of ease of access – In the US, LHCONE access locations might be co- located with the existing R&E exchange points and/or national backbone nodes – A similar situation exists in Europe and Southeast Asia. 13 – 10/28/2015, © 2011 Internet2 LHC Open Network Environment (LHCONE)

Proposed installation of two nodes to provide immediate service – Chicago – New York Interconnected via Internet2 IP Network – Generally has 9 Gbps of available capacity for initial best-effort traffic use – Potential to provide a dedicated backbone circuit to provide 10G of capacity just for LHCONE (or shared with other scientific VOs) – It is certain that this bandwidth will grow as the Internet2 network upgrades its backbone links to 100 Gbps in – 10/28/2015, © 2011 Internet2 LHCONE – North America

15 – 10/28/2015, © 2011 Internet2 Sample Architecture and Connectivity

Designed to be “come as you are” – Network connectivity is expensive, budgets are tight – Funding opportunities can accommodate increased connectivity in the future – Short term is to offer several methods There will be three primary methods of connection to the LHCONE-NA architecture. – Direct Connection to LHCONE-NA Nodes – Layer2 Connectivity via Internet2 Network (e.g. ION) – Layer3 Connectivity via Internet2 Network 16 – 10/28/2015, © 2011 Internet2 LHCONE Access Methodology

Normally an expensive option, but one that provides the greatest access Physical connection from end site to connection point – Initially Chicago and New York, others over time – 10GE anticipated Mimics the current Tier1 to Tier2 connectivity via static circuits 17 – 10/28/2015, © 2011 Internet2 Direct Connection to LHCONE-NA Nodes

Two basic approaches discussed – Static connectivity into Internet2 at some other location (e.g. not in Chicago or New York) Facilitates end sites with this network option already in place – Dynamic connectivity via the ION service Inexpensive way to manage traffic through existing network connections Takes advantage of newly deployed infrastructure for DYNES 18 – 10/28/2015, © 2011 Internet2 Layer2 Connectivity via Internet2 Network

Option that will appeal to many Tier3 facilities without dedicated connections for science traffic Cost effective – Additional hardware is not needed – In most cases, R&E IP access is sufficient (e.g. 10G or less) Use the R&E connectivity of their institution – Best effort in terms of bandwidth – Harder to manage traffic flows 19 – 10/28/2015, © 2011 Internet2 Layer3 Connectivity via Internet2 Network

DYNES is in deployment, demonstrations at major conferences expected (SC11) LHCONE Demonstration in Summer 2011 – – LHCONE NA meeting scheduled for May 2011 in Washington DC (participation welcome) Future Work – LHCONE is just the beginning – Opportunity to provide a nationwide “science focused” infrastructure for all VOs Dedicated Bandwidth Cutting edge technology (Open Flow, etc.) Integration with International efforts Open Access and Open Standards 20 – 10/28/2015, © 2011 Internet2 Conclusions/Next Steps

Scientific Networking: The Cause of and Solution to All Problems April 18 th 2011, Workshop on High Performance Applications of Cloud and Grid Tools Jason Zurawski, Research Liaison For more information, visit 21 – 10/28/2015, © 2011 Internet2