IRNC Special Projects: IRIS and DyGIR Eric Boyd, Internet2 October 5, 2011

International Research Network Connections (IRNC) is an NSF Office of Cyberinfrastructure program to enable collaboration, in the research and education community, on a global scale. – These grants facilitate hardware and software solutions to foster access to remote instruments, data, and computational resources located throughout the world. Internet2 was awarded 2 IRNC Special Projects (SP) awards in May of 2010: – IRIS - The International Research Instrumentation System Performance Monitoring using perfSONAR-ps software package – DyGIR - Dynamic Gateway for International Research Building dynamic circuits using OSCARS software package Internet2 was awarded 1 MRI award in August of 2010: – DYNES –DYnamic NEtwork System Deploying equipment to build a distributed virtual instrument 2 – 9/7/2015, © 2011 Internet2 IRNC Program

Campus 3 – 9/7/2015, © 2009 Internet2

Region 4 – 9/7/2015, © 2009 Internet2

Country 5 – 9/7/2015, © 2009 Internet2

World 6 – 9/7/2015, © 2009 Internet2

Protocol – IDC (DICE) – NSI (OGF NSI WG) – “OpenFlow Peering” (GENI, NDDI) Software – OSCARS (Internet2 and ESnet) – OSCARS / DRAGON (Internet2) – AUTOBAHN (GÉANT) – DRAC (NORTEL / SURFNET) – Others … Hardware / Operations – Campus / Region (Campuses, Regionals, Internet2 through DYNES) – National Networks (Internet2, Esnet) – International Links (IRNC Program) – NRENs and CRENs (Nation-specific and continent-specific funding) Components of E2E Dynamic Circuit Infrastructure 7 – 9/7/2015, © 2009 Internet2

Protocol – perfSONAR (OGF NMC WG) – perfSONAR with support for network research slicing (GENI I&M, NDDI) Software – perfSONAR-ps (ESnet and Internet2) – perfSONAR-MDM (GÉANT) Hardware / Operations – Campus / Region (Campuses, Regionals, Internet2 through DYNES) – National Networks (Internet2, Esnet) – International Links (IRNC PRONET) – NRENs and CRENs (Nation-specific and continent-specific funding) Components of E2E Performance Monitoring Infrastructure 8 – 9/7/2015, © 2009 Internet2

Software Development – perfSONAR-ps (IRIS, DOE Office of Science) – OSCARS (DyGIR, DOE Office of Science) Hardware – Campus / Region (DYNES) – National (BTOP, DOE) – International Links (IRNC PRONET) Network Engineering and Operations – Campus / Region (DYNES, Campus or Regional Contribution) – National (Internet2, ESnet) – International Links (IRNC PRONET with initial help from IRIS, DyGIR) Funding of OSCARS / perfSONAR-ps 9 – 9/7/2015, © 2009 Internet2

We know how to build operational dynamic services using IDC protocol (which is widely deployed, but not a standard) We know how to build operational performance services using perfSONAR protocol (which is widely deployed and nearly a standard) OGF NSI WG is developing a standard (can think of it as an evolution of the IDC protocol) GLIF is pushing the envelope, promoting functionality and interoperability on a global scale and move to a standard OpenFlow is pushing the envelope under the hood, within and outside of GENI – Initially connect OpenFlow “islands” by Layer 2 VLANs using IDC protocol – Eventually, we’ll figure out what “OpenFlow peering” means How do the pieces fit? 10 – 9/7/2015, © 2009 Internet2

NSF Grant # IRIS will provide a software framework to simplify the task of end-to-end network performance monitoring and diagnostics – Based on the widely deployed perfSONAR-PS infrastructure and protocols – Facilitates broader deployment of perfSONAR enabled resources; increasing the likelihood of diagnostic resources being available along the end-to-end paths Integrates with existing deployments on R&E networks (ESnet, GÉANT, Internet2, Regional and NRENs) as well as those maintained by scientific VOs (USATLAS, LHCOPN, eVLBI, REDDnet) Will work with IRNC ProNet awardees to customize deployment for target network functionality Target end date is April – 9/7/2015, © 2011 Internet2 IRNC SP:IRIS

perfSONAR-PS v3.2.1 release imminent – New features supporting IRNC Pronet awardees Nagios Plug-ins Improved IPv6 support Improved bandwidth/latency analysis Continue to meet with all awardees to advise on hardware choices and best practices for tool deployments IRNC SP:IRIS Development Status 12 – 9/7/2015, © 2009 Internet2

NSF Grant # DyGIR will provide a component based solution for scheduling dynamic circuits on IRNC ProNet infrastructure – Utilizes the OSCARS software suite, developed by ESnet – Integrates circuit statistics and networking monitoring via the perfSONAR-PS framework Capabilities will integrate with existing backbone networks (ESnet SDN, GÉANT AutoBAHN, Internet2 ION), as well as emerging campus deployments (DYNES – an NSF MRI Funded effort). Will work with IRNC ProNet awardees to customize deployment for target network functionality Target end date is April – 9/7/2015, © 2011 Internet2 IRNC SP:DyGIR

OSCARS 0.6 is in testing phase – Release expected Fall, 2011 Brocade switch was selected for ACE – OSCARS port will (?) be needed, work should be complete 1 st quarter 2012 OSCARS ported to support OpenFlow (NOX controller) – Demonstrated at SC11 - SCinet Research Sandbox IRNC SP:DyGIR Development Status 14 – 9/7/2015, © 2009 Internet2

ACE – perfSONAR-PS test points to be available, along with periodic monitoring to select locations within GEANT – Planned IDC at MANLAN 1 st Quarter of 2012 AmLight – IDC available in the AMPATH exchange for use on International Links – IDC Peering with RNP established – perfSONAR-PS monitoring available GLORIAD – Discussed current DYGIR and IRIS activities – Currently have ability to create VLANs on infrastructure. Looking to integrate with dynamic circuit networks over next couple of years. – Exploring current GLORIAD peerings and heavy users to determine what use cases could benefit from dynamic circuits. – Exploring best way to capitalize on perfSONAR – specifically looking at ways to publish passive flow data currently collected by GLORIAD using perfsONAR protocols 15 – 9/7/2015, © 2011 Internet2 IRNC Outreach to ProNET Awardees

TransLight/Pacific Wave – Established working group to install perfSONAR-PS monitoring software at endpoints and participants in the region – Have put out a questionnaire to participants and end sites to determine perfSONAR availability. Some deployments already happening – Evaluating dynamic capabilities TransPAC3 – Have traded topology information and are determining what switches in the infrastructure to put under dynamic control. Will likely peer with ION and JGN2. – Currently has perfSONAR-PS test points, and periodic tests with APAN 16 – 9/7/2015, © 2011 Internet2 IRNC Outreach to ProNET Awardees

Data movement to support science: – Increasing in size (100s of TBs in the LHC World, approaching PB sizes) – Becoming more frequent (multiple times per day) – Reaching more consumers (VO sizes stand to increase, more VOs) – Time sensitivity (data may grow “stale” if not processed immediately) Traditional networking: – R&E or Commodity “IP” connectivity is subject to congestion by other users – Supporting large sporadic flows is challenging for the engineers, and frustrating for the scientists 17 – 9/7/2015, © 2011 Internet2 DYNES Motivation

Solution – Dedicated bandwidth (over the entire end to end path) to move scientific data – Invoke this “on demand” instead of relying on permanent capacity (cost, complexity) – Exists in harmony with traditional IP networking – Connect to facilities that scientists need to access – Integration with data movement applications Invoke the connectivity when the need it, based on network conditions Proposed Deployment: – Software and hardware support spanning domain boundaries Campus Regional Backbone – Integration with existing technologies and deployments 18 – 9/7/2015, © 2011 Internet2 DYNES Motivation

What is it?: – A nationwide cyber-instrument spanning ~40 US universities and ~14 Internet2 connectors Extends Internet2’s ION service into regional networks and campuses, based on OSCARS implementation of IDC protocol (developed in partnership with ESnet) High-performance file store at sites Who is it? – A collaborative team including Internet2, Caltech, University of Michigan, and Vanderbilt University – Community of regional networks and campuses – LHC, astrophysics community, OSG, WLCG, other virtual organizations DYNES Summary 19 – 9/7/2015, © 2011 Internet2

20 – 9/7/2015, © 2011 Internet2 DYNES Projected Topology (October 2011)

September 2011 Demonstration of end-to-end Dynamic Circuit capabilities – International collaborations spanning 3 continents (South America, North America, and Europe) – Use of several software packages OSCARS for inter-domain control of Dynamic Circuits perfSONAR-PS for end-to-end Monitoring FDT to facilitate data transfer over IP or circuit networks – Science components – collaboration in the LHC VO (ATLAS and CMS) – DYNES, IRIS, and DyGIR NSF grants touted 21 – 9/7/2015, © 2011 Internet2 GLIF 2011

22 – 9/7/2015, © 2011 Internet2 GLIF Topology

23 – 9/7/2015, © 2011 Internet2 GLIF Participants