Slide 1 E-Science: The Impact of Science DMZs on Research Presenter: Alex Berryman Performance Engineer, OARnet Paul Schopis, Marcio Faerman OARnet, Ohio Supercomputer Center

Slide 2 Context and Motivation The goal of the NSF CC-NIE project is to improve the last mile connectivity for e-Science applications that exists all over campus –The last mile is considered the area of the network that extends from the Universities central IT level down to the individual departments –A large number of e-Science applications exist out side of data centers and labs where high speed connections are normally found

Slide 3 Areas of Impact The e-Science applications that benefit from the Science DMZ architecture are those that involve interactive collaboration tools and local visualization of data Medicine –Remote physical therapist conducting exercises with patients removing the need to travel. –Collaborative review and processing of patient records Physics –Remote instrumentation and control of electron microscopes from the classroom –Transfer simulation data from supercomputer center to lab workstations for better interactive visualization

Slide 4 Collaboration Efforts Brazil Gateway with the Ohio State University –Project between University of Sao Paulo and OSU Prof. Tereza Carvalho, LARC USP –Gathering current baseline of network performance using perfSONAR monitoring – Evaluate the interoperability of US and Brazil Science DMZs Dynamic Layer 2 circuits compatibility will be tested –More info

Slide 5 Role of the Performance Engineer The purpose of the Science DMZ is to accelerate the research efforts on campus, not just build faster networks e-Science applications rely on interconnectivity of multiple domains and layers that can effect performance –Is this a hardware or network issue? –Network path: Department -> Central -> Regional -> National -> International Networks The performance engineer is a resource for researchers to troubleshoot current issues and in planning future research proposals

Slide 6 Two Connection Models: Low Friction Layer 3 Layer 2 Circuit (AL2S) Component: Cisco Nexus 7000 (100Gig) NEC PF5820 (40Gig) Campus OpenFlow Switches (10Gig) Data Transfer Node (10Gig) Science DMZ Design at OSU

Slide 7 Connection Models Low-Friction Layer 3 connectivity –Separate business ( , grades) traffic from research flows. This allows research traffic to bypass the firewalls that are required for day-to-day operations in a normal network –Useful in applications that support normal TCP/IP traffic Dynamic Layer 2 Circuits –Some applications only work layer 2 Emerging data transfer protocols (RoCE) GENI dataplane connectivity Remote Instrumentation tools

Slide 8 Researcher Network-as-a-Service Dashboard Researcher uses Shibboleth credentials to define Application and end points. Researchers also classify the data’s privacy type. After approval, Performance Engineer pushes endpoint info into a RESTful API that is polled by OpenFlow controller. Real-time software defined network monitoring of critical network parameters for bottleneck identification and troubleshooting

Slide 9 Multi-physics Use Case (Low Friction Layer 3) Researchers in OSU’s ME Department move large files to and from HPC at WPAFB in Dayton, OH. (80 Miles West of Central Ohio) Problems were discovered at different layers: 1.Transfers of 80Gb files take over 12 hours 2.Hosts are using shared and firewalled 1Gig connection 3.The DREN path was not being correctly advertised to OSU by OARnet and was going all the way to Los Angeles 4.Researchers are using SCP Planned Actions: 1.Use OpenFlow to switch research traffic onto a non-firewalled dedicated fiber to OSU core 2.Correct BGP Peering so OSU uses OARnet’s direct connection to DREN 3.Still using SCP due to lack of control on DoD HPC side Transfers currently take under two hours, but we are working of improving this further.

Slide 10 Multi-physics Topology (Low Friction Layer 3)

Slide 11 Transfer Neuroblastoma cell images from University of Missouri for processing at OSU using custom ADTS tool: Supports RDMA over converged Ethernet (RoCE) Falls back to TCP based transfers if necessary RoCE Protocol Requirements: Layer 2 Connection RDMA compatible network cards 10Gig bandwidth Lowest latency possible Biomedical Use Case (Layer 2 Circuit)

Slide 12 Science DMZ Security Real time utilization and event monitoring using sFlow If sFlow detects an event the traffic flow is mirrored to a Bro Cluster for packet inspection –Bro Cluster is a capable of monitoring ~40Gbit/s –Once a flow from the 100Gbit link is deemed safe, or not malicious it can be removed from the Bro Cluster All flows are tied to an application in Science DMZ Dashboard –This dashboard maintains the identity of application owner and who approved the access incase security concerns arise

Slide 13 Thank you! Questions? Contact –Alex Berryman, Performance Engineer, OARnet