Presentation is loading. Please wait.

Presentation is loading. Please wait.

“The Pacific Research Platform”

Published byRebecca Gallagher Modified over 9 years ago

Similar presentations

Presentation on theme: "“The Pacific Research Platform”"— Presentation transcript:

1 “The Pacific Research Platform”
Briefing to The Quilt Visit to Calit2’s Qualcomm Institute University of California, San Diego February 10, 2016 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD

2 Use Lightpaths to Connect All Data Generators and Consumers,
Vision: Creating a West Coast “Big Data Freeway” Connected by CENIC/Pacific Wave Use Lightpaths to Connect All Data Generators and Consumers, Creating a “Big Data” Freeway Integrated With High Performance Global Networks “The Bisection Bandwidth of a Cluster Interconnect, but Deployed on a 20-Campus Scale.” This Vision Has Been Building for Over a Decade

3 NSF’s OptIPuter Project: Demonstrating How SuperNetworks Can Meet the Needs of Data-Intensive Researchers LS Slide 2005 $13,500,000 OptIPortal– Termination Device for the OptIPuter Global Backplane In August 2003, Jason Leigh and his students used RBUDP to blast data from NCSA to SDSC over the TeraGrid DTFnet, achieving18Gbps file transfer out of the available 20Gbps Calit2 (UCSD, UCI), SDSC, and UIC Leads—Larry Smarr PI Univ. Partners: NCSA, USC, SDSU, NW, TA&M, UvA, SARA, KISTI, AIST Industry: IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent

4 DOE ESnet’s Science DMZ: A Scalable Network Design Model for Optimizing Science Data Transfers
A Science DMZ integrates 4 key concepts into a unified whole: A network architecture designed for high-performance applications, with the science network distinct from the general-purpose network The use of dedicated systems for data transfer Performance measurement and network testing systems that are regularly used to characterize and troubleshoot the network Security policies and enforcement mechanisms that are tailored for high performance science environments The DOE ESnet Science DMZ and the NSF “Campus Bridging” Taskforce Report Formed the Basis for the NSF Campus Cyberinfrastructure Network Infrastructure and Engineering (CC-NIE) Program Science DMZ Coined 2010

5 Based on Community Input and on ESnet’s Science DMZ Concept, NSF Has Funded Over 100 Campuses to Build Local Big Data Freeways Red CC-NIE Awardees Yellow 2013 CC-NIE Awardees Green CC*IIE Awardees Blue CC*DNI Awardees Purple Multiple Time Awardees CC-NIE / CC*IIE / CC*DNI Programs Source: NSF

6 The Pacific Research Platform: The Next Logical Step – Connect Multiple Campus Science DMZs with Gbps Lightpaths NSF CC*DNI Grant $5M 10/ /2020 PI: Larry Smarr, UC San Diego Calit2 Co-Pis: Camille Crittenden, UC Berkeley CITRIS, Tom DeFanti, UC San Diego Calit2, Philip Papadopoulos, UC San Diego SDSC, Frank Wuerthwein, UC San Diego Physics and SDSC

7 FIONA – Flash I/O Network Appliance: Termination Device for 10-100Gbps Flows
FIONAs Are Science DMZ Data Transfer Nodes & Optical Network Termination Devices UCSD CC-NIE Prism Award & UCOP Phil Papadopoulos & Tom DeFanti Joe Keefe & John Graham UCOP Rack-Mount Build: John Graham, Calit2’s QI Cost $8,000 $20,000 Intel Xeon Haswell Multicore E v3 6-Core 2x E v3 14-Core RAM 128 GB 256 GB SSD SATA 3.8 TB Network Interface 10/40GbE Mellanox 2x40GbE Chelsio+Mellanox GPU NVIDIA Tesla K80 RAID Drives 0 to 112TB (add ~$100/TB)

8 FIONAs as Uniform DTN End Points
FIONA DTNs UC FIONAs Funded by UCOP “Momentum” Grant FIONAs as Uniform DTN End Points Existing DTNs As of October 2015

9 FIONA DTNs Now Deployed to All UC Campuses
Ten Week Sprint to Demonstrate the West Coast Big Data Freeway System: PRPv0 FIONA DTNs Now Deployed to All UC Campuses And Most PRP Sites Presented at CENIC 2015 March 9, 2015

10 Pacific Research Platform Multi-Campus Science Driver Teams
Jupyter Hub Biomedical Cancer Genomics Hub/Browser Microbiome and Integrative ‘Omics Integrative Structural Biology Earth Sciences Data Analysis and Simulation for Earthquakes and Natural Disasters Climate Modeling: NCAR/UCAR California/Nevada Regional Climate Data Analysis CO2 Subsurface Modeling Particle Physics Astronomy and Astrophysics Telescope Surveys Galaxy Evolution Gravitational Wave Astronomy Scalable Visualization, Virtual Reality, and Ultra-Resolution Video Campus Cyberinfrastructure – Network Infrastructure and Engineering (CC-NIE) Campus Cyberinfrastructure – Infrastructure, Innovation, and Engineering (CC-IIE) Campus Cyberinfrastructure – Data, Networking, and Innovation (CC-DNI) NSF incorporates Data Infrastructure Building Blocks (CC-DNI-DIBBs) – Multi-Campus / Multi-Institution Model Implementation from Program Solicitation NSF

11 PRP First Application: Distributed IPython/Jupyter Notebooks: Cross-Platform, Browser-Based Application Interleaves Code, Text, & Images IScilab IMatlab ICSharp Bash Clojure Kernel Hy Kernel Redis Kernel jove, a kernel for io.js IJavascript Calysto Scheme Calysto Processing idl_kernel Mochi Kernel Lua (used in Splash) Spark Kernel Skulpt Python Kernel MetaKernel Bash MetaKernel Python Brython Kernel IVisual VPython Kernel IJulia IHaskell IFSharp IRuby IGo IScala IMathics Ialdor LuaJIT/Torch Lua Kernel IRKernel (for the R language) IErlang IOCaml IForth IPerl IPerl6 Ioctave Calico Project kernels implemented in Mono, including Java, IronPython, Boo, Logo, BASIC, and many others Source: John Graham, QI

12 PRP UC-JupyterHub Backbone Next Step: Deploy Across PRP
Source: John Graham, Calit2 UC Berkeley UC San Diego GPU JupyterHub: 2 x 14-core CPUs 256GB RAM 1.2TB FLASH 3.8TB SSD Nvidia K80 GPU Dual 40GbE NICs And a Trusted Platform Module GPU JupyterHub: 1 x 18-core CPUs 128GB RAM 3.8TB SSD Nvidia K80 GPU Dual 40GbE NICs And a Trusted Platform Module 40Gbps

13 OSG Federates Clusters in 40/50 States: A Major XSEDE Resource
Source: Miron Livny, Frank Wuerthwein, OSG

14 Open Science Grid Has Had a Huge Growth Over the Last Decade - Currently Federating Over 130 Clusters Source: Miron Livny, Frank Wuerthwein, OSG CMS Crossed 100 Million Core-Hours/Month In Dec 2015 Supported Over 200 Million Jobs In 2015 ATLAS Over 1 Billion Data Transfers Moved 200 Petabytes In 2015

15 PRP Prototype of LambdaGrid Aggregation of OSG Software & Services Across California Universities in a Regional DMZ ATLAS CMS other physics life sciences other sciences Aggregate Petabytes of Disk Space & PetaFLOPs of Compute, Connected at Gbps Transparently Compute on Data at Their Home Institutions & Systems at SLAC, NERSC, Caltech, UCSD, & SDSC UCD SLAC OSG Hours by Science Domain UCSC CSU Fresno PRP Builds on SDSC’s LHC-UC Project UCSB Caltech UCR UCI Source: Frank Wuerthwein, UCSD Physics; SDSC; co-PI PRP UCSD & SDSC

16 Two Automated Telescope Surveys Creating Huge Datasets Will Drive PRP
Precursors to LSST and NCSA PRP Allows Researchers to Bring Datasets from NERSC to Their Local Clusters for In-Depth Science Analysis 300 images per night. 100MB per raw image 30GB per night 120GB per night 250 images per night. 530MB per raw image 150 GB per night 800GB per night When processed at NERSC Increased by 4x Source: Peter Nugent, Division Deputy for Scientific Engagement, LBL Professor of Astronomy, UC Berkeley

17 Creates 10M Alerts/Night Within 1 Minute of Observing
Global Scientific Instruments Will Produce Ultralarge Datasets Continuously Requiring Dedicated Optic Fiber and Supercomputers Square Kilometer Array Large Synoptic Survey Telescope Tracks ~40B Objects, Creates 10M Alerts/Night Within 1 Minute of Observing 2x40Gb/s

18 PRP Will Support the Computation and Data Analysis in the Search for Sources of Gravitational Radiation Augment the aLIGO Data and Computing Systems at Caltech, by connecting at 10Gb/s to SDSC Comet supercomputer, enabling LIGO computations to enter via the same PRP “job cache” as for LHC.

19 PRP CENIC 10G Link UCSD to SDSU
HPWREN Users and Public Safety Clients Gain Redundancy and Resilience from PRP Upgrade PRP CENIC 10G Link UCSD to SDSU DTN FIONAs Endpoints Data Redundancy Disaster Recovery High Availability Network Redundancy San Diego Countywide Sensors and Camera Resources UCSD & SDSU Data & Compute UCSD UCR SDSU UCI 10X Increase During Wildfires Data From Hans-Werner Braun UCI & UCR Data Replication and PRP FIONA Anchors as HPWREN Expands Northward Source: Frank Vernon, Greg Hidley, UCSD

20 PRP Backbone Sets Stage for 2016 Expansion of HPWREN, Connected to CENIC, into Orange and Riverside Counties Anchor to CENIC at UCI PRP FIONA Connects to CalREN-HPR Network Data Replication Site Potential Future UCR CENIC Anchor Camera and Relay Sites at: Santiago Peak Sierra Peak Lake View Bolero Peak Modjeska Peak Elsinore Peak Sitton Peak Via Marconi UCR UCI UCSD Collaborations through COAST – County of Orange Safety Task Force SDSU Source: Frank Vernon, Greg Hidley, UCSD

21 PRP Links FIONA Clusters Creating Distributed Virtual Reality
20x40G PRP-connected 40G FIONAs San Diego Merced

22 PRP is NOT Just for Big Data Science and Engineering: Linking Cultural Heritage and Archaeology Datasets UCD UCSF Stanford NASA AMES/ NREN UCSC UCSB Caltech USC UCLA UCI UCSD SDSU UCR Esnet DoE Labs UW/ PNWGP Seattle Berkeley UCM Los Nettos Internet2 Building on CENIC’s Expansion To Libraries, Museums, and Cultural Sites We already have 11 major research universities in California poised to partner. “In an ideal world –Extremely high bandwidth to move large cultural heritage datasets around the PRP cloud for processing & viewing in CAVEs around PRP with Unlimited Storage for permanent archiving.” -Tom Levy, UCSD * Institutions with Active Archaeology Programs Note: This diagram represents a subset of sites and connections.

23 Current International
Next Step: Global Research Platform Building on CENIC/Pacific Wave and GLIF Current International GRP Partners

Download ppt "“The Pacific Research Platform”"

Similar presentations

Calit2-Living in the Future Keynote Sharecase 2006 University of California, San Diego March 29, 2006 Dr. Larry Smarr Director, California Institute.

Calit2-Living in the Future " Keynote Sharecase 2006 University of California, San Diego March 29, 2006 Dr. Larry Smarr Director, California Institute.
Bringing Mexico Into the Global LambdaGrid Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber.

Bringing Mexico Into the Global LambdaGrid Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber.
High Performance Cyberinfrastructure Discovery Tools for Data Intensive Research Larry Smarr Prof. Computer Science and Engineering Director, Calit2 (UC.

High Performance Cyberinfrastructure Discovery Tools for Data Intensive Research Larry Smarr Prof. Computer Science and Engineering Director, Calit2 (UC.
Why Optical Networks Are Emerging as the 21 st Century Driver Scientific American, January 2001.

Why Optical Networks Are Emerging as the 21 st Century Driver Scientific American, January 2001.
The OptIPuter: an IP Over Lambda Testbed Invited Talk NREN Workshop VII: Optical Network Testbeds (ONT) NASA Ames Research Center Mountain View, CA August.

"The OptIPuter: an IP Over Lambda Testbed" Invited Talk NREN Workshop VII: Optical Network Testbeds (ONT) NASA Ames Research Center Mountain View, CA August.
The First Year of Cal-(IT) 2 Report to The University of California Regents UCSF San Francisco, CA March 13, 2002 Dr. Larry Smarr Director, California.

The First Year of Cal-(IT) 2 Report to The University of California Regents UCSF San Francisco, CA March 13, 2002 Dr. Larry Smarr Director, California.
“A California-Wide Cyberinfrastructure for Data-Intensive Research” Invited Presentation CENIC Annual Retreat Santa Rosa, CA July 22, 2014 Dr. Larry Smarr.

“A California-Wide Cyberinfrastructure for Data-Intensive Research” Invited Presentation CENIC Annual Retreat Santa Rosa, CA July 22, 2014 Dr. Larry Smarr.
AHM Overview OptIPuter Overview Third All Hands Meeting OptIPuter Project San Diego Supercomputer Center University of California, San Diego January 26,

AHM Overview OptIPuter Overview Third All Hands Meeting OptIPuter Project San Diego Supercomputer Center University of California, San Diego January 26,
“Building US/Mexico Collaborations Using Optical Networks” Opening Workshop Welcome Big Data Big Network 2 Calit2’s Qualcomm Institute February 10, 2014.

“Building US/Mexico Collaborations Using Optical Networks” Opening Workshop Welcome Big Data Big Network 2 Calit2’s Qualcomm Institute February 10, 2014.
SAN DIEGO SUPERCOMPUTER CENTER Emerging HIPAA and Protected Data Requirements for Research Computing at SDSC Ron Hawkins Director of Industry Relations.

SAN DIEGO SUPERCOMPUTER CENTER Emerging HIPAA and Protected Data Requirements for Research Computing at SDSC Ron Hawkins Director of Industry Relations.
PRISM: High-Capacity Networks that Augment Campus’ General Utility Production Infrastructure Philip Papadopoulos, PhD. Calit2 and SDSC.

PRISM: High-Capacity Networks that Augment Campus’ General Utility Production Infrastructure Philip Papadopoulos, PhD. Calit2 and SDSC.
The ADAMANT Project: Linking Scientific Workflows and Networks “Adaptive Data-Aware Multi-Domain Application Network Topologies” Ilia Baldine, Charles.

The ADAMANT Project: Linking Scientific Workflows and Networks “Adaptive Data-Aware Multi-Domain Application Network Topologies” Ilia Baldine, Charles.
SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA; SAN DIEGO IEEE Symposium of Massive Storage Systems, May 3-5, 2010 Data-Intensive Solutions.

SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA; SAN DIEGO IEEE Symposium of Massive Storage Systems, May 3-5, 2010 Data-Intensive Solutions.
“An Integrated West Coast Science DMZ for Data-Intensive Research” Panel CENIC Annual Conference University of California, Irvine Irvine, CA March 9, 2015.

“An Integrated West Coast Science DMZ for Data-Intensive Research” Panel CENIC Annual Conference University of California, Irvine Irvine, CA March 9, 2015.
“Introduction to UC San Diego’s Integrated Digital Infrastructure” Opening Talk IDI Showcase 2015 University of California, San Diego May 6-7, 2015 Dr.

“Introduction to UC San Diego’s Integrated Digital Infrastructure” Opening Talk IDI Showcase 2015 University of California, San Diego May 6-7, 2015 Dr.
1 Building National Cyberinfrastructure Alan Blatecky Office of Cyberinfrastructure EPSCoR Meeting May 21, 2012 1.

1 Building National Cyberinfrastructure Alan Blatecky Office of Cyberinfrastructure EPSCoR Meeting May 21,
“A UC-Wide Cyberinfrastructure for Data-Intensive Research” Invited Presentation UC IT Leadership Council Oakland, CA May 19, 2014 Dr. Larry Smarr Director,

“A UC-Wide Cyberinfrastructure for Data-Intensive Research” Invited Presentation UC IT Leadership Council Oakland, CA May 19, 2014 Dr. Larry Smarr Director,
Open Science Grid For CI-Days Internet2: Fall Member Meeting, 2007 John McGee – OSG Engagement Manager Renaissance Computing Institute.

Open Science Grid For CI-Days Internet2: Fall Member Meeting, 2007 John McGee – OSG Engagement Manager Renaissance Computing Institute.
Why Optical Networks Will Become the 21 st Century Driver Scientific American, January 2001 Number of Years 012345 Performance per Dollar Spent Data Storage.

Why Optical Networks Will Become the 21 st Century Driver Scientific American, January 2001 Number of Years Performance per Dollar Spent Data Storage.
“An Integrated Science Cyberinfrastructure for Data-Intensive Research” Panel CISCO Executive Symposium San Diego, CA June 9, 2015 Dr. Larry Smarr Director,

“An Integrated Science Cyberinfrastructure for Data-Intensive Research” Panel CISCO Executive Symposium San Diego, CA June 9, 2015 Dr. Larry Smarr Director,

Similar presentations

Ads by Google