SCEC Community Modeling Environment (SCEC/CME) Philip J. Maechling SCEC4 Site Review June 21, 2010

SCEC3 Organization Special Projects Disciplinary Committees SCEC Director Board of Directors External Advisory Council Information Architect Planning Committee Center Administration CEO Program CME PetaSHA Earthquake Geology Unified Structural Representation Seismic Hazard & Risk Analysis WGCEP Tectonic Geodesy Fault & Rupture Mechanics Knowledge Transfer CSEP Seismology Crustal Deformation Modeling Public Outreach SoSAFE Lithospheric Architecture & Dynamics K-12 & Informal Education ExGM Earthquake Forecasting & Predictability Experiential Learning & Career Advancement Ground Motion Prediction Special Projects Disciplinary Committees CEO Activities Focus Groups 2

Seismic Hazard Analysis Requires Integration of Computational and Structural Models Seismicity Paleoseismology Local site effects Geologic structure Faults Seismic Hazard Model SHA is a system-level problem, requiring information from geology, seismology, geodesy, rock mechanics, computational physics, and other disciplines. Like other system-level problems, it depends on model-based inference. Stress transfer Rupture dynamics Crustal motion Crustal deformation Seismic velocity structure

Overview SCEC/CME Computational Research SCEC/CME HPC Collaborative Research SCEC/CME Forecast Testing and Evaluation

SCEC/CME Scientific Research Objectives Improve the resolution of dynamic rupture simulations by an order of magnitude to investigate realistic friction laws, near-fault stress states, and off-fault plasticity. Investigate the upper frequency limit of deterministic ground-motion prediction by simulating strong motions above 3 Hz using realistic 3D structural models for Southern California. Validate and improve the Southern California structural models using full 3D waveform tomography. Transform probabilistic seismic hazard analysis (PSHA) into a physics-based science.

Community Modeling Environment (CME) SCEC/CME is a collaboration of SCEC scientists and computer scientists that performs computationally-intensive seismic hazard research. Current SCEC/CME NSF research awards include: PetaSHA (NSF/EAR Geoinformatics) PetaShake (NSF/OCI PetaApps) SCEC Blue Waters (NSF/OCI PRAC) SCEC/CME projects include: CSEP (W. M. Keck Foundation) CISN Earthquake Early Warning Testing (USGS NEHRP) SCEC Broadband Platform (Pacific Gas and Electric)

SCEC Computational Research Experience SCEC/CME experience is that it requires more than a single program to get a significant computational research result. SCEC/CME research computing typically requires: Carefully prepared input data Multiple simulation codes Verification and validation problems and solutions Scientific and software expertise Software and computer infrastructure

SCEC Computational Platform Concept Definition of Computational Platform A vertically integrated collection of hardware, software, and people that can repeatedly produce a useful research result Computational Platform Characteristics Validated simulation software and geophysical models Re-usable simulation capabilities Imports parameters from other systems. Exports results to other systems IT/geoscience collaboration involved in operation Access to High-performance hardware and large scale data and metadata management. May use Workflow management tools

CME Project Funding

Overview SCEC/CME Computational Research SCEC/CME HPC Collaborative Research SCEC/CME Forecast Testing and Evaluation

SCEC/CME High Performance Computing (HPC) Research SCEC/CME seeks to improve predictive ground motion forecast models by continuing to integrate more realistic physics into the numerical simulations. Increasing seismic hazard model complexity and accuracy increases computing requirements As simulations become more physically realistic, the computational and data management requirements increase enormously

SCEC Pursuing Leadership Class Computer Systems Data-oriented Science and Engineering Environment Scientific Computing SCEC 100 TF Systems 10’s of Projects SCEC Key function of the NSF Supercomputer Centers: Provide facilities over and above what can be found in the typical campus/lab environment 10’s of 10 TF Systems 1,000’s of Users HPC Centers Data (more BYTES) 100’s of 1 TF Systems 10,000’s of Users Home, Lab, Campus, Desktop Traditional HPC environment Departmental HPC GigaFLOPS Millions of Users Workstations Compute (more FLOPS) SCEC: An NSF + USGS Research Center

TeraGrid Computing Resources SCEC, USC HPCC, TeraGrid resources used for SCEC Research USC HPCC Resources Grid SCEC Computing Resources TeraGrid Computing Resources SCEC: An NSF + USGS Research Center 13

SCEC is a interdisciplinary, multi-institutional center originally founded as an STC and now jointly funded by NSF/EAR and USGS. Perspective of a large user community that is aggressively using the TeraGrid for system-level science of national importance. Mechanism through which we engage the CS community and IT providers is the SCEC Collaboratory, which has been building a new cyberinfrastructure for earthquake system science called the CME. Funded by CISE+GEO under NSF/ITR SCEC: An NSF + USGS Research Center

HPC Resources Utilized by SCEC Resource Provider Site Est. 2010 SU (Million Hours) Est. Storage (TB) USC – HPCC USC 1 70 NSF - TeraGrid NICS 35 150 PSC 2 50 TACC 25 200 NCSA 15 SDSC 4 300 DOE-INCITE Argonne 10 Oak Ridge NSF-Blue Waters Open Science Grid Totals 118 M 1,321 TB

Computer Scientist Resources Provided to SCEC by HPC Organizations SDSC Strategic Application Collaborations (SAC) - 1.0 FTE x 3yrs MPI code Improvements, Simulation Results Archive - 0.5 FTE x 2yrs Scientific Visualization Advanced Support for TeraGrid Applications (ASTA) - 1.0 FTE x 3yrs Simulation support and optimizations Summary: HPC Resource Providers have provided more than 7 person-years of computer science support for SCEC/CME research during SCEC3

SCEC HPC Computer Allocations

SCEC MPI Code Peak Performance

Petascale Research in Earthquake System Science on Blue Waters PI: Thomas H. Jordan - University of Southern California Co-PI: Jacobo Bielak - Carnegie Mellon University Southern California Earthquake Center (SCEC)

Overview SCEC/CME Computational Research SCEC/CME HPC Collaborative Research SCEC/CME Forecast Testing and Evaluation

USGS hazard mapping results in dramatic change in building codes Seismic element of 1996 Building Codes based on 1970’s maps Seismic element of 2000 & 2003 Int’l Building Code based on the 1996 USGS national seismic hazard map

Forecast Testing Should Increase Along with Forecast Impact SCEC Computational Forecast Users Scientific and Engineering Requirements for Forecast Modeling Systems Public and Governmental Forecasts Automated prospective testing of forecast models over time within collaborative forecast testing center. Engineering and Interdisciplinary Research Automated retrospective testing of forecast models using community defined validation problems. Collaborative Research Project Computational codes, structural models, and simulation results versioned with associated tests. Individual Research Project Development of new computational, data, and physical models.

CISN EEW Algorithm Performance Testing Developments CISN EEW Performance Testing Developments: Implemented the Location/Magnitude oriented performance Summaries. Generating nightly performance summaries (2mo,4mo,6mo and cumulative time periods). Generating Summaries for ElarmS (UCB), TauC (Caltech),and Virtual Seismologist ( ETH) CISN EEW Testing site available at: http://www.scec.org/eew Magnitude Accuracy Actual Warning Delay with Current CISN Network Theoretical Warning Delay with Zero Telemetry and Processing Delays 23

SCEC/CME Web Site: http://ww.scec.org/cme End