1. SCEC Community Modeling Environment (SCEC/CME)
Philip J. Maechling
SCEC4 Site Review
June 21, 2010

2. SCEC3 Organization Special Projects Disciplinary Committees
SCEC Director
Board of Directors
External Advisory
Council
Information
Architect
Planning
Committee
Center
Administration
CEO Program
CME
PetaSHA
Earthquake Geology
Unified Structural
Representation
Seismic Hazard
& Risk Analysis
WGCEP
Tectonic Geodesy
Fault & Rupture
Mechanics
Knowledge
Transfer
CSEP
Seismology
Crustal Deformation
Modeling
Public Outreach
SoSAFE
Lithospheric
Architecture
& Dynamics
K-12 & Informal
Education
ExGM
Earthquake
Forecasting &
Predictability
Experiential
Learning & Career
Advancement
Ground Motion
Prediction
Special Projects
Disciplinary
Committees
CEO Activities
Focus Groups 2

3. Seismic Hazard Analysis Requires Integration of Computational and Structural Models
Seismicity
Paleoseismology
Local site effects
Geologic structure
Faults
Seismic
Hazard
Model
SHA is a system-level problem, requiring information from geology, seismology, geodesy, rock mechanics, computational physics, and other disciplines.
Like other system-level problems, it depends on model-based inference.
Stress
transfer
Rupture
dynamics
Crustal
motion
Crustal
deformation
Seismic velocity
structure

4. Overview SCEC/CME Computational Research
SCEC/CME HPC Collaborative Research
SCEC/CME Forecast Testing and Evaluation

5. SCEC/CME Scientific Research Objectives
Improve the resolution of dynamic rupture simulations by an order of magnitude to investigate realistic friction laws, near-fault stress states, and off-fault plasticity.
Investigate the upper frequency limit of deterministic ground-motion prediction by simulating strong motions above 3 Hz using realistic 3D structural models for Southern California.
Validate and improve the Southern California structural models using full 3D waveform tomography.
Transform probabilistic seismic hazard analysis (PSHA) into a physics-based science.

6. Community Modeling Environment (CME)
SCEC/CME is a collaboration of SCEC scientists and computer scientists that performs computationally-intensive seismic hazard research. Current SCEC/CME NSF research awards include:
PetaSHA (NSF/EAR Geoinformatics)
PetaShake (NSF/OCI PetaApps)
SCEC Blue Waters (NSF/OCI PRAC)
SCEC/CME projects include:
CSEP (W. M. Keck Foundation)
CISN Earthquake Early Warning Testing (USGS NEHRP)
SCEC Broadband Platform (Pacific Gas and Electric)

7. SCEC Computational Research Experience
SCEC/CME experience is that it requires more than a single program to get a significant computational research result.
SCEC/CME research computing typically requires:
Carefully prepared input data
Multiple simulation codes
Verification and validation problems and solutions
Scientific and software expertise
Software and computer infrastructure

8. SCEC Computational Platform Concept
Definition of Computational Platform
A vertically integrated collection of hardware, software, and people that can repeatedly produce a useful research result
Computational Platform Characteristics
Validated simulation software and geophysical models
Re-usable simulation capabilities
Imports parameters from other systems. Exports results to other systems
IT/geoscience collaboration involved in operation
Access to High-performance hardware and large scale data and metadata management.
May use Workflow management tools

9. CME Project Funding

10. Overview SCEC/CME Computational Research
SCEC/CME HPC Collaborative Research
SCEC/CME Forecast Testing and Evaluation

11. SCEC/CME High Performance Computing (HPC) Research
SCEC/CME seeks to improve predictive ground motion forecast models by continuing to integrate more realistic physics into the numerical simulations.
Increasing seismic hazard model complexity and accuracy increases computing requirements
As simulations become more physically realistic, the computational and data management requirements increase enormously

12. SCEC Pursuing Leadership Class Computer Systems
Data-oriented Science and Engineering Environment
Scientific Computing
SCEC
100 TF Systems
10’s of Projects
SCEC
Key function of the NSF Supercomputer Centers:
Provide facilities over and above what can be found in the typical campus/lab environment
10’s of 10 TF Systems
1,000’s of Users
HPC
Centers
Data (more BYTES)
100’s of 1 TF Systems
10,000’s of Users
Home, Lab, Campus, Desktop
Traditional HPC
environment
Departmental HPC
GigaFLOPS
Millions of Users
Workstations
Compute (more FLOPS)
SCEC: An NSF + USGS Research Center

13. TeraGrid Computing Resources
SCEC, USC HPCC, TeraGrid resources used for SCEC Research
USC HPCC Resources
Grid
SCEC Computing
Resources
TeraGrid Computing Resources
SCEC: An NSF + USGS Research Center 13

14. SCEC is a interdisciplinary, multi-institutional center originally founded as an STC and now jointly funded by NSF/EAR and USGS.
Perspective of a large user community that is aggressively using the TeraGrid for system-level science of national importance.
Mechanism through which we engage the CS community and IT providers is the SCEC Collaboratory, which has been building a new cyberinfrastructure for earthquake system science called the CME.
Funded by CISE+GEO under NSF/ITR
SCEC: An NSF + USGS Research Center

15. HPC Resources Utilized by SCEC
Resource Provider
Site
Est SU
(Million Hours)
Est. Storage (TB)
USC – HPCC
USC 1 70
NSF - TeraGrid
NICS 35
150
PSC 2 50
TACC 25
200
NCSA 15
SDSC 4
300
DOE-INCITE
Argonne 10
Oak Ridge
NSF-Blue Waters
Open Science Grid
Totals
118 M
1,321 TB

16. Computer Scientist Resources Provided to SCEC by HPC Organizations
SDSC Strategic Application Collaborations (SAC)
- 1.0 FTE x 3yrs MPI code Improvements,
Simulation Results Archive
- 0.5 FTE x 2yrs Scientific Visualization
Advanced Support for TeraGrid Applications (ASTA)
- 1.0 FTE x 3yrs Simulation support and
optimizations
Summary: HPC Resource Providers have provided more than 7 person-years of computer science support for SCEC/CME research during SCEC3

17. SCEC HPC Computer Allocations

18. SCEC MPI Code Peak Performance

19. Petascale Research in Earthquake System Science on Blue Waters
PI: Thomas H. Jordan - University of Southern California
Co-PI: Jacobo Bielak - Carnegie Mellon University
Southern California Earthquake Center (SCEC)

20. Overview SCEC/CME Computational Research
SCEC/CME HPC Collaborative Research
SCEC/CME Forecast Testing and Evaluation

21. USGS hazard mapping results in dramatic change in building codes
Seismic element of 1996 Building Codes based on 1970’s maps
Seismic element of 2000 & 2003 Int’l Building Code based on the 1996 USGS national seismic hazard map

22. Forecast Testing Should Increase Along with Forecast Impact
SCEC Computational Forecast Users
Scientific and Engineering Requirements for Forecast Modeling Systems
Public and Governmental Forecasts
Automated prospective testing of forecast models over time within collaborative forecast testing center.
Engineering and Interdisciplinary Research
Automated retrospective testing of forecast models using community defined validation problems.
Collaborative Research Project
Computational codes, structural models, and simulation results versioned with associated tests.
Individual Research Project
Development of new computational, data, and physical models.

23. CISN EEW Algorithm Performance Testing Developments
CISN EEW Performance Testing Developments:
Implemented the Location/Magnitude oriented performance Summaries.
Generating nightly performance summaries (2mo,4mo,6mo and cumulative time periods).
Generating Summaries for ElarmS (UCB), TauC (Caltech),and Virtual Seismologist ( ETH)
CISN EEW Testing site available at:
Magnitude Accuracy
Actual Warning Delay with Current CISN Network
Theoretical Warning Delay with Zero Telemetry and Processing Delays 23

25. SCEC/CME Web Site: http://ww.scec.org/cme End