SCEC Community Modeling Environment (SCEC/CME): Cyberinfrastructure for Earthquake Science Philip Maechling Southern California Earthquake Center University of Southern California SCEC/UseIT Intern Program June 6, 2005

People on the SCEC/CME Project 02/13/04

SCEC/CME Researchers Principal Investigators: Research Leads: Tom Jordan (USC) Bernard Minster (Scripps Institution of Oceanography) Carl Kesselman (USC/ISI) Reagan Moore (San Diego Supercomputer Center) Research Leads: Ned Field (USGS) -- Jacobo Bielak (CMU) Kim Olsen (SDSU) -- Dave O’Hallaron (CMU) Steve Day (SDSU) -- Ralph Archuleta (UCSB) Tim Ahern (IRIS) -- Hans Chalupsky (ISI) Yolanda Gil (ISI) Project Manager: Phil Maechling (USC) 02/13/04

SCEC/CME Project NSF SCEC/ITR Project Goal: To develop a cyberinfrastructure that can support system-level earthquake science – the SCEC Community Modeling Environment (CME) Support: 5-yr project funded by the NSF/ITR program under the CISE and Geoscience Directorates Start date: Oct 1, 2001 NSF CISE GEO SCEC/ITR Project ISI USGS Information Science Earth Science SDSC IRIS SCEC Institutions www.scec.org/cme 02/13/04

Seismic Hazard Analysis Definition: Specification of the maximum intensity of shaking expected at a site during a fixed time interval Example: National seismic hazard maps (http://geohazards.cr.usgs.gov/eq/) Intensity measure: peak ground acceleration (PGA) Interval: 50 years Probability of exceedance: 2% 02/13/04

Seismic Hazard Analysis Definition: Specification of the maximum intensity of shaking expected at a site during a fixed time interval Example: National seismic hazard maps (http://geohazards.cr.usgs.gov/eq/) Intensity measure: peak ground acceleration (PGA) Interval: 50 years Probability of exceedance: 2% 02/13/04

Risk Analysis: A System-Level Problem Risk = Probable Loss (lives & dollars) = Hazard  Exposure  Fragility Faulting, shaking, landsliding, liquifaction Extent & density of built environment Structural fragility 02/13/04

The FEMA 366 Report “HAZUS’99 Estimates of Annual Earthquake Losses for the United States”, September, 2000 U.S. annualized earthquake loss (AEL) is about $4.4 billion/yr. For 25 states, AEL > $10 million/yr 74% of the total is concentrated in California 25% is in Los Angeles County alone 02/13/04

Pathway 1: Puente Hills M 7.1 Scenario Peak Ground Acceleration (% g) Downtown LA 0 - 12 12 - 24 24 - 36 36 - 48 48 - 60 60 - 72 Los Angeles County 02/13/04

Three Global Geosystems Climate System Three Global Geosystems Mantle Convection Atmosphere Hydrosphere Cryosphere Lithosphere Asthenosphere Deep Mantle Outer Core Biosphere Inner Core Core Dynamo 02/13/04

SHA Computational Pathways 1 Standardized Seismic Hazard Analysis Ground motion simulation Physics-based earthquake forecasting Ground-motion inverse problem 2 3 Other Data Geology Geodesy 4 Improvement of models Unified Structural Representation Invert 4 Faults Motions Stresses Anelastic model Ground Motions Physics-based simulations FSM RDM AWM SRM 3 2 Empirical relationships Earthquake Forecast Model Attenuation Relationship Intensity Measures 1 FSM = Fault System Model RDM = Rupture Dynamics Model AWP = Anelastic Wave Propagation SRM = Site Response Model 02/13/04

KNOWLEDGE REPRESENTATION  & REASONING SCEC Community Modeling Environment A collaboratory for system-level earthquake science KNOWLEDGE REPRESENTATION  & REASONING Knowledge Server Knowledge base access, Inference Translation Services Syntactic & semantic translation Knowledge Base DIGITAL LIBRARIES Navigation & Queries Versioning, Replication Mediated Collections Federated access Ontologies Curated taxonomies, Relations & constraints Pathway Models Pathway templates, Models of simulation codes KNOWLEDGE ACQUISITION Acquisition Interfaces Dialog planning, Pathway construction strategies Pathway Assembly Template instantiation, Resource selection, Constraint checking Code Repositories FSM RDM AWM SRM Users Data Collections Data & Simulation Products GRID Pathway Execution Policy, Data ingest, Repository access Grid Services Compute & storage management, Security Pathway Instantiations Computing Storage 02/13/04

SCEC/CME Computational Pathway Construction A major SCEC/CME objective is the ability to construct and run complex computational pathways for SHA Define Scenario Earthquake Lat/Long/Amp (xyz file) with 3000 datapoints (100Kb) ERF Definition Calculate Hazard Curves Extract IMR Value Plot Hazard Map 9000 Hazard Curve files (9000 x 0.5 Mb = 4.5Gb) IMR Definition GMT Map Configuration Parameters Gridded Region Definition Probability of Exceedence and IMR Definition Pathway 1 example 02/13/04

Example Application of Pathway 1: Scenarios for M 7 Example Application of Pathway 1: Scenarios for M 7.4 Southern San Andreas Rupture Without soil & basin effects With soil & basin effects Courtesy of Ned Field, USGS, Pasadena 02/13/04

SCEC Collaboratory for system-level earthquake science 02/13/04

PGV data for Northridge from SCSN System Pathway Comparisons SCEC/CME computational testbed was used to generate PGV Hazard Maps utilizing Pathway 1 and Pathway 2 data sets and SCSN observed data. Pathway 1 (All Firm Soil) Pathway 1 (SCEC CVM 3.0) Pathway 2 (Olsen AWM) PGV data for Northridge from SCSN System (Pathway 0) 02/13/04

SCEC IT Challenges Many geophysical models, computational programs, and data sets and data types. Large scale simulations, high performance computing and large-scale data management are required in a physics-based approach to earthquake modeling at high spatial-temporal resolution requires. Communication tools, distributed model development, and computer resource sharing are required by the distributed SCEC collaboration. 02/13/04

SCEC/CME Research Areas Geoscience Research Areas: Probabilistic Seismic Hazard Analysis Anelastic Wave Propagation Modeling Rupture Dynamics Modeling Data Inversion IT Research Areas: High Performance Computing Grid Digital Library Knowledge Representation and Reasoning 4D Data Visualization Creation of Computational Pathways Web Services Data Integration Data Standards Community Computational Models Outreach and Education: Undergraduate and Graduate Research Opportunities Access to non-scientific Users (Emergency Management, Public) 02/13/04

Composition Analysis Tool (CAT) Interface User building a pathway specification from library of components Errors and fixes generated by ErrorScan algorithm 02/13/04

Earthquake Simulation SCEC ITR Collaboration The SCEC Earthquake Simulation SCEC ITR Collaboration

Major Earthquakes on the San Andreas Fault, 1680-present 1906 M 7.8 Major Earthquakes on the San Andreas Fault, 1680-present 1857 M 7.9 ~1680 M 7.7 02/13/04

TeraShake Simulation Area 02/13/04

33 researchers, 8 Institutions Southern California Earthquake Center San Diego Supercomputer Center Information Sciences Institute Institute of Geophysics and Planetary Physics (UC) University of Southern California San Diego State University University of California, Santa Barbara Carnegie-Mellon University EXonMobil 02/13/04

TeraShake Peak Ground Velocity Maps NW to SE rupture SE to NW rupture 02/13/04

SCEC/CME Grid Infrastructure SCEC/CME has established grid-based connectivity, job-scheduling, and user and host authentication between SCEC, USC, ISI, SDSC, and PSC. SCEC Grid Testbed SCEC USC SDSC epi.usc.edu SUN E3800 8CPUs - 8GB RAM gravity.usc.edu Linux 4 CPUs - 4GB RAM condor.usc.edu Condor pool A collection of 320 SUN workstations hpc.usc.edu IBM Linux cluster 640 CPUs 320GB RAM almaak.usc.edu - SUN Sunfire 15K 64 CPUs 256GB RAM horizon.sdsc.edu IBM Blue Horizon 1152 CPUs 576GB RAM ISI PSC pinto.isi.edu Linux 2CPUs,500MHz 380MB RAM giis.scec.org/ scec-giis.isi.edu Linux 2CPUs, 1GHz 1GB RAM sidecar.psc.edu Linux 1CPU - 1GB RAM Current SCEC Grid configuration 02/13/04

SCEC Community Library Data grid architecture using SDSC Storage Resource Broker Supports user customizable portals Maintains associations between data and metadata Current collections contain 1.6 million files (10 terabytes) 3D ground motion for LA Basin (36 scenarios) Rupture Dynamics 4D Wavefield http://www.sdsc.edu/SCEC 02/13/04

SCEC UseIT Undergraduate Intern Program 02/13/04

LA3D GeoWall Software 02/13/04

Group Interaction and Collaboration Tools Applying communication tools to help with this distributed collaboration. Web Sites Software Configuration Management Tools Document Management Tools Bug Tracking Tools Data File and Metadata Tools Email Lists 02/13/04

Hosting of SCEC Community Models Provide access to SCEC Community Models, possibly alternative models, and utilities for working with the models. Community Fault Model (CFM-A) Community Velocity Model (CVM.3.0) Community Crustal Motion Map (CMM.3.0.1) 02/13/04

Validation Exercises for Simulation Codes Comparisons for 09/03/02 Yorba Linda Earthquake Data in black, SCEC CVM (FD) in blue, Harvard model (SEM) in red Comparison of Dynamic Rupture Models Rupture Test Case Contours 02/13/04

Hosting of SCEC Community Codes Provide access to SCEC geophysical codes Pathway 2: Olsen AWM CMU Hercules AWM Pathway 1: OpenSHA Pathway 4: Synthetic Seismograms Fréchet Kernels 02/13/04

Supporting and Running Large Scale Simulations 8 Processors (in 2002) 240 Processors (in 2004) 02/13/04

Establishment of SCEC Grid Infrastructure SCEC/CME has established grid-based connectivity, job-scheduling, and user and host authentication between SCEC, USC, ISI, SDSC, PSC, and TeraGrid sites. SCEC Grid Testbed SCEC USC SDSC epi.usc.edu SUN E3800 8CPUs - 8GB RAM gravity.usc.edu Linux 4 CPUs - 4GB RAM condor.usc.edu Condor pool A collection of 320 SUN workstations hpc.usc.edu IBM Linux cluster 640 CPUs 320GB RAM almaak.usc.edu - SUN Sunfire 15K 64 CPUs 256GB RAM horizon.sdsc.edu IBM DataStar 1152 CPUs 576GB RAM ISI PSC TeraGrid pinto.isi.edu Linux 2CPUs,500MHz 380MB RAM giis.scec.org/ scec-giis.isi.edu Linux 2CPUs, 1GHz 1GB RAM sidecar.psc.edu Linux 1CPU - 1GB RAM horizon.sdsc.edu IBM Blue Horizon 1152 CPUs 576GB RAM 02/13/04

SCEC Digital Library - Providing Data Management Capabilities Storage Resource Broker based Digital Library Collection now includes SCEC/PEER Scenario Ground Motion data collection, USC Green Tensors data collection (40TB+ Storage), TeraShake Simulations (40 TB+), and Puente Hills Simulation. SCEC Community Library Select Receiver (Lat/Lon) Output Time History Seismograms Select Scenario Fault Model Source Model 02/13/04

SDSC Data Visualization 02/13/04

ISI Data Visualization 02/13/04

Example SCEC and SCEC/CME IT-oriented Activities UseIT Intern Program Unified Structural Representation Ground Motion Prediction Communication, Education, Outreach OpenSHA Seismic Hazard Analysis Earthquake Forecasting 02/13/04

Example SCEC and SCEC/CME IT-oriented Activities CyberShake Waveform-based Seismic Hazard Analysis Unified Structural Representation Earthquake Rupture Dynamics Ground Motion Prediction Communication, Education, Outreach TeraShake 2 Dynamic Rupture-based Simulations 02/13/04

CyberShake Project Using 3D Synthetic Seismic Waveforms In Seismic Hazard Analysis 02/13/04

Various IMR types (subclasses) Attenuation Relationships Gaussian dist. is assumed; mean and std. from various parameters IMT, IML(s) Multi-Site IMRs compute joint prob. of exceeding IML(s) at multiple sites (e.g., Wesson & Perkins, 2002) Site(s) Rupture Intensity-Measure Relationship List of Supported IMTs List of Site-Related Ind. Params Vector IMRs compute joint prob. of exceeding multiple IMTs (Bazzurro & Cornell, 2002) Simulation IMRs exceed. prob. computed using a suite of synthetic seismograms

Ruptures in ERF within 200KM of USC 02/13/04

CyberShake Computational Elements Large (TeraShake Scale) forward calculations for each site. Requires calculation of 100,000+ seismogram for each site. SCEC/CME Grid-based scientific workflow system required to work at this scale. Access to distributed computing resources Large scale file management High performance and high throughput computing. TeraGrid allocation awarded for effort 145K SU (TG-BCS050001N) 02/13/04

End 02/13/04