SCEC Information Technology Overview for 2012 Philip J. Maechling Information Technology Architect Southern California Earthquake Center SCEC Board of Directors Meeting 30 Jan 2012

SCEC3 Information Technology (IT) Work Areas 1. Administrative IT including membership, awards, contracts o John McRaney, Tran Huynh, John Yu, Karen Young and others 2.Collaboration IT supports collaborative research projects, communications, meetings, creation and distribution of research results. o John Marquis, Tran Huynh, John Yu, John McRaney and others 3.Outreach IT prepares, presents, and preserves research results to collaborative partners, funding agencies, and public o Thomas Jordan, Mark Benthien, Philip Maechling, Tran Huynh, and others 4.Computational Science IT provides access to computer hardware, scientific software, and data management. o Philip Maechling, Yifeng Cui, Ewa Deelman, Scott Callaghan, Maria Liukis, Patrick Small, Kevin Milner, John Yu and others

SCEC Scientific Software Distributions 1.CSEP Software Framework – Last release Jan CVM-S Community Velocity Model – Last release Feb CVM-H Community Velocity Model – Last release Oct UCVM Software Framework – Last release Jan Broadband Platform – Last release Oct SCEC-VDO visualization of 3D geo-referenced data – Available on request

SCEC Scientific Software In Active Use 1.AWP-ODC – Finite difference dynamic ruptures and anelastic wave propagation – Simulations Jan Hercules – Finite element anelastic wave propagation – Simulations Jan R. Graves Finite Difference Wave Propagation – Simulations in Jan OpenSHA probabilistic seismic hazard analysis – Calculations in Jan Tera3D full 3d tomography processing system – California regional inversion iteration in Jan CyberShake physics-based PSHA hazard curve calculation – Research hazard curve in Jan CSEP Testing Framework – Daily use in CSEP, EEW, and TD Testing 8.Broadband Platform – Use on SCEC servers by PEER and NGA-E modelers in Jan SCEC-VDO – Used by UCERF Scientific Reviewers in Jan 2012

Unified Community Velocity Model Development UCVM development included (a) CVM evaluation tools, (b) CVM integration framework, and (c) combining best of existing models into consistent state-wide CVM for use with state-wide 1Hz 3D wave propagation simulations. Standardizes query API, elevation DEM, bathymetry, Moho, GTL, and distribution format

Inversion-based Update to CVM-S4 Performed two seismic inversion iterations using different data sets 1.CVM4SI1: improved CVM4 using earthquake data. 2.CVM4SI2: improved CVM4SI1 using ambient-noise Green’s function data. Perturbation obtained in 2nd iteration enhances the perturbation obtained in the 1st iteration. Waveform improvements for both earthquake recordings and ambient noise Green’s functions

SCEC CyberShake system produced hazard curves with alternative (improved) rupture generator and hazard curves based on CVM-H Used SCEC allocations on NSF supercomputers to calculate CyberShake PSHA hazard curves for sites of interest (San Onofre and Diablo Canyon)

SCEC Broadband Platform Released SCEC released Broadband Platform which can calculate 10Hz seismograms using rupture generators, 1D wave propagations codes, and stochastic high frequency software from multiple SCEC groups.

Broadband CyberShake Added R. Graves high-frequency stochastic modules from the BBP into CyberShake Seismogram calculation –Stochastic high-frequency –Non-linear site response –Filtered and combined with low-frequency (<0.5 Hz) –Validate with SCSN, PBR sites Perris (PBR) 1 sec0.1 sec

Current Broadband CyberShake Run 92 hazard curves –CVM-H 11.2 and CVM-S4, Graves and Pitarka (2010) –PBR, SCSN sites –64 SGTs, 92 post-processing –1.5 million SUs Data requirements (broadband, 2- component) –SGTs: 2.5 TB –Seismograms: 3.9 TB PBR=red, SCSN=orange, near SCSN=yellow

SCEC Computational Science Software Projects

The Quake Group at Carnegie Mellon Current Efforts and Future Plans Jacobo Bielak, Ricardo Taborda, Haydar Karaoglu, Yigit Isbiliroglu The Effect of Minimum S-wave Velocity Vs min 200 m/sVs min 500 m/s 2011 SCEC CME Meeting Anelastic Wave Propagation (AWP) Various applications Hercules Quadratic Elements No Intrinsic Attenuation f max 2 Hz Vs min 200 m/s Domain85 x 85 x 42.5 km 3 Elements226,670,602 Cores18,000 Walltime4 hrs SUs72,000 Short-Period Chino Hills Simulation f max 4 Hz Vs min 200 m/s Domain180 x 135 x 62 km 3 Elements5 billion Cores24,000 Walltime31 hrs SUs744, trillion grid points for a regular grid; 20 times the # of SUs Blue – simulation Red – data Source model: Chen Ji Source model: Graves & Pitarka

SCEC Computational Science Forecast Testing Centers 1.Earthquake forecasts (CSEP) – Active SCEC CSEP Operational Testing Center 2.Transient Detection (TD) – Active Operational TD Testing Center 3.EEW CISN Testing Center – Active Operational CTC Testing Center

CSEP Development and SCEC CSEP Testing Center

CISN EEW Testing Center

SCEC Transient Detection Testing Center

SCEC Computational Science Organizational Interactions 1.National Science Foundation (NSF) 2.United States Geological Survey (USGS) 3.W. M. Keck Foundation 4.Department of Homeland Security (DHS) 5.Pacific Gas and Electric (PG and E) 6.USC High Performance Computing and Communications (HPCC) 7.Argonne Leadership Computing Facility (ALCF) 8.Oak Ridge National Laboratory (ORNL) 9.National Institute for Computational Science (NICS) 10.Blue Waters NSF Track 1 Computing Facility (Univ. of Illinois) 11.Extreme Science and Engineering Discovery Environment (XSEDE) 12.San Diego Supercomputer Center (SDSC)

SCEC Computational Science Organizational Interactions 13.Texas Advanced Computing Center (TACC) 14.Pittsburgh Supercomputing Center (PSC) 15.Open Science Grid (OSG) 16.National Center for Atmospheric Research (NCAR) 17.NCAR-Wyoming Supercomputing Center (NWSC) 18.USC Information Sciences Institute (ISI) 19.Data Intensive Cyber Environments (DICE) Center at the University of North Carolina at Chapel Hill 20.Incorporated Research Institutions for Seismology (IRIS) Data Management Center 21.California Integrated Seismic Network (CISN) 22.Computational Infrastructure in Geodynamics (CIG) 23.Global Earthquake Model (GEM)

SCEC Computational Science In DOE Publication

SCEC CME researchers have access to three leadership class Supercomputers (NSF Blue Waters (2012), DOE Titan (2012), DOE Mira (2013)) to use for SCEC research calculations.

UCERF3.0 Extended Earthquake Rupture Forecast contains slip-time histories and rupture variations for all UCERF3.0 ruptures. UCVM Unified California Velocity Model UCERF 3.0 Extended Earthquake Rupture Forecast (June 2012) UCVM – Unified California Velocity Model, a statewide 3-D velocity model assembled from existing and improved California CVM’s (Small, SCEC USR)

CME CyberShake 3.0 calculation will build on UCERF3.0 in order to calculate a “physics-based” California PSHA Map CyberShake 3.0 California Wave Propagation-based PSHA Map up to 10Hz (Fall 2012) CyberShake 1.0 Los Angeles Region Wave Propagation- based PSHA Map up to 0.5Hz (Fall 2009) will increase in range and frequencies.

CyberShake 3.0 Computational Estimates CyberShake 3.0 Hazard Map Calculation Estimates: Number of Sites on Map: 4240 (10km spacing) Number of Jobs: 3.6 billion Est. CPU-hours: million Data products (seismograms, spectral acceleration): 76.5 TB Runtime on half-Jaguar: 3096 hrs (129 days) Runtime on half-Blue Waters: 770 hrs (32.1 days) Database entries: 6.95 billion

2011 Funded Projects (estimates per year) 1.CSEP Software Framework Development and Operations (200k) 2.OpenSHA and SCEC-VDO in support of UCERF 3.0 (100k) 3.NSF PetaSHA-3 (850k) 4.NSF PRAC Blue Waters (5k) 5.USGS CISN EEW Project (50k) 6.Broadband Platform from PG and E (150k) 7.Transient Detection Testing (60k)

2012 Funded Projects (estimates per year) 1.CSEP Software Framework Development and Operations (200k) 2.OpenSHA and SCEC-VDO in support of UCERF 3.0 (80k) 3.NSF SI2 SEISM Project (700k) 4.NSF PetaSHA-3 (425k) 5.NSF PRAC Blue Waters (5k) 6.USGS Supported CISN EEW Project (25k) 7.Broadband Platform from PG and E (150k)

NSF Software Infrastructure for Sustained Innovation Proposal (Submitted July 2011) SI2-SSI: A Sustainable Community Software Framework for Petascale Earthquake Modeling PI: T. H. Jordan (USC); Co-PIs: Jacobo Bielak (CMU), Y. Cui (SDSC), and K. Olsen (SDSU) Year 1: – Computational milestone: simulation of Table 1 earthquake records at 0-4 Hz (deterministic) and 1-10 Hz (stochastic) using the Broadband platform. – Validation effort: GMSV of these simulations against the earthquake records of Table 1 using linear and nonlinear SDoF oscillators. UCVM comparison of tomographic results for Southern California. – Software releases: UCVM platform, Broadband platform; Kraken release of SEISM-IO. Year 2: – Computational milestone: Time-independent CSHM-LA (UCEF2-based). – Validation efforts: GMSV for geotechnical systems; comparison of CSHM-LA with NGA hazard maps and PBR data set. Full-3D evaluation of statewide UCVM. – Software releases: UCVM platform, Broadband platform; Blue Waters release of SEISM-IO. Year 3: – Computational milestone: Time-dependent CSHM-CA (UCEF3-based). – Validation efforts: GMSV for nonlinear MDoF systems; comparison of CSHM-CA with NGA hazard maps and PBR data set. – Software releases: UCVM platform, Broadband platform; Mira release of SEISM-IO.

NSF Geoinformatics Proposal (Submitted Jan 13, 2012) Geoinformatics: Community Computational Platforms for Developing Three- Dimensional Models of Earth Structure PI: T. H. Jordan (USC); Co-PIs: P. Chen (U. Wyoming), Y. Cui (SDSC), and J. Tromp (Princeton) Objective: Establish an interoperable set of community computational platforms that will (a) implement the techniques of full-3D waveform tomography, and (b) facilitate the development and delivery of Earth models at a variety of scales. Two tomographic platforms will be built on highly scalable codes for solving the forward problem of anelastic wave propagation (AWP): AWP-ODC platform. AWP-ODC is 4 th -order, staggered-grid, finite-difference code developed by K. Olsen and his colleagues. SPECFEM3D platform. SPECFEM3D is spectral element code, developed by D. Komatitsch and J. Tromp. UCVM platform developed by SCEC enables users to build meshes from models with topography, hydrology, bathymetry, and standardized models of surficial (geotechnical) layers.

For More Information Please Visit: End