Southern California Earthquake Center CyberShake Progress Update 3 November 2014 through 4 May 2015 UGMS May 2015 Meeting Philip Maechling SCEC IT Architect.

Slides:

Advertisements

Similar presentations

A Pseudo-Dynamic Rupture Model Generator for Earthquakes on Geometrically Complex Faults Daniel Trugman, July 2013.

Advertisements

2/21/ USGS NSHMP CA Workshop II1 UCERF3.2: Hazard Implications Hazard comparison metrics Inversion testing –Convergence and eqn. set weighting.

10/09/2007CIG/SPICE/IRIS/USAF1 Broadband Ground Motion Simulations for a Mw 7.8 Southern San Andreas Earthquake: ShakeOut Robert W. Graves (URS Corporation)

Prague, March 18, 2005Antonio Emolo1 Seismic Hazard Assessment for a Characteristic Earthquake Scenario: Integrating Probabilistic and Deterministic Approaches.

Deterministic Seismic Hazard Analysis Earliest approach taken to seismic hazard analysis Originated in nuclear power industry applications Still used for.

1 High Performance Computing at SCEC Scott Callaghan Southern California Earthquake Center University of Southern California.

Ground Motion Prediction Equations for Eastern North America Gail M. Atkinson, UWO David M. Boore, USGS (BSSA, 2006)

Characterization of Ground Motion Hazard PEER Summative Meeting - June 13, 2007 Yousef Bozorgnia PEER Associate Director.

SCEC: An NSF + USGS Research Center ShakeAlert CISN Testing Center (CTC) Development Philip Maechling Information Technology Architect Southern California.

CyberShake Study 14.2 Technical Readiness Review.

Southern California Earthquake Center pynga: A python package for the Next Generation Attenuation relations Feng Wang Oct. 12 th,

Overview of Broadband Platform Software as used in SWUS Project Philip Maechling BBP Modelers Meeting 12 June 2013.

Seismic Hazard Assessment for the Kingdom of Saudi Arabia

FEASIBILITY STUDY OF A REGIONAL EEW SYSTEM FOR THE EASTERN CARIBBEAN REGION ZUCCOLO Elisa, SALAZAR Walter, DI SARNO Luigi, FARRELL Anthony, GIBBS Tony,

SCEC Information Technology Overview for 2012 Philip J. Maechling Information Technology Architect Southern California Earthquake Center SCEC Board of.

NSF Geoinformatics Project (Sept 2012 – August 2014) Geoinformatics: Community Computational Platforms for Developing Three-Dimensional Models of Earth.

IMPLEMENTATION OF SCEC RESEARCH IN EARTHQUAKE ENGINEERING ONGOING PROJECTS SCEC PROPOSAL TO NSF SCEC 2004 RFP.

SCEC/CME Project - How Earthquake Simulations Drive Middleware Requirements Philip Maechling SCEC IT Architect 24 June 2005.

1 SCEC Broadband Platform Development Using USC HPCC Philip Maechling 12 Nov 2012.

1.UCERF3 development (Field/Milner) 2.Broadband Platform development (Silva/Goulet/Somerville and others) 3.CVM development to support higher frequencies.

CyberShake Study 15.4 Technical Readiness Review.

CyberShake Study 2.3 Technical Readiness Review. Study re-versioning SCEC software uses year.month versioning Suggest renaming this study to 13.4.

The SCEC Broadband Platform Recent Activities and Developments Philip Maechling, Fabio Silva, Scott Callaghan, Thomas H. Jordan Southern California Earthquake.

Fig. 1. A wiring diagram for the SCEC computational pathways of earthquake system science (left) and large-scale calculations exemplifying each of the.

SCEC Workshop on Earthquake Ground Motion Simulation and Validation Development of an Integrated Ground Motion Simulation Validation Program.

Probabilistic Ground Motions for Scoggins Dam, Oregon Chris Wood Seismotectonics & Geophysics Group Technical Service Center July 2012.

CyberShake Study 15.3 Science Readiness Review. Study 15.3 Scientific Goals Calculate a 1 Hz map of Southern California Produce meaningful 2 second results.

Phase 1: Comparison of Results at 4Hz Phase 1 Goal: Compare 4Hz ground motion results from different codes to establish whether the codes produce equivalent.

HIGH FREQUENCY GROUND MOTION SCALING IN THE YUNNAN REGION W. Winston Chan, Multimax, Inc., Largo, MD W. Winston Chan, Multimax, Inc., Largo, MD Robert.

Experiences Running Seismic Hazard Workflows Scott Callaghan Southern California Earthquake Center University of Southern California SC13 Workflow BoF.

Southern California Earthquake Center SCEC Collaboratory for Interseismic Simulation and Modeling (CISM) Infrastructure Philip J. Maechling (SCEC) September.

UCERF3 Uniform California Earthquake Rupture Forecast (UCERF3) 14 Full-3D tomographic model CVM-S4.26 of S. California 2 CyberShake 14.2 seismic hazard.

Funded by the NSF OCI program grants OCI and OCI Mats Rynge, Gideon Juve, Karan Vahi, Gaurang Mehta, Ewa Deelman Information Sciences Institute,

California Earthquake Rupture Model Satisfying Accepted Scaling Laws (SCEC 2010, 1-129) David Jackson, Yan Kagan and Qi Wang Department of Earth and Space.

1 1.Used AWP-ODC-GPU to run 10Hz Wave propagation simulation with rough fault rupture in half-space with and without small scale heterogeneities. 2.Used.

Rapid Centroid Moment Tensor (CMT) Inversion in 3D Earth Structure Model for Earthquakes in Southern California 1 En-Jui Lee, 1 Po Chen, 2 Thomas H. Jordan,

Southern California Earthquake Center SI2-SSI: Community Software for Extreme-Scale Computing in Earthquake System Science (SEISM2) Wrap-up Session Thomas.

CyberShake and NGA MCER Results Scott Callaghan UGMS Meeting November 3, 2014.

Southern California Earthquake Center CyberShake Progress Update November 3, 2014 – 4 May 2015 UGMS May 2015 Meeting Philip Maechling SCEC IT Architect.

Welcome to the CME Project Meeting 2013 Philip J. Maechling Information Technology Architect Southern California Earthquake Center.

1 USC Information Sciences InstituteYolanda Gil AAAI-08 Tutorial July 13, 2008 Part IV Workflow Mapping and Execution in Pegasus (Thanks.

INTRODUCTION TO XSEDE. INTRODUCTION  Extreme Science and Engineering Discovery Environment (XSEDE)  “most advanced, powerful, and robust collection.

SCEC CyberShake on TG & OSG: Options and Experiments Allan Espinosa*°, Daniel S. Katz*, Michael Wilde*, Ian Foster*°,

National Center for Supercomputing Applications University of Illinois at Urbana-Champaign Recent TeraGrid Visualization Support Projects at NCSA Dave.

Special Project Highlights ( )

CyberShake Study 2.3 Readiness Review

CyberShake Study 16.9 Science Readiness Review

High Performance Computing at SCEC

SCEC UGMS Committee Meeting

2020 NEHRP Provisions Issues Ground Motion

Seismic Hazard Analysis Using Distributed Workflows

Scott Callaghan Southern California Earthquake Center

The SCEC Broadband Platform: Computational Infrastructure For Transparent And Reproducible Ground Motion Simulation Philip J. Maechling [1], Fabio Silva.

CyberShake Study 16.9 Discussion

SCEC Community Modeling Environment (SCEC/CME)

High-F Project Southern California Earthquake Center

Philip J. Maechling (SCEC) September 13, 2015

2University of Southern California, 3San Diego Supercomputer Center

High-Performance Computing (HPC) IS Transforming Seismology

CyberShake Study 17.3 Science Readiness Review

SICHUAN EARTHQUAKE May 12, 2008

SCEC UGMS Committee Meeting No. 6

CyberShake Study 2.2: Science Review Scott Callaghan 1.

March 21-22, University of Washington, Seattle

rvGAHP – Push-Based Job Submission Using Reverse SSH Connections

CyberShake Study 14.2 Science Readiness Review

Southern California Earthquake Center

CyberShake Study 18.8 Technical Readiness Review

CyberShake Study 2.2: Computational Review Scott Callaghan 1.

CyberShake Study 18.8 Planning

Presentation transcript:

Southern California Earthquake Center CyberShake Progress Update 3 November 2014 through 4 May 2015 UGMS May 2015 Meeting Philip Maechling SCEC IT Architect 4 May 2015

Southern California Earthquake Center November 2014 UGMS Meeting Reviewed CyberShake Study 14.2 Results Using: UCERF2 No Background Seismicity 3D Velocity Model: CVM-S4.26 Min Vs: 500 m/s Velocity Meshing: 200m Fault Meshing: 1000m Rupture Generator: genslip v3.2 (Graves & Pitarka 2010) Maximum Frequency: 0.5Hz PSHA 3.0s, 5.0s, 10.0s curves RotD s, 5.0s, 10.0s curves

Southern California Earthquake Center CyberShake Study 14.2 Hazard Map

Southern California Earthquake Center November 2014 UGMS Meeting CyberShake 14.2 Results Reviewed PSA3.0, PSA5.0, PSA10.0 hazard curves and maps Results from 286 Sites RotD s, 5.0s, 10s hazard curves Results from 14 sites MCER Probabilistic, Deterministic, Combined, Overall Curves Results from 14 Sites CyberShake 14.2 Results Posted:

Southern California Earthquake Center November 2014 UGMS MCER Sites

Southern California Earthquake Center CyberShake Platform: Physics-Based PSHA

Southern California Earthquake Center CyberShake 14.2 Probabilistic Hazard Model Products CS 14.2 RotD s Probabilistic Hazard Curve (LADT) CS 14.2 PSA3.0 Hazard Model (286 sites) CS 14.2 RotD s Probabilistic, Deterministic, Combined, Overall MCER Curves (LADT) GMPE Comparison Maps Ratio (CS 14.2/ NGA-2) RotD s (286 Sites) CS 14.2 RotD s Probabilistic, Deterministic, Deterministic Lower Limit, Combined MCER Contour Maps (286 Sites)

Southern California Earthquake Center CyberShake 14.2 RotD100 3s Probabilistic Hazard Curve (LADT)

Southern California Earthquake Center CyberShake 14.2 Probabilistic MCER Curve (LADT)

Southern California Earthquake Center CyberShake 14.2 Deterministic MCER Curve (LADT)

Southern California Earthquake Center CyberShake 14.2 Combined MCER Result (LADT)

Southern California Earthquake Center CyberShake 14.2 Overall MCER Result (LADT)

Southern California Earthquake Center CyberShake 14.2 Probabilistic MCER 3s RotD100 (286 Sites)

Southern California Earthquake Center CyberShake 14.2 Deterministic MCER 3s RotD100 (286 Sites)

Southern California Earthquake Center CyberShake 14.2 Deterministic Lower Limit MCER 3s RotD100 (286 Sites)

Southern California Earthquake Center CyberShake 14.2 Combined MCER Results 3s RotD100 (286 Sites)

Southern California Earthquake Center GMPE (NGA-2) Combined MCER Results 3s RotD100 (286 Sites)

Southern California Earthquake Center GMPE Comparison Map Ratio (CyberShake 14.2 / NGA-W2) MCER Results 3s RotD100 (286 Sites)

Southern California Earthquake Center New CyberShake Computational Results May RotD100 3S, 5S, 10S amplitudes calculated for all 286 CyberShake 14.2 sites 2.MCER RotD100 3S, 5S, 10S Contour Maps using 286 CyberShake 14.2 sites 3.Comparison Maps CyberShake MCER 14.2 versus GMPE MCER NGA-2 4.Started CyberShake 15.4 Study to calculate 1Hz CS results for 336 sites in Los Angeles Region 5.Preliminary CyberShake 15.4 results for 14 sites

Southern California Earthquake Center November 2014 UGMS Meeting CyberShake Study 14.2 UCERF2 No Background Seismicity 3D Velocity Model: CVM-S4.26 Min Vs: 500 m/s Velocity Meshing: 200m Fault Meshing: 1000m Rupture Generator: genslip v3.2 (Graves & Pitarka 2010) Maximum Frequency: 0.5Hz PSHA 3.0s, 5.0s, 10.0s curves RotD s, 5.0s, 10.0s curves

Southern California Earthquake Center May 2015 UGMS Meeting CyberShake Study 15.4 UCERF2 No Background Seismicity 3D Velocity Model: CVM-S4.26 Min Vs: 500 m/s Velocity Meshing: 100m Fault Meshing: 200m Rupture Generator: genslip v3.3.1 (Graves & Pitarka 2014) Maximum Frequency: 1.0Hz PSHA 2.0s, 3.0s, 5.0s, 10.0s curves RotD s, 3.0s, 5.0s, 10.0s curves

Southern California Earthquake Center CyberShake Study 15.4 sites (336) 336 sites (10 km mesh + points of interest + “gap” sites) Green sites are the 50 new “gap” sites (Run 14 UGMS sites first )

Southern California Earthquake Center Study 15.4 Parameters 1.0 Hz deterministic –100 m spacing –dt=0.005 sec –nt=40000 timesteps CVM-S 4.26 –Vs min = 500 m/s UCERF 2 Graves & Pitarka (2014) rupture variations –200 m rupture grid point spacing Source filtered at 2.0 Hz

Southern California Earthquake Center Expected Study 15.4 Data Products CVM-S4.26 Los Angeles-area hazard maps RotD100 2, 3, 4, 5, 7.5, 10 sec RotD50 2, 3, 4, 5, 7.5, 10 sec Geometric mean 2, 3, 5, 10 sec Hazard curves for 286 sites, at 2s, 3s, 5s, 10s 336 sets of 2-component SGTs Seismograms for all ruptures (~160M) Peak amplitudes in DB for 2s, 3s, 5s, 10s RotD100, RotD50 and geometric mean SA

Southern California Earthquake Center New Computational Results For May 2015 Initiate CyberShake 15.4 Study using NSF and DOE Computers Los Angeles region Hazard Model based on 336 sites at 1Hz Estimated 40M Computer Hours Estimated 500TB+ temporary data at DOE,NSF Computers Estimated 11TB persistent data at SCEC

Southern California Earthquake Center Estimated CyberShake 15.4 Computational Duration Estimated computational run-time is 12 weeks (11 running + 1 downtime) –Using DOE Titan (Oak Ridge Leadership Computing Facility) –Using NSF Blue Waters (NCSA) Current Status: Preliminary results for 40 of 336 Sites Completed

Southern California Earthquake Center May 2015 UGMS Meeting Discussion Topics Review of new CyberShake 14.2 Computational Data Products (0.5Hz Results) –RotD100 MCER results for 286 sites –MCER Contour Maps using 286 sites –CyberShake / GMPE Comparison Maps using 286 Sites Preliminary Analysis of CyberShake 15.4 results for 14 Sites (1.0Hz Results) –RotD100 MCER results for 14 sites

Southern California Earthquake Center End

Southern California Earthquake Center Rupture Generator changes Previous number of realizations related to fault length # of realizations = max(10, C * Area/10.0) –C = 0.5 Each realization is unique slip + hypocenter location Supports either random or uniform hypocenter distribution

Southern California Earthquake Center Rupture Generator v3.3.1 Use of new G&P rupture generator (v3.3.1) brought 1000m and 200m curves into agreement TEST site, black = 200 m, magenta = 1000 m 0.5Hz UCERF2 3 sec SA CVM-S4

Southern California Earthquake Center Random vs Uniform Hypocenters Variation counts –G&P 2010: 423k –Uniform: 485k –Random: 542k Uniform easier to interpolate 0.5Hz UCERF2 3 sec SA CVM- S4.26 WNGC site: black=random, magenta=uniform

Southern California Earthquake Center Storage Requirements Titan –Purged: 526 TB (for SGTs and temp data) Blue Waters –Delayed purge: 506 TB (for Titan SGTs) –Purged: 526 TB SGTs + 9 TB data products SCEC –Archived: 9.1 TB (seismograms, PSA, RotD) –Database: 268 GB 4 periods, 6) –Temporary: 608 GB (workflow logs) –Shared SCEC disks have 171 TB free

Southern California Earthquake Center Computational Requirements Per site: ~3720 node-hrs –SGTs: depends on execution site (~50%) Titan = 2110 node-hrs / 63,300 SUs Blue Waters = 1760 node-hrs / 30,200 SUs More expensive for Titan because of padding in pilot jobs and different node-hrs -> SU conversion –PP: 1880 node-hrs / 60,200 SUs (~50%) Computational time: –Titan (SGTs): 355K node-hours / 10.7M SUs –Blue Waters: 928K node-hours SGTs: 275K GPU node-hrs, 21K CPU node-hrs PP: 632K CPU node-hrs Titan has 104M SUs remaining Blue Waters has 5.3M node-hrs remaining

Southern California Earthquake Center End