1. The SCEC Broadband Platform Recent Activities and Developments Philip Maechling, Fabio Silva, Scott Callaghan, Thomas H. Jordan Southern California Earthquake Center August 29, 2014

2. Overview of Broadband Platform The Broadband Platform (BBP) is an open-source software distribution that: Calculates earthquake ground motions at frequencies above 10Hz across regional distances. Contains software tools for comparing simulation results against GMPEs and observed ground motion recordings. Can run large numbers of ground motion simulations and manage the simulation results. Available for download at: http://scec.usc.edu/scecpedia/Broadband_Platform. 2

3. Typical Broadband Platform Computational Stages Including Validation Stages 3

4. Common BBP Simulation Inputs Simple Source Description Station List Velocity Model 1D Profile 4 MAGNITUDE = 6.67 FAULT_LENGTH = 20.01 DLEN = 0.2 FAULT_WIDTH = 25.01 DWID = 0.2 DEPTH_TO_TOP = 5.0 STRIKE = 122 RAKE = 90 DIP = 40 LAT_TOP_CENTER = 34.344 LON_TOP_CENTER = -118.515 HYPO_ALONG_STK = 6.0 HYPO_DOWN_DIP = 19.4 DT = 0.01 SEED = 3092096

5. BBP Simulation Methods Implement Computational Stages Differently 5 Green Simula ExSIM UCrmgSyn1D GenSlipJbSimHfsim GenSlipJbSimBBToolbox GP Method  GP  SDSU  UCSB  EXSIM  CSM Common post- processing methods Example: RotD50 Common Inputs

6. WAVEFORM COMPARISON: GP, SDSU, & ExSIM STATION ON HANGING-WALL SIDE: R X = 10.5 km, R JB = 6.5 km, R RUP = 9.2 km 6

7. Validation Data Products Seismogram Comparison 7 Goodness-of-fit

8. Recent SCEC BBP Activities 1.Add New Modules and Post-Processing 2.Create Open-source Software Distributions 3.Support Validation Exercises 4.Run Scenario Simulation Studies 5.Archive Simulation Datasets 6.Present BBP 7.Conclusions and Discussions 8

9. Broadband Platform Supports Multiple Ground Motion Simulation Methods BBP simulations can use one or more of the currently integrated ground motion simulation methods –Graves & Pitarka – Rob Graves, Arben Pitraka –SDSU – Kim Olsen, Rumi Takedatsu –UCSB – Ralph Archuleta and Jorge Crempien –ExSIM (UWO) – Gail Atkinson, Karen Assatourians –Composite Source Model (UNR) – John Anderson 9

10. Add New Modules and Post-Processing Integrated new versions of 6 simulation methods in Fall 2013 Integrated multiple post-processing code (e.g. RotD50, Map Plots, Bias Plots) Updated 4 simulation methods in Spring 2014 (G&P, SDSU, ExSIM, UCSB) Added 4 of 5 NGA-W2 GMPE (Not Idriss) Added 3 Eastern GMPE’s 10

11. Recent SCEC BBP Activities 1.Add New Modules and Post-Processing 2.Create Open-source Software Distributions 3.Support Validation Exercises 4.Run Scenario Simulation Studies 5.Archive Simulation Datasets 6.Present BBP 7.Conclusions and Discussions 11

12. Created Open-Source Software Distributions Released BBP v14.3.0 in March 2014 –Version used in NGA-E activities Released BBP v13.6.1 in June 2013 –Version used in SWUS activities Public Software Releases include: –Best available version of each ground motion simulation models –Greens functions for multiple regions (LA, Mojave, NorCal, Japan etc) –Best available Post-processing software –Processing utilities to support cluster-based ensemble runs –Installation and basic users guides 12

13. Supported Broadband Platform Software Release 13

14. Recent SCEC BBP Activities 1.Add New Modules and Post-Processing 2.Create Open-source Software Distributions 3.Support Validation Exercises 4.Run Scenario Simulation Studies 5.Archive Simulation Datasets 6.Present BBP 7.Conclusions and Discussions 14

15. Comparison Data Products 15 Spectral response comparison Seismogram comparison Goodness-of-fit MAP based Goodness-of-fit

16. 16

17. Support Validation Exercises Supported Multiple SWUS SSHAC Meetings –Sept 2013, Oct 2013, Jan 2014, March 2014 –Ran validation studies using BBP for all available methods (5 methods) and delivered results for evaluation Supported NGA-E Meetings –April 2014 –Ran Validation Studies for all available methods (4 methods) and delivered results for evaluation Helped to develop evaluation tools –Formatted output to produce “SuperTable” to support evaluation and comparison of results 17

18. SWUS Evaluation of Results

19. Support Validation Exercises SWUS validation from May/June 2013: –Number of Scenarios: 140 –Number of Methods: 6 –Number of Seismograms Produced: 246,000 –Data Size: 4.8TB –Number of CPU Hours: 45,600 Hrs NGA-E validation runs from April 2014: –Number of Scenarios: 105 –Number of Methods: 4 –Number of Seismograms Produced: 180,000 –Data Size: 766 (GB) –Number of CPU Hours: 883 Hrs 19

20. Recent SCEC BBP Activities 1.Add New Modules and Post-Processing 2.Create Open-source Software Distributions 3.Support Validation Exercises 4.Run Scenario Simulation Studies 5.Archive Simulation Datasets 6.Present BBP 7.Conclusions and Discussions 20

21. Run Scenario Simulation Studies Ran Scenario Simulation Studies using released version of the BBP. (e.g. Use the Platform for Research) –NGA-E MW Scaling Scenario – August 2014 –Scott Dam – Lake Pillsbury Scenarios - June 2014 –Complex and Splay Fault Scenarios – March 2014 –SWUS Forward Scenarios - April 2014 –SWUS Forward Scenarios - August 2013 21

22. Run Scenario Simulation Studies NGA-E MW Scaling scenarios from August 2014: –Number of Scenarios: 336 –Number of Methods: 4 –Number of Seismograms Produced: 55,580 –Data Size: 526 GB –Number of CPU Hours: 6,567 Hrs Scott Dam – Pillsbury Lake Scenario June 2014: –Number of Scenarios: 6 –Number of Methods: 3 –Number of Seismograms Produced: 39,600 –Data Size: 295 GB –Number of CPU Hours: 20 Hrs 22

23. Run Scenario Simulation Studies SWUS GMC SSHAC scenarios from April 2014: –Number of Scenarios: 30 –Number of Methods: 3 –Number of Seismograms Produced: 174,720 –Data Size: 1.436 TB –Number of CPU Hours: 1935 Hrs SWUS scenarios from August/September 2013: –Number of Scenarios: 183 –Number of Methods: 3 –Number of Seismograms Produced: 860,000 –Data Size: 8 TB –Number of CPU Hours: 110,000 Hrs 23

24. Complex 1-a: Hosgri - Shoreline Notes Mw 7.0, 7.2, 7.4 Lengths – Shoreline = 25.5 km (fixed) – Hosgri (HS) = varies w/ Leonard Dip = 90 Rake = 180 (RL SS) Depth to top: 0 km 24

25. Recent SCEC BBP Activities 1.Add New Modules and Post-Processing 2.Create Open-source Software Distributions 3.Support Validation Exercises 4.Run Scenario Simulation Studies 5.Archive Simulation Datasets 6.Present BBP 7.Conclusions and Discussions 25

26. Archive Simulation Datasets SCEC currently has about 15TB of BBP Data archived at SCEC in multiple formats. Estimated storage lifetime: 18 months Interactive Web accessible: –http://bbpvault.usc.edu/bbp/BBP_March2014_Study/http://bbpvault.usc.edu/bbp/BBP_March2014_Study/ –http://bbpvault.usc.edu/bbp/BBP_June2013_Study/http://bbpvault.usc.edu/bbp/BBP_June2013_Study/ Web accessible zip files: –http://bbpvault.usc.edu/bbp/swus/swus-april-2014/http://bbpvault.usc.edu/bbp/swus/swus-april-2014/ –http://bbpvault.usc.edu/bbp/swus/swus-summer-2013/http://bbpvault.usc.edu/bbp/swus/swus-summer-2013/ Offline storage: –NGAE_MW_Scaling_-_August_2014_Simulations 26

27. Recent SCEC BBP Activities 1.Add New Modules and Post-Processing 2.Create Open-source Software Distributions 3.Support Validation Exercises 4.Run Scenario Simulation Studies 5.Archive Simulation Datasets 6.Present BBP 7.Conclusions and Discussions 27

28. Present BBP SCEC software group presents and publishes status and recent developments about the BBP at scientific and engineering conferences –SCEC Annual Meeting - September 2013 –Fall AGU - December 2013 –National Science Foundation PI Meeting - Jan 2014 –Spring SSA Meeting - May 2014 –SRL Paper - May 2014 –NCEE Meeting - August 2014 –SCEC Annual Meeting - September 2014 –Fall AGU - December 2014 28

29. Recent SCEC BBP Activities 1.Add New Modules and Post-Processing 2.Create Open-source Software Distributions 3.Support Validation Exercises 4.Run Scenario Simulation Studies 5.Archive Simulation Datasets 6.Present BBP 7.Conclusions and Discussions 29

30. Description of What is Needed to Support Engineering Validation Process Demonstrate that the simulations work –Comparison with ground motions from past earthquakes (part A) –Check of the method to simulate future earthquakes (Part B) Repeatable results (stable versions) –Move from researcher codes (frequent tinkering) to fixed versions Usable by someone other than author of method –Clear guidelines on how to implement the methods –IT Support for running methods Reviewable –Peer review by someone other than author of code needs to be possible Reasonable costs 30

31. Lessons Learned from Past Validations 1.Need clear documentation of fixed and optimized parameters from modelers for each region 2.Provide source information so it is consistent between methods 3.Provide a unique definition of crustal structure to be used by all groups (Vs, Q) 4.Consider multiple realizations 5.Run simulations for reference site conditions – correct data with empirical site factors 6.Provide uniform orientation of motions 7.Make all validation metrics and plots in uniform units/format 8.Streamline the process to allow fast feedback to modelers 31

32. SCEC’s Broadband Platform Software Sustainability Strategies 1.Keep the software development costs at a modest level (~2-3 FTE). 2.Integrate the best available scientific modules into the software. 3.Regularly release improved versions of Broadband as open-source software. 4.Release the software under a license (Apache 2) that is acceptable to commercial companies, and state and Federal agencies. 5.Continue the ongoing group verification and validation exercises to build user confidence in the software. 6.Implement software features and enhancements guided by a users needs and priorities 32

33. Possible Future BBP Developments 1.Continue to integrate ground motion modules (e.g. SMSIM, Irikura, Seok-Goo Rupture Generator) 2.Integrate GF generator modules 3.Support bent faults and multi-segment faults 4.Initial comparisons of results with 3D methods 5.Improve user interface to simplify running platform 6.Improve definition and documentation of file formats 7.Improve distribution techniques and download usage 8.Simplify and document configuration of simple problems 9.Longer-term data management 33

34. Thank you! The BBP Software is available for download at: http://scec.usc.edu/scecpedia/Broadband_Platform 34

35. New/Improved Modules Site amplification –Site effects in time series Improve simulation modules –UCSB, UNR, Irikura Recipe –M(A) scaling limitations Make the BBP modules work for Rupture Generators –Key for aleatory variability Empirical event terms variability Rupture source variability 35

36. SplayComplex Reminder on Terminology 36

37. SCEC Software Engineering Improvements to Research Codes 1.Source code Version Control : Subversion 2.Selected Coding Standards: Python PEP8 Style Guide 3.Use of Code Checker: pylint 4.Simplified Codebase: Removed unneeded features 5.Reduced required compilers: Now Gnu and Intel. 6.Documentation: Online Installation and User Guide 7.Open-Source License: Apache 2 8.Validation: Scientific Review of Results on Standard Problems by SWUS and NGA-E 9.Public Software Release: Open-source distributions provide transparency Sept 2013, March 2014 37