1. CyberShake Study 2.3 Readiness Review

2. Study 2.3 Overview Compare codes and velocity models
RWG V3.0.3 vs AWP-ODC-SGT
CVM-S 4 vs CVM-H 11.9
Different version of CVM-H than previous runs
Adds San Bernardino, Santa Maria basins
286 Southern California sites
0.5 Hz
Deterministic post-processing only

3. Proposed Study sites

4. SGT Computational Requirements
SGTs on Blue Waters
Computational time: 8.4 M SUs
RWG: 16k SUs/site x 286 sites = 4.6 M SUs
AWP: 13.5k Sus/site x 286 sites = 3.8 M SUs
22.35 M SU allocation, 22 M SUs remaining
Storage: TB
160 GB/site x 286 sites = 44.7 TB

5. PP computational requirements
Post-processing on Stampede
Computational time:
4000 SUs/site x 286 sites = 1.1 M SUs
4.1 M SU allocation, 3.9 M remaining
Storage: 44.7 TB input, 13 TB output
44.7 TB of SGT inputs; will need to rotate out
Seismograms: 46 GB/site x 286 sites = 12.8 TB
PSA files: 0.8 GB/site x 286 sites = 0.2 TB

6. Long-term storage 44.7 TB SGTs: 13 TB Seismograms, PSA data
To be archived to tape (NCSA? TACC? Somewhere else?)
13 TB Seismograms, PSA data
Have been using SCEC storage - scec-04?
5.5 TB workflow logs
Can compress after mining for stats
CyberShake database
1.4 B entries, 330 GB data (scaling issues?)

7. Verification work 4 sites (WNGC, USC, PAS, SBSM)
RWG V3.0.3, CVM-S
RWG V3.0.3, CVM-H
AWP, CVM-S
AWP, CVM-H
Plotted with previously calculated RWG V3
Expect RWG V3 slightly higher than the others

8. WNGC
CVM-S
CVM-H
RWG V Green
AWP - Purple
RWG V3 - Orange

9. USC
CVM-S
CVM-H
RWG V Green
AWP - Purple
RWG V3 - Orange

10. PAS
CVM-S
CVM-H
RWG V Green
AWP - Purple
RWG V3 - Orange

11. SBSM
CVM-S
CVM-H
RWG V Green
AWP - Purple
RWG V3 - Orange

12. SBSM Velocity Profile

13. Estimated Duration Limiting factors:
Blue Waters queue time
Uncertain how many sites in parallel
Blue Waters → Stampede transfer
100 MB/sec seems sustainable from tests, but could get much worse
50 sites/day; unlikely to reach
Estimated completion by end of June

14. Risks Stampede becomes busier
Post-processing still probably shorter than SGTs
CyberShake database unable to handle data
Would need to create other DBs, distributed DB, change technologies