CISN Testing Center (CTC) Development Update Philip Maechling 12 May 2011 Maria Liukis Thomas H. Jordan

CTC Development Update Implemented CTC processing software that matches CISN EEW Reports with ANSS and ShakeMap observational data - Implemented Event Forecast (e.g. Forecast Final Magnitude) and Site Forecast summaries (e.g Forecast PGV at CISN Station Sites for event) using CIT OnSite and ETH Virtual Seismologists EEW reports

CTC Development Plans Verify results for selected evaluation summaries. Release a version of the EEW Report evaluation codes Reprocess 2011 EEW reports, then 2010 EEW reports using a the released version of software. Implement reader for ShakeAlert forecast information.

CTC Software Description Software designed to evaluate (post-process) existing EEW reports including CIT OnSite, ETH Virtual Seismologists. Post-processing of UCB ElarmS Reports will be supported if available. Software capable of evaluating CISN ShakeAlert EEW Reports. Need definition of EEW report information and format. Software designed to be used for both (a) interactive post-processing, and (b) by the CTC testing web site.

CISN Testing Center (CTC) Forecast Evaluation Processing System ShakeMap RSS Feed Ground Motion Observations Retrieve Filter Data Catalog Observed ANSS EQ Parameter and Ground Motion Data Filtered Earthquake Earthquake Catalog Catalog ANSS Earthquake Catalog Evaluation tests comparing Forecasts and Observations ShakeAlert Earthquake Parameter Forecast CISN Decision Modules Load Reports ShakeAlert Ground Motion Forecast ShakeAlert Forecast Information CISN User Modules CISN EEW Testing Center (CTC) and Web Site

CTC EEW Report Processing Event Forecast Summary CIT OnSite Reports ETH VS Reports Parse EEW Reports into Forecasts Generate Forecast Evaluation Summaries Forecast with Station Locations Forecast with ANSS Event Info Forecast with ShakeMap Site Observ. Site Forecast Summary UCB Elarms Reports CISN ShakeAlert Reports Forecast Catalog Summary CISN Station Locations ANSS Catalog ShakeMap Event Reports

Example CISN EEW Reports Triggers_4.9.2011.xml Triggers_8.6.2010.xml eewvs-report-20110218174840@0008001.xml eewvs-report-20110331202346@0008253.xml

Example CISN EEW Reports

Example CISN EEW Reports

Proposed Performance Measures Summary Reports for each M ≥ M-min: CISN algorithm testing document specifies six types of tests. Summary 1: Magnitude Summary 2: Location Summary 3: Ground Motion Summary 4: System Performance Summary 5: False Triggers Summary 6: Missed Triggers

Developing capability to process current catalog and EEW reports ANSS Event 4/13/2011 22:16 4.2 REN 2.01104E+11 4/13/2011 22:10 4.3 4/9/2011 8:58 4.13 CI 10922333 3/31/2011 20:22 4.35 14960652 3/6/2011 13:46 4.54 NC 71538646 3/1/2011 2:19 4.46 71530230 2/23/2011 4:50 4.21 71048354 2/23/2011 4:49 4.29 71527390 2/20/2011 19:34 4.5 14937988 2/18/2011 17:47 5.07 14937372 2/13/2011 23:25 4.08 10892285 1/23/2011 6:47 4.15 14920876 1/22/2011 19:21 NN 325134 1/22/2011 17:29 4.32 325135 1/22/2011 17:26 4.01 325137 1/12/2011 8:51 71508850 1/11/2011 1:36 4.06 324385

Event Forecast Summary Event ID: 14937372 Magnitude: 5.07 Origin Time: 2011/02/18 17:47:35.77 Hypocenter: 32.0470 -115.0622 15.00 km Forecast Source: smi://caltech.gps.edu/onsite/cinnabar/vs/eewvs Forecast Time: 2011-02-18T17:48:12Z First Forecast Secs after Origin: 37 Net:Sta Dist(km) Wd(secs) PGV(obs) PGV(fcst)m/s I(obs) I(fcst)MMI abs(Io-If) Skill obs GM fcst GM CI:WES 101.0 37.0 0.0276 0.0011 4.31 4.44 0.13 Very Good Strong shaking/light damage potential Strong shaking/light damage potential CI:BTC 108.0 37.0 0.0152 0.0001 3.51 2.20 1.32 Moderate Light shaking/no damage potential Moderate shaking/very light damage CI:SWS 121.3 37.0 0.0038 0.0005 2.23 3.58 1.35 Moderate Moderate shaking/very light damage Light shaking/no damage potential CI:RXH 136.5 37.0 0.0051 0.0002 1.97 2.48 0.51 Good Light shaking/no damage potential Weak shaking/no damage potential CI:ERR 138.5 37.0 0.0076 0.0002 2.97 2.83 0.13 Very Good Light shaking/no damage potential Light shaking/no damage potential AZ:MONP2 158.5 37.0 0.0030 0.0002 2.59 2.57 0.03 Very Good Light shaking/no damage potential Light shaking/no damage potential CI:SAL 161.9 37.0 0.0094 0.0001 2.91 2.32 0.58 Good Light shaking/no damage potential Light shaking/no damage potential CI:BC3 182.1 37.0 0.0027 0.0001 1.86 1.46 0.39 Very Good Weak shaking/no damage potential Weak shaking/no damage potential CI:JEM 184.2 37.0 0.0045 0.0001 2.77 2.02 0.75 Good Light shaking/no damage potential Light shaking/no damage potential CI:BOR 185.7 37.0 0.0011 0.0001 1.49 1.99 0.50 Good Weak shaking/no damage potential Weak shaking/no damage potential CI:OLP 186.6 37.0 0.0030 0.0001 3.00 1.66 1.33 Moderate Weak shaking/no damage potential Light shaking/no damage potential CI:NSS2 186.7 37.0 0.0037 0.0001 1.86 1.74 0.12 Very Good Weak shaking/no damage potential Weak shaking/no damage potential CI:EML 192.0 37.0 0.0026 0.0001 2.51 1.77 0.74 Good Weak shaking/no damage potential Light shaking/no damage potential CI:SDR 192.7 37.0 0.0024 0.0001 2.50 1.66 0.84 Good Weak shaking/no damage potential Light shaking/no damage potential CI:BLY 195.5 37.0 0.0028 0.0000 2.15 0.78 1.38 Moderate Not felt/no damage potential Light shaking/no damage potential CI:CTC 198.4 37.0 0.0021 0.0001 1.62 1.56 0.06 Very Good Weak shaking/no damage potential Weak shaking/no damage potential … Summary of 51 site-specific forecast ratings for event:14937372 Very Good: 41.2% Good : 29.4% Moderate : 23.5% Poor : 3.9% Very Poor: 2.0%

Experiment Design Summary 1.1: Magnitude X-Y Diagram Measure of Goodness: Data points fall on diagonal line Relevant: T2,T3,T4 Drawbacks: Timeliness element not represented Which in series of magnitude estimates should be used in plot.

Example CISN EEW Reports

Example CISN EEW Reports Triggers_4.9.2011.xml Triggers_8.6.2010.xml eewvs-report-20110218174840@0008001.xml eewvs-report-20110331202346@0008253.xml

Experiment Design Summary 3.2: Intensity X-Y Diagram Measure of Goodness: Data points fall on diagonal line Relevant: T1,T2,T4 Drawbacks: Timeliness element not represented Which in series of intensity estimate should be used in plots T3.

Experiment Design Summary 3.1 : Intensity Map Comparisons Measure of Goodness: Forecast map matches observed map Relevant: T4 Drawbacks: Not a quantitative results

Summary 3.1 : Intensity Map Comparisons Forecast – Observations, and Goodness of Fit between two.

End