1. CISN Testing Center (CTC)
Development Update
Philip Maechling May 2011
Maria Liukis
Thomas H. Jordan

2. CTC Development Update
Implemented CTC processing software that matches CISN EEW Reports with ANSS and ShakeMap observational data
- Implemented Event Forecast (e.g. Forecast Final Magnitude) and Site Forecast summaries (e.g Forecast PGV at CISN Station Sites for event) using CIT OnSite and ETH Virtual Seismologists EEW reports

3. CTC Development Plans
Verify results for selected evaluation summaries.
Release a version of the EEW Report evaluation codes
Reprocess 2011 EEW reports, then 2010 EEW reports using a the released version of software.
Implement reader for ShakeAlert forecast information.

4. CTC Software Description
Software designed to evaluate (post-process) existing EEW reports including CIT OnSite, ETH Virtual Seismologists. Post-processing of UCB ElarmS Reports will be supported if available.
Software capable of evaluating CISN ShakeAlert EEW Reports. Need definition of EEW report information and format.
Software designed to be used for both (a) interactive post-processing, and (b) by the CTC testing web site.

5. CISN Testing Center (CTC) Forecast Evaluation Processing System
ShakeMap
RSS Feed
Ground
Motion Observations
Retrieve
Filter
Data
Catalog
Observed ANSS EQ Parameter and Ground Motion Data
Filtered
Earthquake
Earthquake
Catalog
Catalog
ANSS Earthquake Catalog
Evaluation tests
comparing
Forecasts and
Observations
ShakeAlert
Earthquake
Parameter
Forecast
CISN Decision Modules
Load
Reports
ShakeAlert
Ground
Motion
Forecast
ShakeAlert Forecast Information
CISN User Modules
CISN EEW Testing Center (CTC) and Web Site

6. CTC EEW Report Processing
Event
Forecast Summary
CIT OnSite Reports
ETH VS Reports
Parse EEW Reports into Forecasts
Generate
Forecast
Evaluation
Summaries
Forecast with Station Locations
Forecast with
ANSS Event Info
Forecast with ShakeMap Site Observ.
Site Forecast Summary
UCB Elarms Reports
CISN ShakeAlert
Reports
Forecast
Catalog Summary
CISN Station Locations
ANSS Catalog
ShakeMap Event Reports

7. Example CISN EEW Reports
Triggers_ xml
Triggers_ xml

8. Example CISN EEW Reports

9. Example CISN EEW Reports

10. Proposed Performance Measures
Summary Reports for each M ≥ M-min: CISN algorithm testing document specifies six types of tests.
Summary 1: Magnitude
Summary 2: Location
Summary 3: Ground Motion
Summary 4: System Performance
Summary 5: False Triggers
Summary 6: Missed Triggers

11. Developing capability to process current catalog and EEW reports
ANSS Event
4/13/ :16
4.2
REN
E+11
4/13/ :10
4.3
4/9/2011 8:58
4.13 CI
3/31/ :22
4.35
3/6/ :46
4.54 NC
3/1/2011 2:19
4.46
2/23/2011 4:50
4.21
2/23/2011 4:49
4.29
2/20/ :34
4.5
2/18/ :47
5.07
2/13/ :25
4.08
1/23/2011 6:47
4.15
1/22/ :21 NN
325134
1/22/ :29
4.32
325135
1/22/ :26
4.01
325137
1/12/2011 8:51
1/11/2011 1:36
4.06
324385

12. Event Forecast Summary
Event ID:
Magnitude: 5.07
Origin Time: 2011/02/18 17:47:35.77
Hypocenter: km
Forecast Source: smi://caltech.gps.edu/onsite/cinnabar/vs/eewvs
Forecast Time: T17:48:12Z
First Forecast Secs after Origin: 37
Net:Sta Dist(km) Wd(secs) PGV(obs) PGV(fcst)m/s I(obs) I(fcst)MMI abs(Io-If) Skill obs GM fcst GM
CI:WES Very Good Strong shaking/light damage potential Strong shaking/light damage potential
CI:BTC Moderate Light shaking/no damage potential Moderate shaking/very light damage
CI:SWS Moderate Moderate shaking/very light damage Light shaking/no damage potential
CI:RXH Good Light shaking/no damage potential Weak shaking/no damage potential
CI:ERR Very Good Light shaking/no damage potential Light shaking/no damage potential
AZ:MONP Very Good Light shaking/no damage potential Light shaking/no damage potential
CI:SAL Good Light shaking/no damage potential Light shaking/no damage potential
CI:BC Very Good Weak shaking/no damage potential Weak shaking/no damage potential
CI:JEM Good Light shaking/no damage potential Light shaking/no damage potential
CI:BOR Good Weak shaking/no damage potential Weak shaking/no damage potential
CI:OLP Moderate Weak shaking/no damage potential Light shaking/no damage potential
CI:NSS Very Good Weak shaking/no damage potential Weak shaking/no damage potential
CI:EML Good Weak shaking/no damage potential Light shaking/no damage potential
CI:SDR Good Weak shaking/no damage potential Light shaking/no damage potential
CI:BLY Moderate Not felt/no damage potential Light shaking/no damage potential
CI:CTC Very Good Weak shaking/no damage potential Weak shaking/no damage potential …
Summary of 51 site-specific forecast ratings for event:
Very Good: 41.2%
Good : 29.4%
Moderate : 23.5%
Poor : 3.9%
Very Poor: 2.0%

13. Experiment Design Summary 1.1: Magnitude X-Y Diagram
Measure of Goodness: Data points fall on diagonal line
Relevant: T2,T3,T4
Drawbacks: Timeliness element not represented
Which in series of magnitude estimates should be used in plot.

14. Example CISN EEW Reports

15. Example CISN EEW Reports
Triggers_ xml
Triggers_ xml

16. Experiment Design Summary 3.2: Intensity X-Y Diagram
Measure of Goodness: Data points fall on diagonal line
Relevant: T1,T2,T4
Drawbacks: Timeliness element not represented
Which in series of intensity estimate should be used in plots T3.

18. Experiment Design Summary 3.1 : Intensity Map Comparisons
Measure of Goodness: Forecast map matches observed map
Relevant: T4
Drawbacks: Not a quantitative results

19. Summary 3.1 : Intensity Map Comparisons Forecast – Observations, and Goodness of Fit between two.

20. End