Download presentation
Presentation is loading. Please wait.
Published byAudrey Long Modified over 9 years ago
1
1 How Are We Doing? A Verification Briefing for the SAWS III Workshop April 23, 2010 Chuck Kluepfel National Weather Service Headquarters Silver Spring, Maryland 301-713-0090 x132 Charles.Kluepfel@noaa.gov Prepared October 2008
2
2 Part 1 Traditional Statistics
3
3 The Basics: POD and FAR You can drive up your POD (also called hit rate) by over-forecasting IFR and below conditions. This practice simultaneously drives up the FAR. The CSI provides a mathematical way of correcting an inflated POD by using the FAR. The 2-category Heidke Skill Score has a similar affect, and it passes tests for equitability (statistical balance). Heidke also considers the “not forecast / not observed” situations (in an appropriately balanced manner), which are ignored by the CSI.
4
4 Prevailing vs. Operational Impact Forecast (OIF) Which should we use? OIF considers TEMPO groups. GPRA system uses OIF. MOS / LAMP – Do not produce TEMPOs When comparing to guidance, I used prevailing.
5
5 Modified SW US, March 2009 to Feb 2010 Flight Category: IFR and Below
6
6 Traditional Stats for Modified Southwest United States: Colorado New Mexico Utah Arizona Nevada California Minus these WFOs: San Diego, Los Angeles, San Francisco
7
7
8
8 Modified SW US, March 2009 to Feb 2010 Flight Category: IFR and Below
9
9
10
10 Modified SW United States TAF Performance vs. Projection March 2009 to February 2010
11
11 Modified SW US - March 2009 to Feb 2010 Scheduled 3-6 hr IFR and Below GFS LAMP (77K - 36K) K ÷ (77K – 63K) ~ 2.9 The 3-6 hr GFS LAMP false alarmed almost 3 times for every additional hit it got over the forecasters! POD 0.55 FAR 0.50 CSI 0.35 Forecast YesNo ObsObs Yes77 K64 K No77 K 3.4 Million Prevailing POD 0.45 FAR 0.36 CSI 0.36 Forecast YesNo ObsObs Yes63 K77 K No36 K 3.4 Million
12
12 Modified SW US - March 2009 to Feb 2010 Scheduled 3-6 hr IFR and Below NAM MOS (117K – 36K) ÷ (75K - 63 K) ~ 6.2 3-6 hr NAM MOS false alarmed over 6 times for every additional hit it got over the forecasters! POD 0.54 FAR 0.60 CSI 0.30 Forecast YesNo ObsObs Yes76 K64 K No117 K 3.3 Million Prevailing POD 0.45 FAR 0.36 CSI 0.36 Forecast YesNo ObsObs Yes63 K77 K No36 K 3.4 Million
13
13 WFOs San Diego, Los Angeles, San Francisco TAF Performance vs. Projection IFR and Below March 2009 to February 2010
14
14 WFOs El Paso, Tucson, Phoenix (Low Desert Southwest) IFR and Below March 2009 to February 2010
15
15 Part 2 Lead-Time Software
16
16
17
17 Part 3 The Future
18
18 The Future Output to CSV files (just posted) oStarting with Flight Category and Sig Wx Data oCeiling / Visibility (next) oWinds (last) Plots of POD / FAR / CSI Sort Elements by Sig Wx Type
19
OBSERVED (A) FORECAST (F) (X) (Y) (W) (Z) 0 1 0 1 POD FOM POFD PON FOHFAR DFR FOCN (Ma,Mf) A(F): Regression of Observations upon the forecast F(A): Regression of Forecast upon observations Ma: Average of Observations (x+y)/N Mf: Average of Forecasts (x+z)/N Improving the Current System: Geometric Interpretation
20
20 Observed ForecastYesNoTotal Yes606 No066 Total6612 Observed ForecastYesNoTotal Yes066 No606 Total6612 Basic Interpretation: Extreme Cases PERFECT FORECAST RESIGN FROM THE NWS
21
21 Observed ForecastYesNoTotal Yes336 No336 Total6612 RANDOM CHANCE Basic Interpretation: Random Chance
22
22 Observed ForecastYesNoTotal Yes500 No50100150 Total10010200 Observed ForecastYesNoTotal Yes50 100 No0100 Total50150200 Under-forecast Over-forecast Assesses Bias in one glance!! Basic Interpretation: False Alarms vs. Misses
23
23 Finis
24
24 Traditional Stats Entire National Weather Service
25
25 Nation, March 2009 to Feb 2010 Flight Category: IFR and Below
26
26 Nation, March 2009 to Feb 2010 Flight Category: IFR and Below
27
27 Nation, March 2009 to Feb 2010 Flight Category: IFR and Below
28
28 Nation Performance vs. Projection March 2009 to February 2010
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.