Presentation is loading. Please wait.

Presentation is loading. Please wait.

Improving Probabilistic Ensemble Forecasts of Convection through the Application of QPF-POP Relationships Christopher J. Schaffer 1 William A. Gallus Jr.

Published by Modified over 9 years ago

Similar presentations

Presentation on theme: "Improving Probabilistic Ensemble Forecasts of Convection through the Application of QPF-POP Relationships Christopher J. Schaffer 1 William A. Gallus Jr."— Presentation transcript:

1 Improving Probabilistic Ensemble Forecasts of Convection through the Application of QPF-POP Relationships Christopher J. Schaffer 1 William A. Gallus Jr. 2 Moti Segal 2 1 National Weather Service, WFO Goodland 2 Iowa State University, Ames, IA

2 Ensemble vs. Deterministic Probabilistic forecasts provide uncertainty Small errors in forecast’s initial conditions grow exponentially (Hamill and Colucci 1997) Ensemble mean forecasts tend to be more skillful (Smith and Mullen 1993, Ebert 2001, Chakraborty and Krishnamurti 2006)

3 Gallus and Segal (2004) and Gallus et al. (2007) Precipitation-binning technique for deterministic forecasts Larger forecasted precipitation => greater probability to receive precipitation POPs increased further if different models showed an intersection of grid points with rain in a bin

4 Overview of study Goals – Apply post-processing techniques similar to the Gallus and Segal (2004) technique to ensemble forecasts – Examine how the forecasts compare to those from more traditional approaches

5 Data NOAA Hazardous Weather Testbed (HWT) Spring Experiments (2007 and 2008) Ensemble of ten WRF-ARW members with 4 km grid spacing run by Center for Analysis and Prediction of Storms (CAPS) 30 hours per case (five 6-hour time periods); 00Z Present study uses a subdomain of 2007/2008 Coarsened onto 20 km grid spacing

6 1980 km x 1840 km rather than 3000 km x 2500 km (2007) Subdomain of Present Study

7 Methodology Creation of 2D POP tables – Forecasted precipitation amount within a bin Maximum or average amount – Number of ensemble members forecasting agreement on precipitation amounts above a threshold

8 Methodology continued Seven precipitation bins POPs assigned through hit rates NCEP Stage IV observations designated hits Three thresholds: 0.01, 0.10, and 0.25 inch -h is the number of “hits”, or points where the observed precipitation also exceeded the specified threshold -f is the number of grid points with precipitation forecasted for a given bin/member scenario

9 Approach #1 Two-parameter point forecast approach

10 <0.01 0.01- 0.05 0.05- 0.10 0.10- 0.25 0.25- 0.50 0.50- 1.0>1.0 Col Ave (Cali_trad) 0%2.8------ 10%-11.415.418.118.619.728.712.8 20%-14.319.222.323.826.731.318 30%-16.123.526.230.530.639.123.4 40%-18.52531.536.339.9 29.3 50%-19.427.63642.545.846.835.5 60%-19.728.339.347.452.955.341.6 70%-2331.442.553.15761.747.9 80%-21.53347.256.963.366.354.5 90%-15.835.453.666.871.477.665.5 100%-16.827.655.273.383.989.278.6 Row Ave2.813.723.736.653.165.674.519.4

11 Max_thr POPs for April 23, 2007 06Z – 12Z

12 Cali_trad POPs for April 23, 2007 06Z – 12Z

13 Max_thr – Cali_trad

14 Score MethodBSReliResolUncertBSSBias GSD 0.01 inch 0.11750.00730.03540.14560.19321.3488 0.10 inch 0.06530.00460.01610.07670.14891.6043 0.25 inch 0.03860.00290.00720.04290.10061.9621 Uncali_trad 0.01 inch 0.12340.02570.04800.14560.15301.4707 0.10 inch 0.07050.01520.02140.07670.08101.6305 0.25 inch 0.04400.01050.00950.0429-0.02431.9159 Cali_trad 0.01 inch 0.10400.00640.04800.14560.28551.2609 0.10 inch 0.05930.00400.02140.07670.22671.4582 0.25 inch 0.03630.00280.00950.04290.15471.7572 Max_thr 0.01 inch 0.10130.00970.05400.14560.30411.2501 0.10 inch 0.05860.00590.02400.07670.23571.4192 0.25 inch 0.03590.00370.01080.04290.16331.6722

15 Approach #2 Two-parameter neighborhood approach

16 Neighborhoods: Theis et al. (2005), Ebert (2009) Within a specified square area around a center point, the max or ave precip. amount is determined and binned Number of points within the neighborhood that have forecast precip. amounts greater than a threshold Spatially generated ensemble Forecasts for each member 123 456 789 3x3 Neighborhood

17 MemberScore BSReliResolUncertBSSBiasROC area 0.01 inch Mem1 0.10430.02790.06910.14560.28361.48900.862 Mem2 0.11130.02990.06420.14560.23541.42520.850 Mem3 0.10910.02610.06260.14560.25071.06030.829 Mem4 0.11020.02710.06250.14560.24301.18540.835 Mem5 0.11090.02800.06280.14560.23851.28270.836 Mem6 0.09900.02320.06990.14560.32031.15320.860 Mem7 0.10370.02700.06900.14560.28811.41370.863 Mem8 0.09880.02350.07030.14560.32181.17300.861 Mem9 0.09960.02390.06990.14560.31630.95870.861 Mem10 0.10070.02520.07010.14560.30851.36270.869 Statistics for Ave_nbh (15x15 g.p.)

18 Scatterplot of Brier scores (0.01 inch threshold)

19 Scatterplot of Brier scores (0.10 and 0.25 inch thresholds)

20 Approach #3 Combination of methods

21 Considers each method as an ensemble member that itself consists of ensemble members Uses the different POP tables to determine POPs for each method, then averages POPs Different trends in POP fields Many variations of the approach

22 Score ThresholdBSReliResolUncertBSSBiasROC area 3x3 0.01 inch 0.09950.00970.05590.14560.31701.30980.818 0.10 inch 0.05730.00610.02550.07670.25291.54350.872 0.25 inch 0.03490.00390.01190.04290.18701.87000.894 15x15 0.01 inch 0.09590.01040.06010.14560.34111.25120.875 0.10 inch 0.05560.00660.02780.07670.27591.41260.903 0.25 inch 0.03400.00420.01310.04290.20831.64980.916 P-value (compared to Cali_trad): 0.07884 (90% C.I.)

23 Combination approach

24 Max_thr

25 Conclusions Two-parameter point forecast approach Improvements over Cali_trad, which encouraged the development of other approaches Two-parameter neighborhood approach Deterministic, but comparable to Cali_trad Improvements due to spatial ensembles Increased neighborhood size led to better Brier scores Combination approach Brings several methods/approaches together by averaging POPs Statistically significantly different Brier scores compared to Cali_trad at 90% C.I.

26 This research was funded in part by National Science Foundation grants ATM-0537043 and ATM-0848200, with funds from the American Recovery and Reinvestment Act of 2009. christopher.schaffer@noaa.gov

Download ppt "Improving Probabilistic Ensemble Forecasts of Convection through the Application of QPF-POP Relationships Christopher J. Schaffer 1 William A. Gallus Jr."

Similar presentations

Model Evaluation Tools MET. What is MET Model Evaluation Tools ( MET )- a powerful and highly configurable verification package developed by DTC offering:

Model Evaluation Tools MET. What is MET Model Evaluation Tools ( MET )- a powerful and highly configurable verification package developed by DTC offering:
5 th International Conference of Mesoscale Meteor. And Typhoons, Boulder, CO 31 October 2006 National Scale Probabilistic Storm Forecasting for Aviation.

5 th International Conference of Mesoscale Meteor. And Typhoons, Boulder, CO 31 October 2006 National Scale Probabilistic Storm Forecasting for Aviation.
Verification and calibration of probabilistic precipitation forecasts derived from neighborhood and object based methods for a convection-allowing ensemble.

Verification and calibration of probabilistic precipitation forecasts derived from neighborhood and object based methods for a convection-allowing ensemble.
Toward Improving Representation of Model Microphysics Errors in a Convection-Allowing Ensemble: Evaluation and Diagnosis of mixed- Microphysics and Perturbed.

Toward Improving Representation of Model Microphysics Errors in a Convection-Allowing Ensemble: Evaluation and Diagnosis of mixed- Microphysics and Perturbed.
Gridded OCF Probabilistic Forecasting For Australia For more information please contact © Commonwealth of Australia 2011 Shaun Cooper.

Gridded OCF Probabilistic Forecasting For Australia For more information please contact © Commonwealth of Australia 2011 Shaun Cooper.
Improving Excessive Rainfall Forecasts at HPC by using the “Neighborhood - Spatial Density“ Approach to High Res Models Michael Eckert, David Novak, and.

Improving Excessive Rainfall Forecasts at HPC by using the “Neighborhood - Spatial Density“ Approach to High Res Models Michael Eckert, David Novak, and.
Validation of the Ensemble Tropical Rainfall Potential (eTRaP) for Landfalling Tropical Cyclones Elizabeth E. Ebert Centre for Australian Weather and Climate.

Validation of the Ensemble Tropical Rainfall Potential (eTRaP) for Landfalling Tropical Cyclones Elizabeth E. Ebert Centre for Australian Weather and Climate.
NWP Verification with Shape- matching Algorithms: Hydrologic Applications and Extension to Ensembles Barbara Brown 1, Edward Tollerud 2, Tara Jensen 1,

NWP Verification with Shape- matching Algorithms: Hydrologic Applications and Extension to Ensembles Barbara Brown 1, Edward Tollerud 2, Tara Jensen 1,
Reliability Trends of the Global Forecast System Model Output Statistical Guidance in the Northeastern U.S. A Statistical Analysis with Operational Forecasting.

Reliability Trends of the Global Forecast System Model Output Statistical Guidance in the Northeastern U.S. A Statistical Analysis with Operational Forecasting.
Exploring the Use of Object- Oriented Verification at the Hydrometeorological Prediction Center Faye E. Barthold 1,2, Keith F. Brill 1, and David R. Novak.

Exploring the Use of Object- Oriented Verification at the Hydrometeorological Prediction Center Faye E. Barthold 1,2, Keith F. Brill 1, and David R. Novak.
Univ of AZ WRF Model Verification. Method NCEP Stage IV data used for precipitation verification – Stage IV is composite of rain fall observations and.

Univ of AZ WRF Model Verification. Method NCEP Stage IV data used for precipitation verification – Stage IV is composite of rain fall observations and.
The Consideration of Noise in the Direct NWP Model Output Susanne Theis Andreas Hense Ulrich Damrath Volker Renner.

The Consideration of Noise in the Direct NWP Model Output Susanne Theis Andreas Hense Ulrich Damrath Volker Renner.
¿How sensitive are probabilistic precipitation forecasts to the choice of the ensemble generation method and the calibration algorithm? Juan J. Ruiz 1,2.

¿How sensitive are probabilistic precipitation forecasts to the choice of the ensemble generation method and the calibration algorithm? Juan J. Ruiz 1,2.
Juan Ruiz 1,2, Celeste Saulo 1,2, Soledad Cardazzo 1, Eugenia Kalnay 3 1 Departamento de Cs. de la Atmósfera y los Océanos (FCEyN-UBA), 2 Centro de Investigaciones.

Juan Ruiz 1,2, Celeste Saulo 1,2, Soledad Cardazzo 1, Eugenia Kalnay 3 1 Departamento de Cs. de la Atmósfera y los Océanos (FCEyN-UBA), 2 Centro de Investigaciones.
Ensemble Post-Processing and it’s Potential Benefits for the Operational Forecaster Michael Erickson and Brian A. Colle School of Marine and Atmospheric.

Ensemble Post-Processing and it’s Potential Benefits for the Operational Forecaster Michael Erickson and Brian A. Colle School of Marine and Atmospheric.
Tara Jensen for DTC Staff 1, Steve Weiss 2, Jack Kain 3, Mike Coniglio 3 1 NCAR/RAL and NOAA/GSD, Boulder Colorado, USA 2 NOAA/NWS/Storm Prediction Center,

Tara Jensen for DTC Staff 1, Steve Weiss 2, Jack Kain 3, Mike Coniglio 3 1 NCAR/RAL and NOAA/GSD, Boulder Colorado, USA 2 NOAA/NWS/Storm Prediction Center,
Chapter 13 – Weather Analysis and Forecasting. The National Weather Service The National Weather Service (NWS) is responsible for forecasts several times.

Chapter 13 – Weather Analysis and Forecasting. The National Weather Service The National Weather Service (NWS) is responsible for forecasts several times.
Evaluation of Potential Performance Measures for the Advanced Hydrologic Prediction Service Gary A. Wick NOAA Environmental Technology Laboratory On Rotational.

Evaluation of Potential Performance Measures for the Advanced Hydrologic Prediction Service Gary A. Wick NOAA Environmental Technology Laboratory On Rotational.
Evaluation and Comparison of Multiple Convection-Allowing Ensembles Examined in Recent HWT Spring Forecasting Experiments Israel Jirak, Steve Weiss, and.

Evaluation and Comparison of Multiple Convection-Allowing Ensembles Examined in Recent HWT Spring Forecasting Experiments Israel Jirak, Steve Weiss, and.
Using Short Range Ensemble Model Data in National Fire Weather Outlooks Sarah J. Taylor David Bright, Greg Carbin, Phillip Bothwell NWS/Storm Prediction.

Using Short Range Ensemble Model Data in National Fire Weather Outlooks Sarah J. Taylor David Bright, Greg Carbin, Phillip Bothwell NWS/Storm Prediction.

Similar presentations

Ads by Google