1. An Experiment to Evaluate the Use of Quantitative Precipitation Forecasts from Numerical Guidance by Operational Forecasters Joshua M. Boustead and Daniel Nietfeld NOAA/NWS WFO Omaha/Valley, NE Ray Wolf NOAA/NWS WFO Davenport, IA

2. Presentation Overview Study purpose and methodology Data results –Survey results –Snowfall forecast –Watch/warning statistics –Gridded forecast results Forecasting implications, conclusions, and future work

3. Study Motivation Strong interest in the role of the future forecaster –Can we still add value to the everyday forecast? –How can we better concentrate on high- impact weather? –How can we better utilize increasingly high- tech tools into the everyday forecast? How does this increasingly high-tech information affect the forecaster?

4. Example NSSL 4km WRF 00Z 8/14/07

5. Results

6. Study Purpose To evaluate if and how operational forecasters are biased by numerically generated quantitative precipitation forecasts (QPF) Use these results to develop an updated methodology for operational forecasters on how to approach a daily forecast and utilize the latest technology, including high resolution model output

7. Study Methodology Utilizing the National Weather Service’s (NWS) Warning Event Simulator (WES) operational forecasters from two NWS offices made two forecasts for two winter weather case – The forecasters first completed the forecast, including making a warning decision, without the use of model QPF – The forecasters then went through the same case again with model QPF, again making a snowfall forecast as well as a warning decision Once each scenario was completed, the forecasters completed a survey about the specific case

8. Study Methodology Two winter weather cases were chosen from the Central and Northern Plains – December 7-8, 2005 from the Pleasant Hill, MO (EAX) forecast area – February 28 – March 1, 2004 from the Bismarck, ND (BIS) forecast area

9. Survey Results Forecaster Demographics: –Forecasters were from the NWS offices in Omaha/Valley, NE and Davenport, IA –Operational forecasters involved were of a high experience level

10. Survey Results Forecaster confidence without using model QPF: –Majority of operational forecasters felt confident in making a forecast without model QPF –Potentially due to the high experience level of the forecasters

11. Survey Results Forecaster confidence after seeing QPF: –Most forecasters indicated that seeing QPF either increased their forecast confidence or it was unchanged

12. Snowfall Forecast Results MAE was computed for each location and then averaged for before and after the use of QPF MAE decreased 0.5 inches for both the EAX and BIS case post QPF

13. Snowfall Forecast Results Majority of the forecasts were unchanged post QPF Majority of the forecasts that did change their forecast, increased accuracy

14. Warning Results The probability of detection (POD) and false alarm ratio (FAR) were computed by county for each of the forecast areas Forecasters showed improvement in both the POD and in FAR ratio once QPF was used

15. Warning Results

16. Gridded Forecast Results EAX Case EAX Pre and Post QPF MAE –Forecasters had the most confidence in the northern CWA –Much better agreement over the southern CWA post QPF –Also a 2 to 3 inch decrease in MAE over the south

17. Gridded Forecast Results EAX Case EAX Pre and Post QPF Standard Deviation –Forecast differences decreased over the north and south –Slight increase in differences over the center

18. Gridded Model Forecasts EAX Case Greatest agreement of snow band across central CWA Viewing QPF increased the forecast confidence in the southern CWA

19. Actual Snowfall EAX Case

20. Gridded Forecast Results Bismarck Case BIS Pre and Post QPF MAE –Good agreement and low error over the northwest forecast area –Mean errors of 5 to 6 inches over the southern and eastern forecast area

21. Gridded Forecast Results Bismarck Case Pre and Post QPF Standard Deviation –Significant increase in forecaster clustering across the central forecast area –Greater than 4 inch differences continue over the southern forecast area

22. Gridded Model Forecasts BIS Case Models agree northwest CWA to get least QPF Larger uncertainty in the south Forecasters tended to pick a model –Led to continued large MAE in the southern CWA

23. Actual Snowfall BIS Case

24. Discussion Only a slight improvement in snowfall forecasts was noted once forecasters viewed QPF –When snowfall forecasts were modified, a higher percentage were improved than degraded Model QPF seemed best utilized to resolve snow-no snow areas –This led to improvements in both FAR and POD High MAE did not always mean high standard deviation, which can indicate a systematic forecasting error Doesn’t clearly answer the question does model QPF bias forecasters –Some evidence in the BIS case where model agreement was poor Possible forecast methodology –Make entire forecast without QPF –Utilize QPF for placement for defining snow-no snow areas

25. Future Work Conduct the study using two warm season convective cases Investigate forecaster philosophy from the surveys where standard deviation is low and mean absolute error is higher Investigate what, if any, synoptic patterns increase forecaster uncertainty and MAE Continue to increase the number of forecasters in the study, and from different areas of the CONUS