1. Seth Linden and Jamie Wolff NCAR/RAL Evaluation of Selected Winter ’04/’05 Performance Results

2. Weather Forecast Verification Consensus (RWFS) forecast is compared to individual model components Air-temperature, dewpoint, wind-speed and cloud- cover forecasts –18 UTC runs for the entire season (1 November 2004 to 15 April 2005) Error (RMSE) calculated for: –Colorado Plains: 176 sites –Mountains: 119 sites Blizzard of March 2003

3. Air temperature RMSE Colorado Plains RWFS Colorado Mountains

4. Dewpoint RMSE Colorado Plains Forward Error Correction Colorado Mountains Due to 3-hour MOS data

5. Wind Speed RMSE Colorado Plains Colorado Mountains

6. Colorado Plains Cloud Cover RMSE Colorado Mountains

7. The ensemble approach utilized by the RWFS does improve the predictions on average for all verifiable parameters No single model performs better for all parameters A blend of weather models will provide better results Summary/Recommendations

8. Forecast Model Weights Used by the RWFS System automatically weights forecasts based on skill Distribution of weight values per lead time for air-temperature, dewpoint, and wind- speed –18 UTC run on 3 May 2005 Weights looked at for two sites: –Denver International Airport –I-70 at Genesse Which models have the most skill?

9. Air Temperature Model Weights Denver Int. Airport ETA GFS MOS I-70 at Genesee RUC

10. Dewpoint Model Weights Denver Int. Airport I-70 at Genesee

11. Denver Int. Airport Wind Speed Model Weights MM5 I-70 at Genesee WRF

12. Insolation Weights No one model consistently outperforms the others MM5 and WRF forecast hourly instantaneous values, ETA forecasts 3-hour instantaneous values and GFS forecasts 3-hour averages Clear Conditions For MDSS static weights were applied: - 50/50 split between MM5 and WRF for the 0-23 hour forecast - All Eta for the 24-48 hour forecast

13. QPF Weights ModelGFSEta MM5 2hr MM5 3hr MM5 4hr Total MM5 WRF 2 hr WRF 3hr WRF 4hr Total WRFRUC MAV- MOSTotal % TOTAL MM5+WRF Contribution QPF Weights (%) 911151210371714124300 100.0080 Due to a lack of quality precipitation observations static weights were applied Weights fixed based on expert opinion MM5 and WRF were given 80% of the total weight

14. Weight distribution reflects that the corrected (dynamic MOS) NWS models (ETA, GFS, and RUC) had the most overall skill No one model dominates for all parameters The limitation of the NWS models is their 3-hr temporal resolution WRF and MM5 were given the highest static weights for Insolation and QPF Summary/Recommendations

15. Road Temp Observation Variance T r variance across E-470 corridor –Shading by permanent structures or passing clouds –Make/model/installation/age of temperature sensors

16. E-470 Road/Bridge Sites Colorado Blvd Platte Valley (road and bridge) 6 th Ave Pkwy Plaza A Smokey Hill Rd (road and bridge)

17. SCTBKNOVC LOCAL TIME (19 = noon, 07 = midnight) 27 Nov 200428 Nov 2004

18. OVCCLRBKN SCT LOCAL TIME (19 = noon, 07 = midnight) 29 Nov 200430 Nov 2004

19. Summary/Recommendations Large variations in observed road and bridge temperatures –Over relatively small area (10s of miles) Makes prediction and verification of pavement temperatures very challenging –Difficult to establish ground truth

20. Road/Bridge Forecast Verification Road and bridge temperature forecasts –Using recommended treatments from MDSS Error (MAE) and bias calculated for: –For each lead time (0-48hrs) 18 UTC runs –E-470: 6 roads/2 bridge (1 Nov 2004 – 15 Apr 2005) –Mountains: 5 roads (1 Feb 2004 – 15 Apr 2005) East bound lane of I-70 at the summit of Vail Pass

21. Consistent low bias Lead Time (0 = 18 UTC = noon, 18 = 12 UTC = 6am) Peak insolation Morning hours E-470 road sites Perfect forecast

22. Lead Time (0 = 18 UTC ~ noon, 18 = 12 UTC ~ 6am) Shadowing? evening morning E-470 bridge sites

23. Lead Time (0 = 18 UTC = noon, 18 = 12 UTC = 6am) evening morning CDOT mountain road sites

24. Summary/Recommendations Larger T r differences during times of high solar insolation likely due to several factors: –Errors in measuring pavement skin temp –Mountain shading during low sun angle –Limitations in insolation prediction in models –Limitations in pavement heat balance model Simplified assumptions about pavement characteristics T b analysis compromised by: –Sensors shadowed by bridge rail –Bias results suggest tuning may be beneficial Overall Issue: –Actual/Recommended treatments not the same

25. Case Study Analysis 183 day demonstration –16 winter weather days 10 light snow 5 moderate snow 1 heavy snow

26. November 27-29, 2004 First significant snow storm of the season –5-8” in the Denver area Large variations in parameter predictions –Forecast vs. observations Denver International Airport Ta, Td, Wspd, Cloud Cover and Precipitation 12 UTC 28 th examined –Captured the start time of event

27. LOCAL TIME (19 = noon, 06 = midnight) 28 Nov 2005 8C/14F diff 2C/4F diff Air Temperature Snow

28. LOCAL TIME (19 = noon, 06 = midnight) 28 Nov 2005 6C/11F diff Dewpoint Temperature Snow

29. LOCAL TIME (19 = noon, 06 = midnight) 28 Nov 2005 Snow Wind Speed

30. FEC LOCAL TIME (19 = noon, 06 = midnight) 28 Nov 2005 Cloud Cover Snow

31. LOCAL TIME (19 = noon, 06 = midnight) 28 Nov 2005 Quantitative Precipitation Forecast Snow

32. March 13, 2005 Moderate Snow Event –4-6” along the E-470 corridor Warm air temps before start of snow –Dropped from 11C (52F) to -2C (29F) in 5 hours Large variations in parameter predictions –Forecast vs. observations Denver International Airport Ta, Wspd, Cloud Cover and Precipitation 00 UTC 13 March 2005 run examined –Captured both start and end times

33. LOCAL TIME (18 = noon, 07 = midnight) 13 March 2005 Air Temperature Snow

34. LOCAL TIME (18 = noon, 07 = midnight) 13 March 2005 Wind Speed Snow

35. LOCAL TIME (18 = noon, 07 = midnight) 13 March 2005 SCT - OVC Cloud Cover Snow

36. actualforecast Start time actualforecast End time LOCAL TIME (18 = noon, 07 = midnight) 13 March 2005 Quantitative Precipitation Forecast

37. April 10, 2005 00 and 18 UTC 10 April 2005 run –Capture start and end time of the event, respectively –QPF only presented Heavy snow event – 10-20” along E-470 – 15-25” along southern Denver CDOT routes – 20-30” along western Denver CDOT routes Air temps near freezing (-1C) throughout the event – Initial transition from rain to snow

38. 10 April 2005 Act/Fore start time Nearly 3”!! LOCAL TIME (18 = noon, 06 = midnight) Quantitative Precipitation Forecast

39. 10 April 2005 1” diff in total liquid equivalent precip 0.25” diff LOCAL TIME (18 = noon, 06 = midnight) Quantitative Precipitation Forecast

40. Summary/Recommendations Large discrepancies between weather models in predicting state weather parameters –All too dry for Td and cloud cover –Low wind speed bias during windy conditions –Overall, no ONE model outperforms => Ensemble approach key Supports probabilistic forecast presentation –Atmosphere is unpredictable –Best approach to present uncertainty to end users?

41. Thank You! Questions?