Medium Range Forecasting at the Weather Prediction Center (WPC) – An Ensemble Effort in Big Data Tony Fracasso NROW XVII Nov. 2-3, 2016 Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Outline (?) What is a good forecast? It was the best of times, it was the worst of times… How to Succeed in Business Forecasting Without Really Trying Do no harm 1 – The first 60 minutes of my day can set the “forecast mood” – some examples of good vs bad forecasts 2 – When can automation help us? 3 – When does the forecaster need to adjust a starting point 4 – Rate of increase in data outpaces our ability to verify it – which is how we learn to adjust the forecast I Can’t Drive 55 What is a good forecast? Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Sixty Minutes to Success Top-down approach 500 hPa Surface (others as warranted) Hemispheric North American CONUS Regional ~WFO “Points” Guidance trends (ensembles and deterministic) Verification Anomalies / Impacts Blend! “Making a forecast” at WPC Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
A “Good” 7-Day Forecast (14 Oct 2016) The best of times 3.83°F MAE Avg ~ 5.0°F Max T Error (°F) Dept. from Avg. (°F) WPC Forecast Analysis Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
A “Bad” 7-Day Forecast (11 Oct 2016) The worst of times 6.99°F MAE Avg ~ 5.0°F Max T Error (°F) Dept. from Avg. (°F) WPC Forecast Analysis Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
A “Bad” 7-Day Forecast (11 Oct 2016) Maximum temperature errors from a few “blendable” options. Autoblend ECMWF ENS GEFS NAEFS ECMWF GFS Canadian Obs dept. Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Outside the Ensemble Envelope (Min) 168-h 500 hPa heights (00/12Z GEFS): 1 May – 25 Oct 2016 Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Outside the Ensemble Envelope (Max) 168-h 500 hPa heights (00/12Z GEFS): 1 May – 25 Oct 2016 Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
180hr forecast, valid 12Z 1 Oct 2016 (570dm) Ensemble Bifurcation – A Forecaster’s (Worst) Nightmare 29 Sept – 4 Oct 2016 Canadian GFS ECMWF 180hr forecast, valid 12Z 1 Oct 2016 (570dm) Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Multi-center Ensemble Spread Min/Max Temperatures Cincinnati, OH GFS OBS (67°F) ECMWF N Day 7 Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
20% 00Z ECMWF (bias-corrected) 30% 00Z ECENS Mean (bias-corrected) WPC Autoblend: 20% 00Z ECMWF (bias-corrected) 30% 00Z ECENS Mean (bias-corrected) 10% 06Z GFS (bias-corrected) 10% 06Z GEFS Mean (bias-corrected) 20% 00Z NAEFS 10% 00Z Gridded MOS Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Mean Absolute Error – Day 7 Max T 1 Jan – 14 Oct 2016 (CONUS) Relatively few opportunities to show significant large-scale improvement. Mean Absolute Error (°F) Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
MAE Difference (WPC minus Autoblend) - Day 7 Max T 1 Jan – 14 Oct 2016 (CONUS) Degrading Autoblend MAE Difference (°F) Improving Autoblend Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
‘Polar Vortex’ Event wpc.ncep.noaa.gov Facebook.com/NWSWPC/ @NWSWPC Image adapted from talk-politics.livejournal.com
Relative Placement Frequency 1 Jan – 14 Oct 2016 WPC Autoblend Percent Occurrence NBM Previous WPC Rank (1st – 22nd) Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
What is a good forecast? Least amount of overall error? Getting the extremes right? Best shows the probabilities? Produces the best public response? Gives the most lead time? Incremental changes toward “the right answer”? Other? Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
9 days prior 5 days prior 87°F wpc.ncep.noaa.gov Facebook.com/NWSWPC/
ECMWF Ensemble Trend (ALB) Jan 2016 Northeast Snow Storm 0.0 in. Decreasing lead time Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Summary Ensembles are great, but don’t have all the answers. A blended, bias-corrected dataset can show comparable skill to the human forecaster (Day 7 maximum temperature). “Wins” vs a skillful blend are likely local/regional rather than CONUS-wide. Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Questions?? Thanks! Anthony.Fracasso@noaa.gov wpc.ncep.noaa.gov Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Extra Slides Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
90-day avg 14 Oct 2016 11 Oct 2016 wpc.ncep.noaa.gov Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Where do ensembles fail below the min? GEFS: 1 Jan – 30 Apr2016 Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Mean Absolute Error – Day 7 Max T 1 Jan – 14 Oct 2016 Mean Absolute Error (°F) Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
National Blend Forecast Records from 5/12/16 Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
180 hr outside the GEFS (and ECENS) envelope - valid 12Z 11 Oct 2016 Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Ensemble Forecast Min/Max Temp (BIS) OBS (42°F) N Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Spag verif 12Z Dec 26 2012 wpc.ncep.noaa.gov Facebook.com/NWSWPC/
Forecasting Flow Chart High ensemble spread Low ensemble spread Zonal Flow Defined Waves Blocked Flow Low confidence Use ensemble means High confidence Average confidence Low confidence Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Forecasting Flow Chart High ensemble spread Low ensemble spread Zonal Flow Defined Waves Blocked Flow Low confidence Go in the middle Lean toward one side of the spread High confidence Average confidence Low confidence Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov
Facebook.com/NWSWPC/ @NWSWPC wpc.ncep.noaa.gov