Forecast Skill and Major Forecast Failures over the Northeastern Pacific and Western North America Lynn McMurdie and Cliff Mass University of Washington

Why the North Pacific Ocean? The Pacific is one of the largest areas of sparse insitu observations in the world Uncertainty over the Pacific has a large impact on predictability over the North American continent and beyond There are often large initialization errors and short-term forecasts over the northern Pacific ocean. One symptom of such problems is that short-term forecast skill on the western side of North America is worse than in other areas.

An example of a short-term forecast error Eta 24-h 03 March 00UT 1999 Eta 48-h 03 March 00UT 1999

And the public and media have noticed these failures….

Seattle Times Eugene Register Guard

Some applicable research in the literature Langland et al. (2002) – poor forecast of Jan 2000 storm on East Coast due in part to sensitivity over the Pacific. Bosart et al. (2002) – lack of convection over the midwest not represented in forecasts due to poorly initialized trough along west coast Schultz et al. (2005) – 70% of troughs arriving on the West Coast were underforecast, a portion of which continued to effect short-term forecasts across the North American continent. McMurdie and Mass (2004) – documented forecast failures over the eastern Pacific.

This talk will … Demonstrate that large initial condition and short-term forecast errors still occur over the eastern Pacific and downstream Present a feature-based approach to monitoring errors in this region Discuss implications for THORPEX

How frequent are large numerical forecast errors? Approach: compare buoys/coastal observations of sea level pressure (SLP) to NCEP’s Eta and GFS 00, 24, 36 and 48 hr forecasts. Error = Forecast SLP – Observed SLP At each station, calculated average and absolute error and the standard deviation using winter (Oct – Mar) data. Large Error = |Error| > (average error + 2 * SD)

Station Locations Tatoosh Is. Cape Arago

24 h Large Errors Tatoosh Is., WA Cape Arago, OR Inter-annual variability

48-h Errors 48h errors much larger and more frequent than 24-h errors

GFS vs. Eta 24-h errors NCEP GFS better than Eta on average

48-h errors GFS over forecasts Eta under forecasts

GFS has more accurate SLP initializations and forecasts than Eta over the Northeast Pacific For 00-h forecasts (initial conditions), GFS has smaller mean absolute error (MAE) and standard deviation (SD) than Eta at all 17 stations For 24- and 36-h forecasts, GFS has smaller MAE and SD than Eta at 13/17 buoy and coastal stations For 48-h forecasts, GFS has smaller MAE and SD than Eta at 12/17 stations.

Forecast Verification: The Need for Feature-Based Evaluation Examining statistics at observing sites is not sufficient for understanding the problems. Must also track features to gain an understanding of the deficiencies. Case studies of major failures should reveal important information.

GFS What are these large forecast errors associated with?

How frequently do large forecast errors of synoptic events occur? Number of Events/Season associated with Lows/Troughs/Highs SeasonTotalLowTroughHigh 1999 – – – – (from McMurdie and Mass 2004) Event = large error at 2 or more adjacent stations for 2 or more forecasts periods Data shown for Eta model only

Of the forecast failures associated with lows, what are the central pressure and cyclone position errors? Ave SLP error = 3.4 mb SD = 8.7 mb Absolute error = 7.5 mb Ave position error = km SD = 260 km

Recent examples of major forecast errors February 2002 October 2003 February 2004 November 2004 April 2005 May 2005

An example of a recent high-impact, poorly forecast storm Power outages, large trees uprooted in Eugene, OR Powerful, rapidly developing storm with strong winds (70 kts) Very poor short- term numerical guidance L 1008 L 1004 L 996 L – 8 February 2002 Cyclone

48-hr Forecasts Valid 00 UTC 8 February 2002 AVN UKMO ETA NOGAPS

24-Hr Forecasts Valid 00 UTC 8 February 2002 AVN UKMO ETA NOGAPS

Difference between UKMO and Eta 850 mb Temperature K Valid 00 UTC 7 February 2002 Solid = UKMO, Dashed = ETA, Shades, blue = differences L 1010

20 Oct 03

Flood of 20 October 2003

00hr GFS24hr GFS48hr GFS 00hr + 48hr GFS

GFS Forecasts of 12-hr Precipitation 12h Forecast 24h Forecast 36h Forecast 48h Forecast

February 04

00UTC 16 Feb 04 GFS 00-hr Forecast

00 UTC 16 Feb hr GFS

00 UTC 16 Feb hr GFS

Apr 05

24-hr forecast GFS Position error ~ 420km

Large Short Term Forecast Errors Still Occur Number of slp errors > 10 mb continues to be 10 – 15 per winter (despite the ridge this year) Vast majority of large errors due to mispositioned or under (or over) forecast low centers (see McMurdie and Mass, 2004) For Feb 02 case, forecast errors were likely due to initial condition errors (McMurdie and Mass, 2004)

Some Unanswered Questions What are the origins of these short-term forecast errors – initial condition/data assimilation errors, model errors? Are there particular flow patterns (or regimes) where short-term (or longer term) forecasts are less accurate (e.g., E-T transitions)? How do model sensitivity structures compare for major forecast failure cases? How do they project on obvious initialization problems? How do adjoint-based and ensemble-based sensitivities compare for such cases?

Unanswered Questions continued What are the downstream implications for medium to long- range forecasts when initial condition errors are large over the Pacific? To what degree are downstream errors mitigated by greater data density over North America?

Implications for THORPEX Major forecast failures still occur, even at the short- time ranges. So there is still work to be done! It is important to monitor the quality of model initializations and forecasts to know how well we are doing and where the failures are. Both statistical and feature-based approaches are needed to gain a full understanding of model failures. Case studies can provide important insights into forecast failures

The END

Recent trends in forecast accuracy From Simmons and Hollingsworth (2002) Hemispheric r.m.s. error of SLP Increased skill of 3-5 Day forecast skill of SLP (and 500 hghts) especially last 10 years. Unable to discern forecast skill of storm systems in particular locations from these statistics

Brief Outline Show statistics of short term errors along West Coast Highlight several examples of major forecast failures Briefly discuss the effect of uncertainty over the Pacific on longer term forecasts

Adjoint Sensitivity wrt 850 mb Temperature Area of forecast error projected onto sensitivities Courtesy of Brian Ancell

Nov

6-hr forecast Eta

6-hr Forecast GFS