National Hurricane Center 2008 Forecast Verification James L. Franklin Branch Chief, Hurricane Specialists Unit National Hurricane Center 2009 Interdepartmental Hurricane Conference James L. Franklin Branch Chief, Hurricane Specialists Unit National Hurricane Center 2009 Interdepartmental Hurricane Conference 1

Verification Rules  Verification rules unchanged for Results presented here in both basins are final.  System must be a tropical or subtropical cyclone at both forecast initial time and verification time. All verifications include depression stage except for GPRA goal verification.  Special advisories ignored (original advisory is verified.  Skill baselines are recomputed after the season from operational compute data. Decay- SHIFOR5 is the intensity skill benchmark.

2008 Atlantic Verification VT NT TRACK INT (h) (n mi) (kt) ============================ Values in green exceed all- time records. * 48 h track error for TS and H only (GPRA goal) was 87.5 n mi, just off last year’s record of 86.2.

Atlantic Track Errors by Storm Forecasts for Ike were relatively low, while those for Josephine and Omar were relatively high.

Atlantic Track Errors vs. 5-Year Mean Official forecast was better than the 5-year mean, even though the season’s storms were “harder” than normal.

Atlantic Track Error Trends Errors have been cut in half over the past 15 years was best year ever.

Atlantic Track Skill Trends 2008 was the most skillful year on record at all time periods.

Atlantic 5-Year Mean Track Errors Track errors increase by about n mi per day. 48 hr mean error below 100 n mi for the first time. Intensity errors level off because intensity is a much more bounded problem. New 5-yr means slightly larger than last year’s.

OFCL Error Distributions and Cone Radii Only modest reductions in the cone radii.

2008 Track Guidance Official forecast performance was very close to the consensus models. Best model was ECMWF, which was so good that it as good or better than the consensus. BAMD was similar to the poorest of the 3-D models (UKMET). AEMI excluded due to insufficient availability (less than 67% of the time at 48 or 120 h).

2008 Track Guidance Examine major dynamical models to increase sample size. ECMWF best at all time periods (as opposed to last year, when it was mediocre). GFDL also better than last year (and better than HWRF). As we’ve seen before, GFDL skill declines relatively sharply at days 4-5. NOGAPS and GFNI again performed relatively poorly. GFNI upgrades were delayed.

GFDL-HWRF Comparison Much larger sample than last year shows that the HWRF is competitive with, but has not quite caught up to the GFDL yet. Consensus of the two (mostly) better than either alone.

Guidance Trends Return to more “traditional” relationships among the models after the very limited sample of 2007.

Guidance Trends Relative performance at 120 h is more variable, although GFSI has been strong every year except GFDL is not a good performer at the longer ranges.

Consensus Models Best consensus model was TVCN, the variable member consensus that includes EMXI. It does not appear that the “correction” process was beneficial.

Consensus Models Third year in a row AEMI trailed the control run. Multi- model ensembles remain far more effective for TC forecasting. ECMWF ensemble mean is also not as good as the control run (EEMN v EMX).

Atlantic Intensity Errors vs. 5-Year Mean OFCL errors in 2008 were at or below the 5-yr means, but the 2008 Decay-SHIFOR errors were also at or below their 5-yr means, so not much change in skill.

Atlantic Intensity Error Trends No progress with intensity.

Atlantic Intensity Skill Trends Little net change in skill over the past several years.

2008 Intensity Guidance Split decision between the dynamical vs statistical models. New ICON consensus, introduced this year, was very successful, beating OFCL except at 12 h. OFCL adds most value over guidance at shorter ranges. Modest high bias in 2008 (2007 was a low bias).

2008 Intensity Guidance HWRF competitive through 3 days, with issues at the longer times. Although the sample was smaller, there was a hint of this last year as well. Cannot shut GFDL off yet!

2008 Intensity Guidance When the complication of timing landfall/track dependence is removed, OFCL performs better relative to the guidance. Dynamical models are relatively poor performers.

2008 East Pacific Verification VT NT TRACK INT (h) (n mi) (kt) ============================ Values in green tied or exceeded all-time lows.

2008 vs 5-Year Mean Slightly easier than average year (CLIPER errors below their 5-yr mean). Official errors were also below their 5-yr means (by a slightly larger margin).

EPAC Track Error Trends Since 1990, track errors have decreased by 30%-50%.

EPAC Track Skill Trends Skill continues to improve.

OFCL Error Distributions and Cone Radii Approximate 10% reduction in cone radii.

2008 Track Guidance EMXI, EGRI, AEMI, FSSE, GUNA, TCON excluded due to insufficient availability. Official forecast beat the TVCN consensus at later periods; beat each individual model. OFCL far superior to model guidance at longer time periods (also beat consensus at 4-5 days last year).

2008 Track Guidance Relax selection criteria to see all major dynamical models. ECMWF best overall. OFCL clearly doing something right.

GFDL-HWRF Comparison Overall, HWRF performance not as good as the GFDL. Consensus better than either one alone.

Consensus Models Best was the variable consensus that includes EMXI. Corrected consensus models did not excel.

Consensus Models Similar performance to the Atlantic. Added value at longer lead times.

Eastern North Pacific Intensity Errors vs. 5-year Mean In 2008 both the OFCL and Decay- SHIFOR5 errors were below their 5-yr means. No dramatic changes in skill.

EPAC Intensity Error Trends Perhaps just a hint of improvement?

EPAC Intensity Skill Trends Skill does seem to be inching upward…

2008 Intensity Guidance OFCL mostly beat the individual models and even the consensus at some time periods. OFCL wind biases turn sharply negative at h, which was also true in Statistical models outperformed dynamical models. This year, DSHP beat LGEM (flip from 2007).

2008 Intensity Guidance HWRF competitive through 3 days, with issues at the longer times. Although the sample was smaller, there was a hint of this last year as well. Cannot shut GFDL off yet!

Genesis Forecast Verification

Genesis Forecast Verification Lead-Time Analysis for Disturbances that became Tropical Cyclones Time wrt/Genes is -48 h-42 h-36 h-30 h-24 h-18 h-12 h-6 h Avg. %31% 34%37%41%45%53%61% Time wrt/Genes is -48 h-42 h-36 h-30 h-24 h-18 h-12 h-6 h Avg. %29%30%28%30%31%35%42%52% Atlantic Eastern North Pacific

Genesis Bins for 2009 ATLANTIC Range (%)% Expected% Verified# Forecasts 0-20 (Low) (Med) (High) EASTERN NORTH PACIFIC Range (%)% Expected% Verified# Forecasts 0-20 (Low) (Med) (High) NHC will issue operational public quantitative/categorical genesis forecasts in 2009 and include categorical forecasts in the text Tropical Weather Outlook.

Summary: Atlantic Track  OFCL track errors set records for accuracy at all time periods. Errors continue their downward trends, skill was also up.  OFCL track forecast skill was very close to that of the consensus models, was beaten by EMXI.  EMXI and GFDL provided best dynamical track guidance. UKMET, which performed well in 2007, did not do so in NOGAPS lagged again.  HWRF has not quite attained the skill of the GFDL, but is competitive. A combination of the two is better than either alone.  Best consensus model was TVCN (variable consensus with EMXI). Multi-model consensus – good. Single model consensus – not so good. Not a good year for the “corrected consensus” models.

Summary: Atlantic Intensity  OFCL errors in 2008 were below the 5-yr means, but the 2008 Decay-SHIFOR errors were also lower than its 5-yr mean, so no real change in skill.  Still no progress with intensity errors; OFCL errors have remained unchanged over the last 20 years. Skill has been relatively flat over the past 5-6 years.  Split decision between the statistical and dynamical guidance. Simple four-model consensus (DSHP/LGEM/HWRF/GHMI) beat everything else, including the corrected consensus model FSSE.

Summary: East Pacific Track  OFCL track errors set records at h.  OFCL beat individual dynamical models, and also beat the consensus at 4 and 5 days.  GFDL, HWRF, and ECMWF were strong performers, although ECMWF had trouble holding on to systems through 5 days.  There continues to be a much larger difference between the dynamical models and the consensus in the eastern North Pacific than there is in the Atlantic, which is suggestive of different error mechanisms in the two basins.

Summary: East Pacific Intensity  OFCL mostly beat the individual models and even the consensus at 12 and 36 h. OFCL wind biases turned sharply negative at h, which was also true in  Best model at most time periods was a statistical model. DSHP provided most skillful guidance overall. HWRF continued to have trouble in this basin. Four-model intensity consensus performed very well.