Bob Livezey and Marina Timofeyeva Skill Variability Diagnosis for Discriminating Use of CPC Long-Lead Seasonal Forecasts Bob Livezey and Marina Timofeyeva NOAA/NWS/OCWWS/Climate Services Division Climate Prediction Applications Science Workshop March 9-11, 2004 Tallahassee, FL

Outline Introduction CPC Skill Graphs and Some Other Stratifications Results Conclusions and Lessons

Seasonal Temperature Forecast Skill 1960s to 80s Introduction Seasonal Temperature Forecast Skill 1960s to 80s All Seasons 8.3 Winter 12.6 Spring 8.6 Summer 9.3 Fall 2.8 Users should only care about the performance of forecasts that can potentially benefit their decision process Livezey (1990): There are non-random subsets of seasonal forecasts that were skillful enough to be useful After 1997-98 a common presumption was that forecasts are generally skillful enough to be useful

Introduction (Cont.) This talk will make the point : That was made by Livezey (1990) That there are many non-random subsets of forecasts that do not have useful skill That it is critical for this information to be shared with potential users That skill analyses with different stratifications are highly informative while CPC’s web displays are not

Displays and Stratifications CPC Seasonal Forecasts For 3-equally probable temperature and precipitation classes at 102 Climate Divisions Made every month from 1995 to present for 0.5-, 1.5-, …, 12.5 month leads Skill Measure: Modified Heidke Skill Score of Categorized Forecasts Displays and Stratifications CPC: Summed over all forecasts for each lead and displayed with times series for this lead Here: Summed over all forecasts for each lead and all leads displayed together Stratified further by cold seasons (DJF to FMA) and warm seasons (MAM to NDJ) Stratified further by strong ENSO years vs. other years Stratified by region

Displays and Stratifications Stratification by Lead and Seasons: Temperature CPC Seasonal Forecasts For 3-equally probable temperature and precipitation classes at 102 Climate Divisions Made every month from 1995 to present for 0.5-, 1.5-, …, 12.5 month leads Skill Measure: Modified Heidke Skill Score of Categorized Forecasts Displays and Stratifications CPC: Summed over all forecasts for each lead and displayed with times series for this lead Here: Summed over all forecasts for each lead and all leads displayed together Stratified further by cold seasons (DJF to FMA) and warm seasons (MAM to NDJ) Stratified further by strong ENSO years vs. other years Stratified by region

Further Stratification by Strong-ENSO vs Other Years: Temp.

Further Stratification by Strong-ENSO vs Other Years: Temp.

Displays and Stratifications CPC Seasonal Forecasts For 3-equally probable temperature and precipitation classes at 102 Climate Divisions Made every month from 1995 to present for 0.5-, 1.5-, …, 12.5 month leads Skill Measure: Modified Heidke Skill Score of Categorized Forecasts Displays and Stratifications CPC: Summed over all forecasts for each lead and displayed with times series for this lead Here: Summed over all forecasts for each lead and all leads displayed together Stratified further by cold seasons (DJF to FMA) and warm seasons (MAM to NDJ) Stratified further by strong ENSO years vs. other years Stratified by region

Stratification by Lead and Regions: Temp.

Results Seasonal Temperature: Useable skill confined to strong ENSO years and mainly at short to medium leads Otherwise skill is dominantly level with lead (derived from biased climatologies, ie long-term trend) Best forecasts are for strong-ENSO cold seasons at very short leads Worst forecasts are for cold seasons at longer leads for strong ENSOs and at very-short leads for other years Skill is substantially higher than average in the West, and substantially lower in the East Short-lead forecasts are better now than for the 1960s-80s, ~13 vs ~8 overall, ~20 vs ~13 for the winter

Stratification by Lead and Seasons: Precipitation

Further Stratification by Strong-ENSO vs Other Years: Precip.

Further Stratification by Strong-ENSO vs Other Years: Precip.

Stratification by Lead and Regions: Precip.

Results Seasonal Precipitation: Barely useable skill entirely confined to strong ENSO years in short to medium leads Otherwise skill is either negative or statistically indistinguishable from zero Best forecasts are for strong-ENSO cold seasons at short leads Skill is a little higher than average in the South, and a little lower in the North Short-lead forecasts overall seem to be no better now than for the 1960s-80s

Conclusions and Lessons There are non-random subsets of seasonal forecasts that are skillful enough to be useful There are many non-random subsets of forecasts that do not have useful skill (should we be issuing them?) It is critical for this information to be shared with potential users Skill analyses with different stratifications are highly informative while CPC’s web displays are not Skills mainly reflect ENSO and trend signals Temperature forecasts are getting better