1. 1 Recent Reduction in the Area of “EC” in CPC Outlooks and its Relationship to Value Ed O’Lenic, Kenneth Pelman, and David Unger NOAA-NWS-Climate Prediction Center 34th Climate Diagnostics and Prediction Workshop, October 26-30, 2009, Monterey, California

2. 2 Motivation This paper updates our efforts to improve the skill and utility of CPC official (OFF) 3-Month Temperature and Precipitation Outlooks.  We demonstrate that the skill and value of CPC’s.5- month-lead 3-Month Temperature and Precipitation Outlooks has improved since we began using an objective method to consolidate the forecast tools (CON) in our operations starting in 2006.  To do this we first compare the performance of Official (OFF) outlooks, made in real-time from1995- 2004, with that of outlooks made by objective consolidation (CON) retrospectively over the same time period.

3. 3 Skill Measures We examine measures of performance including: 1)the fractional improvement by the forecast over random forecasts, s n, 2) the frequency (average area), n/T, of non-EC (equal chances) probabilities, and 3)s a, the area-weighted s n.  There is evidence of a sustained improvement in these metrics.  We also make a case for s a as a measure of value.

4. 4 s =fractional improvement over random forecasts T = # all forecasts n= # non-EC forecasts C = # correct forecasts E = # forecasts, out of T, expected correct by chance e= # forecasts, out of n, expected correct by chance EC= # EC forecasts Scores only non-EC areas Scores non-EC and EC areas n/T = frequency non-EC = % coverage

5. 5 The Case for Using s a as a Metric The official metric, s n, rewards larger EC regions, this is not in the interest of users. s a = (n/T) * s n, rewarding larger non-EC areas. Users have made clear that forecasts with fewer EC forecasts have higher overall value. Problem: s a is smaller, on average, than s n. Premise for this paper: s a is a measure of value

6. 6 Consolidation versus Official Forecasts Over 1995-2004 We applied the objective consolidation technique, previously used for our official SST forecasts, to CONUS temperature and precipitation. We compared the skill of consolidated forecasts (CON) with official forecasts (OFF) for 30 forecasts in each of 4 seasons from 1995-2004. OFF forecasts were those made in real-time. CON forecasts used the same 4 tools as OFF. Results are reported in O’Lenic et al, 2008, and Unger et al, 2009.

7. 7 NEW OTLK

8. 8 FMA, MAM, AMJ (30 fcsts) s a Decomposed into s n (contours), and n/T (colors) for.5 mo-lead 3-Month Precipitation Forecasts Change in n/T (CON-minus-OFF) is given in % at bottom. 1995-2004 mean s n *100 is at lower right in each figure. Skill,Official (OFF).5 mo-lead 3-Month Precipitation Outlook, 1995-2004 Skill, Consolidation (CON).5 mo-lead 3-Month Precipitation Outlook, 1995-2004 FMA, MAM, AMJ (30 fcsts) MJJ, JJA, JAS (30 fcsts) ASO, SON, OND (30 fcsts) NDJ, DJF, JFM (30 fcsts) MJJ, JJA, JAS (30 fcsts) ASO, SON, OND (30 fcsts) NDJ, DJF, JFM (30 fcsts) FMA, MAM, AMJ (30 fcsts) MJJ, JJA, JAS (30 fcsts) ASO, SON, OND (30 fcsts) NDJ, DJF, JFM (30 fcsts) +20% +8% +18% +16% 0.5 Month Lead 3-Mo Precipitation Outlooks, 1995-2004: CON Raises U.S. Annual Mean s n from 9 to 12, and Increases non-EC Forecasts in All Seasons 2 13 4 17 4 9 19 12

9. 9 FMA, MAM, AMJ (30 fcsts) Official (OFF).5 mo-lead 3-Month Temperature Outlook, 1995-2004 Consolidation (CON).5 mo-lead 3-Month Temperature Outlook, 1995-2004 NDJ, DJF, JFM (30 fcsts) 0.5 Month Lead 3-Mo Temperature Outlooks, 1995-2004: CON Raises U.S. Annual Mean s n from 18 to 24, and Increases non-EC Forecasts in All Seasons FMA, MAM, AMJ (30 fcsts) MJJ, JJA, JAS (30 fcsts) ASO, SON, OND (30 fcsts) FMA, MAM, AMJ (30 fcsts) MJJ, JJA, JAS (30 fcsts) +11% +31% +40% +55% NDJ, DJF, JFM (30 fcsts) ASO, SON, OND (30 fcsts) s a Decomposed into s n (contours), and n/T (colors) for 0.5 mo-lead 3-Month Temperature Outlooks Change in n/T (CON-minus-OFF) is given in % at bottom, 1995-2004 mean s n *100 is at lower right in each figure. 37 33 16 23 33 22 25

10. 10 GPRA Score Official Skill Metric: 48-Mo. Running Mean of s n Fraction x 100

11. 11 30 27 sd = 38 sd = 34 11 12 CONSOLIDATION IMPLEMENTED snsn sasa s a s n s n -s a n/T Fraction x 100 CONUS Spatial Average, 48-Month Running Mean of area covered,s n, s a, s n -s a 1995-2005 compared with post-2005, Temperature

12. 12 2.5 sd = 34 sd = 31 sasa snsn n/T Fraction x 100 CONUS Spatial Average, 48-Month Running Mean of area covered,s n, s a, s n -s a 1995-2005 compared with post-2005, Precipitation CONSOLIDATION IMPLEMENTED s a s n s n -s a n/T

13. 13 SUMMARY oRetrospective verification shows increase in the skill of objective Consolidation (CON) forecasts compared with Official (OFF), over 1995-2004. CONUS mean s n, n/T rise in all seasons, except winter for both T, P. For temperature, the s a OFF vs CON difference is significant at the 95% level, the s n rise is not. For precipitation, both s n and s a OFF vs CON differences are significant at the 95% level. oOFF forecasts made after 2005: 1. s a (Area-weighted s n ) is a user-centric skill and value metric. 2. Mean s n ~ same/higher than for 1995-2005, without strong ENSO. 3. Sustained increase in skill, frequency (s a ), and therefore, value. 4. The variability of s n drops, another measure of value. 5. Skill of L3MTO has risen as a direct result of use of CON. 6. Better forecasts can be made by focusing on value.

14. 14 CONUS Spatial Average, 48-Month Running Mean of area covered,s n, s a, s n -s a 1995-2005 compared with post-2005, Temperature 31 27 sd = 38 sd = 34 11 12 T, post 2005: N/T + s n rises, stays at levels above those during period including a strong ENSO s a rises from 8 to 12 Coverage rises to above strong ENSO levels sd falls 38 to 34 Mean s n ~ same as that for 1995-2005, but without strong ENSO CONSOLIDATION IMPLEMENTED snsn sasa s a s n s n -s a n/T Fraction x 100

15. 15 2.5 P, post 2005: Coverage rises s n rises to levels comparable to those during strong ENSO s a rises from 0.5 to 2.5 Mean s n ~ same as that for 1995-2005, but without strong ENSO sd falls 34 to 31 sd = 34 sd = 31 CONUS Spatial Average, 48-Month Running Mean of area covered,s n, s a, s n -s a 1995-2005 compared with post-2005, Precipitation CONSOLIDATION IMPLEMENTED sasa snsn s a s n s n -s a n/T Fraction x 100