Skill in the NMME Niño-3.4 Forecasts Following the 2015-16 El Niño Michelle L’Heureux (NOAA CPC) ENSO Forecast team (A. Barnston, E. Becker, G. Bell, T. Di Liberto, J. Gottschalk, M. Halpert, Z. Hu, N. Johnson, W. Wang, Y. Xue), M. Tippett

Niño-3.4 Forecasts from the NMME Target Month

Niño-3.4 Forecasts from the NMME Over-prediction for summer 2017 and beyond… La Niña Watch! Stay tuned… Predictions too cold for Summer ‘16 Predictions hang onto cold too long for spring ‘17 Target Month

Ranked Probability Skill Score (RPSS) Evaluates the entire distribution. Cumulative so depends on probabilities for other categories. Varies between –∞ and +1. Positive scores are better than climatology.

Logarithmic Skill Score (LSS) i is the category that the observations fall into. p Is is the forecast probability for the i-th category Only depends on the category that is observed. The forecaster with the largest probability in the verifying category obtains greater returns. Varies between –∞ and +∞ . Positive scores are better than climatology. Tippett, M.K., M. Ranganathan, M. L’Heureux, A. Barnston, T. DelSole (2017): Assessing probabilistic predictions of ENSO phase and intensity from the North American Multimodel Ensemble. Clim Dyn.

Official and NMME Forecast Comparison ~5 years of Forecasts (n = 66) Verification Data: Oceanic Niño Index (seasonal Niño-3.4 index) using three versions of ERSST Thick black/blue lines: Median Shading: 25% to 75%tile of scores

Official and NMME during 2016 and 2017 (scatter) RPSS 6 = MJJ 2017 7 = JJA 2016

NMME forecasts during 2016-17 3 category LSS (-0.5°C,+0.5°C) Lead Time (month) Lead Time (month) Better than climo Worse than climo

How does 2016-17 Compare to 2012-13 and 2014-15? Target Month Target Month 2016-17 To be continued… Target Month

How does 2016-17 Compare to 2012-13 and 2014-15? NMME Lead-0 to Lead-3 LSS 2012 2014 2016 NMME Lead-4 to Lead-7 LSS NMME Lead-4 to Lead-7 LSS NMME Lead-8 to Lead-11 LSS Target Month

How does 2016-17 Compare to 2012-13 and 2014-15? NMME Lead-0 to Lead-3 RPSS NMME Lead-4 to Lead-7 RPSS NMME Lead-4 to Lead-7 LSS NMME Lead-8 to Lead-11 RPSS

September 2016 ENSO Forecast (possible SubX project?) This had ramifications. CPC dropped its La Niña Watch in September only to re-issue it in October.

CFSv2 (initialized within NMME) Clear increase in the short-lead (0-month) forecast during the run made in early September 2016

CFSv2 (NOAA CPC display) Initialized 8/22/16- 8/31/16 Initialized 9/11/16- 9/20/16 Courtesy of Wanqiu Wang (NOAA CPC)

Forecasted increase triggered by Tropical Instability Waves? Observed Niño-3.4 index ~0.8°C to 1.0°C warm up and cool down during 9/20/16- 9/30/16 Daily OISST 9/19/16- 9/25/16 nnvl.noaa.gov

Wrap-Up After successful predictions of Niño-3.4 during the 2015-16 El Niño, the time since then has been a little rockier. NMME forecasts during the summer of 2016 were generally too cold, as were forecasts transitioning out of the 2016-17 La Niña. Predictions for the summer of 2017 were too warm. Impending period of negative skill scores for longer lead predictions for the fall/winter 2017-18 (possible La Niña?). Historical probabilistic metrics (LSS and RPSS) don’t suggest that the last year or previous set of years have unusually low skill relative to the NMME hindcast going back to 1982 (can look at this further). Q: How much did September 2016 initial conditions contribute to the short-term increase in Niño-3.4 predictions?

Extra Slides

LSS of NMME forecasts during 2015-16 3 category (-0.5°C,+0.5°C) 9 category (-/+0.5, 1.0,1.5,2.0°C) Lead Time (month) Lead Time (month) Better than climo Worse than climo