1. Skill in the NMME Niño-3.4 Forecasts Following the 2015-16 El Niño
Michelle L’Heureux (NOAA CPC)
ENSO Forecast team (A. Barnston, E. Becker, G. Bell, T. Di Liberto, J. Gottschalk, M. Halpert, Z. Hu, N. Johnson, W. Wang, Y. Xue), M. Tippett

2. Niño-3.4 Forecasts from the NMME
Target Month

3. Niño-3.4 Forecasts from the NMME
Over-prediction for summer 2017 and beyond…
La Niña Watch! Stay tuned…
Predictions too cold for
Summer ‘16
Predictions hang onto cold too long for spring ‘17
Target Month

4. Ranked Probability Skill Score (RPSS)
Evaluates the entire distribution. Cumulative so depends on probabilities for other categories.
Varies between –∞ and +1. Positive scores are better than climatology.

5. Logarithmic Skill Score (LSS)
i is the category that the observations fall into.
p Is is the forecast probability for the i-th category
Only depends on the category that is observed. The forecaster with the largest probability in the verifying category obtains greater returns.
Varies between –∞ and +∞ . Positive scores are better than climatology.
Tippett, M.K., M. Ranganathan, M. L’Heureux, A. Barnston, T. DelSole (2017): Assessing probabilistic predictions of ENSO phase and intensity from the North American Multimodel Ensemble. Clim Dyn.

6. Official and NMME Forecast Comparison ~5 years of Forecasts (n = 66)
Verification Data:
Oceanic Niño Index (seasonal Niño-3.4 index) using three versions of ERSST
Thick black/blue lines: Median
Shading: 25% to 75%tile of scores

7. Official and NMME during 2016 and 2017 (scatter)
RPSS
6 = MJJ 2017
7 = JJA 2016

8. NMME forecasts during 2016-17
3 category LSS (-0.5°C,+0.5°C)
Lead Time (month)
Lead Time (month)
Better than climo
Worse
than climo

9. How does 2016-17 Compare to 2012-13 and 2014-15?
Target Month
Target Month
To be continued…
Target Month

10. How does 2016-17 Compare to 2012-13 and 2014-15?
NMME Lead-0 to Lead-3 LSS
2012
2014
2016
NMME Lead-4 to Lead-7 LSS
NMME Lead-4 to Lead-7 LSS
NMME Lead-8 to Lead-11 LSS
Target Month

11. How does 2016-17 Compare to 2012-13 and 2014-15?
NMME Lead-0 to Lead-3 RPSS
NMME Lead-4 to Lead-7 RPSS
NMME Lead-4 to Lead-7 LSS
NMME Lead-8 to Lead-11 RPSS

12. September 2016 ENSO Forecast (possible SubX project?)
This had ramifications. CPC dropped its La Niña Watch in September only to re-issue it in October.

13. CFSv2 (initialized within NMME)
Clear increase in the short-lead (0-month) forecast during the run made in early September 2016

14. CFSv2 (NOAA CPC display)
Initialized 8/22/16-
8/31/16
Initialized 9/11/16-
9/20/16
Courtesy of Wanqiu Wang (NOAA CPC)

15. Forecasted increase triggered by Tropical Instability Waves?
Observed Niño-3.4 index
~0.8°C to 1.0°C warm up and cool down during 9/20/16-
9/30/16
Daily OISST
9/19/16-
9/25/16
nnvl.noaa.gov

16. Wrap-Up
After successful predictions of Niño-3.4 during the El Niño, the time since then has been a little rockier.
NMME forecasts during the summer of 2016 were generally too cold, as were forecasts transitioning out of the La Niña.
Predictions for the summer of 2017 were too warm. Impending period of negative skill scores for longer lead predictions for the fall/winter (possible La Niña?).
Historical probabilistic metrics (LSS and RPSS) don’t suggest that the last year or previous set of years have unusually low skill relative to the NMME hindcast going back to 1982 (can look at this further).
Q: How much did September 2016 initial conditions contribute to the short-term increase in Niño-3.4 predictions?

17. Extra Slides

18. LSS of NMME forecasts during 2015-16
3 category (-0.5°C,+0.5°C)
9 category (-/+0.5, 1.0,1.5,2.0°C)
Lead Time (month)
Lead Time (month)
Better than climo
Worse
than climo