1. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 20101 Operational Seasonal Forecast Systems: a view from ECMWF Tim Stockdale The team: Franco Molteni, Magdalena Balmaseda, Kristian Mogensen, Frederic Vitart, Laura Ferranti European Centre for Medium-Range Weather Forecasts

2. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 20102 Outline Operational seasonal systems at ECMWF  System 3 - configuration  System 3 – products  System 3 – skill measures EUROSIP  ECMWF, Met Office and Météo-France multi-model system Some (relevant) issues in seasonal prediction  Estimating skill and model improvement  Cost effective systems  Multi-model systems, data sharing policy

3. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 20103 Sources of seasonal predictability  KNOWN TO BE IMPORTANT: oEl Nino variability- biggest single signal oOther tropical ocean SST- important, but multifarious oClimate change- especially important in mid-latitudes oLocal land surface conditions- e.g. soil moisture in spring  OTHER FACTORS: oVolcanic eruptions- definitely important for large events oMid-latitude ocean temperatures- still somewhat controversial oRemote soil moisture/ snow cover- not well established oSea ice anomalies- local effects, but remote? oDynamic memory of atmosphere- most likely on 1-2 months oStratospheric influences- solar cycle, QBO, ozone, …  Unknown or Unexpected- ???

4. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 20104 ECMWF operational seasonal forecasts Real time forecasts since 1997  “System 1” initially made public as “experimental” in Dec 1997  System 2 started running in August 2001, released in early 2002  System 3 started running in Sept 2006, operational in March 2007 Burst mode ensemble forecast  Initial conditions are valid for 0Z on the 1 st of a month  Forecast is created typically on the 11 th /12 th (SST data is delayed up to 11 days)  Forecast and product release date is 12Z on the 15 th. Range of operational products  Moderately extensive set of graphical products on web  Raw data in MARS  Formal dissemination of real time forecast data

5. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 20105 ECMWF System 3 – the model IFS (atmosphere)  T L 159L62 Cy31r1, 1.125 deg grid for physics (operational in Sep 2006)  Full set of singular vectors from EPS system to perturb atmosphere initial conditions (more sophisticated than needed …)  Ocean currents coupled to atmosphere boundary layer calculations HOPE (ocean)  Global ocean model, 1x1 mid-latitude resolution, 0.3 near equator  A lot of work in developing the OI ocean analyses, including analysis of salinity, multivariate bias corrections and use of altimetry. Coupling  Fully coupled, no flux adjustments, except no physical model of sea-ice

6. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 20106 System 3 configuration Real time forecasts:  41 member ensemble forecast to 7 months  SST and atmos. perturbations added to each member  11 member ensemble forecast to 13 months  Designed to give an ‘outlook’ for ENSO  Only once per quarter (Feb, May, Aug and Nov starts)  November starts are actually 14 months (to year end) Back integrations from 1981-2005 (25 years)  11 member ensemble every month  5 members to 13 months once per quarter

7. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 20107

8. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 20108

9. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 20109 Other operational plots for DJF 2010/11

10. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201010 Tropical storm forecasts

11. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201011 Rms error of forecasts has been systematically reduced (solid lines) …. Performance – SST and ENSO.. but ensemble spread (dashed lines) is still substantially less than actual forecast error.

12. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201012 More recent SST forecasts are better.... 1981-1993 1994-2007

13. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201013 At longer leads, model spread starts to catch up

14. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201014 How good are the forecasts? Temperature: actual forecastsTemperature: perfect model Deterministic skill: DJF ACC

15. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201015 How good are the forecasts? Precip: actual forecastsPrecip: perfect model Deterministic skill: DJF ACC

16. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201016 How good are the forecasts? Tropical precip < lower tercile, JJANH extratrop temp > upper tercile, DJF Probabilistic skill: Reliability diagrams

17. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201017 How good are the forecasts? Europe: Temp > upper tercile, DJF Probabilistic skill: Reliability diagrams

18. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201018 EUROSIP multi-model ensemble Three European models so far:  ECMWF  Met Office  Meteo-France  Germany planning to contribute NCEP has just become an associate partner  Not yet integrated into system An evolving system  Real-time since mid-2005  Common operational schedule (products released at 12Z on 15 th )  Monthly mean data in ECMWF operational archive (daily from some partners)

19. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201019 EUROSIP

20. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201020 Single modelMulti- model

21. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201021 Reliability diagrams (T2m > 0) 1-month lead, start date May, 1980 - 2001 DEMETER: multi-model vs single-model 0.039 0.899 0.141 0.039 0.899 0.140 0.095 0.926 0.169 -0.001 0.877 0.123 0.065 0.918 0.147 -0.064 0.838 0.099 0.047 0.893 0.153 0.204 0.990 0.213 multi- model Hagedorn et al. (2005) BSS Rel-Sc Res-Sc

22. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201022 Some (relevant) issues in seasonal prediction

23. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201023 Tentative results from ECMWF S4 System 3 Cy36r4 - T159L62 (11 members, 20 years)

24. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201024 Alternate stochastic physics 0.346 vs 0.294 A real improvement, now scoring better than S3 T159L91, plus revised stratospheric physics Only 5 members, but score of 0.342 is much better than L62

25. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201025 T255L91 Score is now 0.390, cf 0.294 for T159L62 T255L91, with alternate stochastic physics Score is 0.273 From the best to the worst! (Also other fields)

26. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201026 Possible interpretations  Statistical testing suggests differences are real, for this 20 year period  Different model configurations give different model “signals” in NH winter  Hope was that hemispheric averaging would increase degrees of freedom enough to make scores meaningful  Hypothesis 1: this is not true - a given set of signals gets a given score for the 20 year period, but this is of no relevance to expected model skill in the future, and cannot be used for model selection.  Hypothesis 2: Some model configurations really do better capture the “balance” of processes affecting NH winter circulation, even if it is via compensation of errors. Better to choose the model with the better score.

27. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201027 Choosing a model configuration Encouraging that some configurations give good results Higher horizontal and vertical resolution are consistently positive Model climate is much improved, again resolution clearly helps Forecast skill?? How should we weight seasonal forecast skill? What other tests should we use for a model? Links to extended/monthly forecast range??

28. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201028 Cost effective systems Back integrations dominate total cost of system  System 3:3300 back integrations (must be in first year)  492 real-time integrations (per year) Back integrations define model climate  Need both climate mean and the pdf, latter needs large sample  May prefer to use a “recent” period (30 years? Or less??)  System 2 had a 75 member “climate”, System 3 has 275.  Sampling is basically OK Back integrations provide information on skill  A forecast cannot be used unless we know (or assume) its level of skill  Observations have only 1 member, so large ensembles are much less helpful than large numbers of cases.  Care needed eg to estimate skill of 41 member ensemble based on past performance of 11 member ensemble  For regions of high signal/noise, System 3 gives adequate skill estimates  For regions of low signal/noise (eg <= 0.5), need hundreds of years

29. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201029 Data policy and exchange issues Present data policy  In Europe, constrains the free distribution/exchange of seasonal forecast data  Policy is not fixed in stone, and may evolve over time Science  Want to make sure that scientific studies are hindered as little as possible  CHFP is main research project on seasonal prediction; data policy has been OK, resources for data exchange were long a sticking point.  High level support for new projects may be helpful Real-time forecasts  Some data can be used by /supplied to WMO  Need to ensure that it is enough  Need to ensure that important “public good” applications are supported

30. Workshop on Sub-seasonal to Seasonal Prediction, Exeter, 1-3 December 201030 Conclusions Seasonal prediction still exciting and challenging Mid-latitude skill and reliability still need much work Higher resolution seems helpful Testing/assessing/selecting models needs to cut across timescales Coordinated experimentation has potential to be valuable, beyond CHFP Careful design will make it easier for operational centre’s to participate