1. ENSEMBLES: Seamless seasonal-to-decadal forecasting
Francisco J Doblas-Reyes, A. Weisheimer, T. N. Palmer
European Centre for Medium-Range Weather Forecasts
with contributions from Jean-Philippe Duvel, Noel Keenlyside, Malcolm McVean, James Murphy, Geert Jan van Oldenborgh, Doug Smith, Frédéric Vitart, Prince Xavier

2. Seamless systems
The WCRP strategic framewok encourages the development of seamless climate prediction systems: since climate in a region is an ensemble of weather events, understanding and predicting regional climate variability and climate change, including changes in extreme events, will require a unified initial-value approach that encompasses weather, intraseasonal oscillations, MJO, PNA, NAO, ENSO, PDO, THC, etc. and long-term trends, in a seamless modelling setup.
ENSEMBLES develops forecast systems predicting at several time scales: seasonal, interannual and beyond.
Downscaling and user needs are a major concern in ENSEMBLES (end-to-end approach).

3. The ENSEMBLES project
Integrated Project funded by the EC within the VIth FP, Sep 2004-Aug 2009, 69 partners.
Main goal: development of an integrated probabilistic prediction system for time scales from seasons to decades, and beyond. Therefore, research on both seasonal-to-decadal (s2d) and anthropogenic climate change (ACC).
Seasonal-to-decadal hindcasts are expected to be used to assess the reliability of forecast systems used for scenario runs, both from a climate modelling and user perspective.
Forecast uncertainty: comparison of the merits of multi-model, perturbed parameters and stochastic physics approaches to deal with model error.
Diversity of applications: health, crop yield, energy…

4. 3-month lead, summer (ASO) predictions of global T2m, ECMWF model
GHG effect in seasonal forecasts
3-month lead, summer (ASO) predictions of global T2m, ECMWF model
Constant GHG
Correlation = 0.29
Variable GHG
Correlation = 0.68

5. Dealing with forecast uncertainty
Model error is a major source of forecast uncertainty. Three approaches to deal with model error are being investigated in ENSEMBLES: multi-model, stochastic physics and perturbed parameters.
The multi-model is built from ECMWF, Met Office, Météo-France operational activities and the DEMETER experience at IfM-Kiel, CERFACS and INGV.
Perturbed parameter system stems from the decadal prediction system (DePreSys) created at the Met Office.
Stochastic physics system uses the ECMWF stochastic backscatter system developed for medium-range forecasts.
A major s2d experiment to determine the benefits of each approach is ongoing (~20,000 years of integrations).

6. S2d ENSEMBLES experiment
Three systems: multi-model (ECMWF, GloSea, DePreSys, Meteo-France, IfM-Kiel, CERFACS, INGV), stochastic physics (ECMWF) and perturbed parameters (DePreSys).
Hindcasts in two streams:
Stream 1: hindcast period , seasonal (7 months, May and November start date), annual (14 months, November start date) and 2 decadal (1965 and 1994), 9 member ensembles, ERA40 initialization in most cases.
Stream 2: As in Stream 1 but over , with 4 start dates for seasonal hindcasts, at least 1 for annual and at least one 3-member decadal hindcast every 5 years.
Additional simulations: DePreSys carries out a 10-year hindcast every year + lots of sensitivity experiments from the other contributors.
Wide range of ensembles of ocean analyses (using the in situ ENACT/ENSEMBLES dataset) to initialize the hindcasts.

7. Stream 1 annual hindcasts
Sea surface temperature RMSE (solid) and spread (dashed) averaged over the Niño3.4 region for the 1st November (bottom row) start dates over the period RMSE of simple persistence in dashed black.
All forecast systems beat persistence. Multi-model is the most skilful system, with highest reliability (RMSE~spread), in the first 6 months, while the perturbed parameter system is as good for longer lead times.
Multi-model (5 models, 45 members)
ECMWF Stochastic Physics (9 members)
Perturbed Parameters (9 members)

8. Stream 1 seasonal hindcasts
Brier skill score of the three different forecast systems for several regions, lead times (1-1, 2-4, 5-7 months), events (anomalies above/below the upper/lower tercile), start dates (May and November) and variables (T2m, precipitation, Z500 and MSLP) computed over the period The inset numbers indicate the number of cases where a system is superior with 95% confidence.
251
383 38 15 5
117

9. Stochastic physics-GPCP
Stream 1 seasonal hindcasts
Precipitation bias (DJF, 1-month lead time, )
The stochastic physics scheme reduces the tropical bias
Control-GPCP
Stochastic physics-GPCP

10. Decadal forecasts
Correlation of averaged Ts for years 6-10 from IfM-Kiel (ECHAM5/OM1) using nine 3-member ensembles hindcasts over the period
Dynamical Hindcast
Persistence
From N. Keenlyside (IfM)
Radiative Forcing Only
Analysis

11. Common archive Common variables (GRIB and NetCDF):
Atmosphere (daily and monthly): 5 pressure levels (850, 500, 200, 100, 50 hPa) Z,T,u,v,q and surface data.
Ocean (monthly): monthly means of 3D and 2D fields (including transports) for analyses and forecasts interpolated into common Levitus regular grid.
Model-level data for dynamical downscaling from 3 models.
Additional ocean/atmosphere variables from several models.
Definitions, units and encoding and archiving rules available from: ata/index.html

12. common data atmosphere common data atmosphere
Archiving and dissemination strategy
Hindcasts run/archived at ECMWF (access to member state users)
common data atmosphere
MARS
common data ocean
additional data
ECFS
ECMWF firewall
MARS client
OPeNDAP server
ENSEMBLES public data server (5 Tb)
common data atmosphere
common data ocean
Diagnostics and
downscaling
Climate Explorer &
Downscaling
portal

13. Public services
ENSEMBLES s2d is committed to provide the scientific community with full access to the data and with automatic tools to perform analysis.
The ENSEMBLES partners are working on:
A public data dissemination system at ECMWF.
Expanding the KNMI Climate Explorer for exploratory analysis.
Developing a downscaling web portal at the University of Cantabria.

14. Public data dissemination
THREDDS (OPeNDAP) aggregation server
Retrieve
monthly and
daily fields
GRIB, NetCDF
and plot data
Retrieve
ERA40

15. Exploratory analysis and downscaling
Climate Explorer
Exploratory analysis tool, including correlations and EOF analysis, forecast skill assessment and extreme event analysis (RCLIM).
Downscaling Web Portal
Statistical downscaling online using ENSEMBLES Stream 1, DEMETER and operational hindcasts.

16. Extremes in s2d: issues
Verification of extreme monthly/seasonal rare events has been “rare” (except for quintiles at the Met Office and 15th percentiles at ECMWF). Definition and interest is different from extremes in ACC and weather forecasting.
Not a thorough definition yet (depends on the time scale); focus on simple extremes and include users perspective.
Ensembles do not automatically provide a probabilistic prediction of an extreme event (model output is different to reality); a probabilistic model is necessary. Ensemble size may play an important role.
Look at the extreme events simulated by the models and the physical processes involved, along with a forecast quality assessment.

17. Need of a probabilistic model
December Niño3.4 observed and forecast (DEMETER, August start date, 5-month lead time) SST above the 85th percentile over
Probability forecasts made using a simple multi-model (single models corrected individually) and forecast assimilation
False alarms
From C. Primo and C. Coelho (Univ. of Reading)

18. A user strategy: the end-to-end approach
A broad range of forecast products might be offered, but a specific analysis of the user requirements is necessary.
End-to-end is based on collaboration and continuous feedback.
End users develop their models taking into account climate prediction limitations.
Users should use objective records of performance.
The final level of forecast quality that provides added value is defined by the application -> user-oriented verification. End users should assess the final value of the predictions.
Forecast reliability becomes a major issue.
A range of forecast products might be offered, but user requirements need to be clearly defined (Communication!). Meteorological products are often only evaluated by the clarity of their presentation but users do not use to be aware of objective records of performance. Information about the quality and the uncertainty of the forecasts is as important as the forecast itself. Quality estimates of the end-user products should improve the climate forecasts. There is no direct relationship between forecast quality and usefulness.

19. Predicting for users: end-to-end
…………
Climate forecast 1 2 3 4 62 63
………… 63 62 4 3 2 1
Downscaling 63
………… 62 4 3 2 1
Application
model
Forecast probability of T or PP
Forecasts probability of e.g. crop yield
non-linear transformation
The end-to-end approach was at the base of the DEMETER experiment. Using that approach a continuous flow of information took place between end users and climate modellers. For an efficient use of the climate information, both statistical and dynamical downscaling methods were developed trying to satisfy the end-user requirements. This strategy allowed to estimate the actual value of the seasonal predictions using a range of application models to predict crop yield and malaria incidence.

20. Downscaling for s2d predictions
Use both dynamical and empirical/statistical methods.
Correct systematic errors of global models and obtain reliable (statistical properties similar to the observed data) probabilistic predictions (with only relatively short, i.e., years, training samples).
Deal with full ensembles, not a deterministic prediction or the ensemble mean, maximising the benefit of limited simulations with regional models.
Consider both model and initial condition uncertainty.
Generate high-resolution (e.g., daily) time series of surface variables (using, e.g., weather generators with statistical methods).
Given the coarse resolution of global models, it is of paramount importance for users to properly downscale the model output. Downscaling of weather and climate forecasts is not intrinsically different from the downscaling of climate change experiments, if it is not for the possibility of verifying the quality of the results with actual data. This is possible because the forecasting horizon is nearer and because all forecast systems make available a set of forecasts for the past. However, the technical problems to deal with are not obvious, especially as far as the availability of data is concerned.

21. Downscaling for s2d predictions

22. Climate forecasts for malaria warning
Precipitation composites for the five years with the highest (top row) and lowest (bottom row) standardised malaria incidence for DJF DEMETER (left) and CMAP (right)
Quartiles define extreme events (outbreaks) for malaria prediction
Areas with epidemic malaria in Africa
Mean precipitation predicted by the multi-model DEMETER system over Africa in years when malaria was anomalously low or high. The seasonal forecasts (started in November, one month lead time) show a strong agreement with the reference dataset. Note the different colour scale, due to the forecasts being the ensemble mean of a 27-member ensemble. Epidemics of malaria are usually defined using quartiles of the long-term distribution of the incidence. Botswana is included inside the green circle. Note that is a region with epidemic (versus endemic) malaria, ie, the disease has interannual variability linked to climate.

23. Climate forecasts for malaria warning
Probabilistic predictions of standardised malaria incidence quartile categories in Botswana with five months lead time
Very low malaria
-- high malaria years
-- low malaria years
Available in March
Available in November
Very high malaria
ROC Score
Precipitation
Incidence
Event
DEMETER
CMAP
Very low
0.95
1.00
Very high
0.52
0.94
0.84
Probability forecasts for the years with the highest (red) and lowest (blue) malaria incidence over Botswana, obtained with the DEMETER multi-model precipitation. The forecast quality is estimated using the ROC score (skill is present when the score is above 0.5) and compared with the skill of the precipitation forecasts and of a system that predicts malaria using monitored precipitation (from CMAP). Note that the DEMETER incidence probability forecasts are available at the beginning of November (a few days after the coupled models have been initialized), while the system based in monitored precipitation has to wait until the end of the rainy season (beginning of March) to provide a skilful forecast. This work is described in Thomson, M.C., F.J. Doblas-Reyes, S.J. Mason, R. Hagedorn, S.J. Connor, T. Phindela, A.P. Morse and T.N. Palmer (2006). Malaria early warnings based on seasonal climate forecasts from multi-model ensembles. Nature, 439,

24. Link climate change and s2d prediction
Merging information from s2d and climate change simulations might allow to assess the reliability in climate change predictions (cf Tim Palmer’s talk).
Furthermore, the possibility of adaptation to climate change via a learning process taking place at the interannual time scale is an obvious way to achieve a high degree of integration (integration of time scales).
This implies:
That similar multi-model systems (or other systems that address model error) are used for climate forecasting and climate change simulations.
Involvement of both climate scientists and end-users, where both scientists and end users/stakeholders should consider the whole range of time scales. For instance, crop managers see the adaptation to long-term climate change as a process that takes place on a yearly basis.

25. Link climate change and s2d: an example
Regional climate change simulations can not be verified; model consensus has been used to estimate reliability. Can seasonal dynamical predictions estimate how reliable climate change estimates are in a seamless framework?
ENSEMBLES seasonal hindcasts.
IPCC AR4 multi-scenario climate change simulations.
Changes in precipitation are presumably due to:
Increased atmospheric water vapour.
Changes in circulation linked to SST anomalies of a few degrees.
Test whether consensus in multi-model climate change simulations can be considered as reliable predictions, using multi-model seasonal forecasts (for which reliability can be actually verified) to establish an additional metric.

26. Conclusions
ENSEMBLES’ main goal is to develop a seamless ensemble forecast system for seasonal, interannual and climate change forecasts and their impacts.
Innovative methods to deal with forecast uncertainty due to model error are being tested.
Forecast quality assessment is a complex multi-faceted task that can help in determining the relative merits of different forecast systems.
A set of public services (data dissemination, downscaling, exploratory analysis) for the research community has been developed.

27. Questions?