1. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Seasonal Forecasting From DEMETER to ENSEMBLES Francisco J. Doblas-Reyes ECMWF

2. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 ●Coupled ocean-atmosphere systems without assimilation of sub-surface ocean observations. ●Multi-model (ECMWF, GloSea, Météo-France, IfM-Kiel, CERFACS, INGV, LODYC) ensemble re-forecasts. ●Re-forecast period 1959-2001, seasonal (6 months, February, May, August and November start date), 9- member ensembles, ERA40 initialization in most cases. DEMETER

3. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 ●Model uncertainty is a major source of forecast error. Three approaches to deal with model uncertainty are being investigated in ENSEMBLES: multi-model (ECMWF, GloSea, DePreSys, Météo-France, IfM-Kiel, CERFACS, INGV), stochastic physics (ECMWF) and perturbed parameters (DePreSys). ●Hindcasts in two streams: o Stream 1: hindcast period 1991-2001, seasonal (7 months, May and November start date), annual (14 months, November start date), 9-member ensembles, ERA40 initialization in most cases; DePreSys (IC and PP ensembles) 10-year runs in every instance. o Stream 2: As in Stream 1 but over 1960-2005, with 4 start dates for seasonal hindcasts, at least 1 for annual and at least one 3- member decadal hindcast every 5 years; DePreSys 10-year runs once a year and 30-year runs every 5 years. ENSEMBLES

4. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Feb 87 May 87 Aug 87 Nov 87 Feb 88... K models x M ensemble members M*K-member ensemble Assume a multi-model ensemble system with coupled initialized GCMs Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 Ensemble climate forecast systems Lead time = 7

5. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Feb 87 May 87 Aug 87 Nov 87 Feb 88... Ensemble climate forecast systems Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 Assume a multi-model ensemble system with coupled initialized GCMs Lead time = 4

6. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Some definitions: o Forecast system: the forecast model (e.g., a coupled ocean/atmosphere model), the initialization (and ensemble generation) method and the statistical model that creates the forecast probabilities (an ensemble is not a probability forecast). o Forecast quality: Statistical description of how well the forecasts match the observations; it has multiple attributes. Important in probability forecast systems: o Systematic error: difference in the statistical properties of the forecast and the reference distribution. o Accuracy: some sort of distance between forecasts and observations. It is used as an indication of the association, the discrimination or resolution of the forecasts. o Reliability: A measure of trustworthiness (not accuracy). Error and accuracy

7. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Main systematic errors in dynamical climate forecasts: o Differences between the model climatological pdf (computed for a lead time from all start dates and ensemble members) and the reference climatological pdf (for the corresponding times of the reference dataset): systematic errors in mean and variability. o Conditional biases in the forecast pdf: errors in conditional probabilities implying that probability forecasts are not trustworthy. This type of systematic error is best assessed using the reliability diagram. Temperature Differences in climatological pdfs Reference pdfModel pdf Systematic error in ensemble forecasts Threshold Forecast PDF t=1 t=2 t=3 Mean bias Different variabilities Actual occurrences

8. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 ENSEMBLES Stream 2 T2m mean bias wrt ERA40/OPS, 1960-2005 First month May Months 2-4 JJA Months 5-7 SON Systematic error in seasonal forecasts ECMWF Météo-France

9. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 ENSEMBLES Stream 2 T2m ratio of standard deviation wrt ERA40/OPS ECMWF Météo-France Systematic error in seasonal forecasts First month May Months 2-4 JJA Months 5-7 SON

10. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 ENSEMBLES Stream 2 precip. mean bias wrt GPCP, 1979-2005 ECMWF Météo-France Systematic error in seasonal forecasts First month May Months 2-4 JJA Months 5-7 SON

11. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 ENSEMBLES Stream 2 precipitation ratio of standard deviation wrt GPCP ECMWF Météo-France Systematic error in seasonal hindcasts First month May Months 2-4 JJA Months 5-7 SON

12. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Lower quintile of DJF T2m for ERA-40/OPS (top row) and ECMWF ENSEMBLES Stream 2 (bottom row) in 1960-2005 (temperatures in °C) ERA/OPS Systematic error in seasonal forecasts ECMWF Stream 2

13. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Climatological PDF of DJF T2m for ERA-40/OPS and ECMWF ENSEMBLES Stream 2 computed over the period 1960-2005 Systematic error: climatological pdf Equatorial Pacific (0, 180°) For deterministic forecasts, compute anomalies with respect to the corresponding mean For probabilistic forecasts, compute probabilities with respect to hindcast and reference thresholds (terciles) Central Europe (50°N, 10°E)

14. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stratospheric winds in seasonal forecasts Daily evolution of the zonal mean of the 50-hPa zonal wind for the ENSEMBLES Stream 1 hindcasts with November start date (the horizontal axis runs from the 1 st of November to the 31 st of May). Results are averaged over 1991-2001. Black lines are for the ERA40 climatology (with shaded region covering ±σ). Every forecast system underestimates the mean wind. The same happens with the intraseasonal variability that prevents stratospheric sudden warming events from happening with the correct frequency and amplitude. From Amanda Maycock (Univ. of Reading)

15. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Blocking frequency in seasonal forecasts Northern Hemisphere DJF blocking frequency for 1-month lead 9-member ensemble seasonal hindcasts for DEMETER (left) and ENSEMBLES Stream 1 (right) over the period 1991-2001. Results are for the Tibaldi and Molteni index (reversal of the meridional gradient of Z500). The dots on top of each panel show the longitudes where the mean frequency of an experiment is significantly equal with 95% confidence to the mean frequency in ERA40. DEMETER ERA40 ECMWF CNRM Met Office LODYC CERFACS INGV MPI ENSEMBLES MM Stream 1 ERA40 ECMWF Glosea CNRM IfM DePreSys

16. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Anomalies: ENSO forecasts

17. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 ECMWF EUROSIP Analysis T2m forecasts for MAM 2009

18. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Reliability of verifiable predictions Users might have problems when dealing with probability forecasts, e.g., what is the meaning of “ 70% chance of rain ” ? Assuming that we agree on what “ rain ” means and that a set of past predictions exist, let us interpret it as “ It will rain 70% of the times the forecast system predicts the event with 70% probability ”. In this case, the user can trust the forecast. What happens if it actually rains only 50% of the times the probability forecast is 70%? The user can not trust the predictions, which are not reliable by being overconfident. This sort of information is essential for the user, especially if the user employs this forecast information in another model. When reliability information is not available, cognitive illusions (Nicholls, 1999) are favoured.

19. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Sea surface temperature RMSE (solid) and spread (dashed) averaged over the Niño3.4 region for the ENSEMBLES Stream 1hindcasts of the 1 st November start dates over the period 1991-2001. Persistence RMSE in dashed black. Multi-model (5 models, 45 members) ECMWF Stochastic Physics (9 members) Perturbed Parameters (9 members) Ensemble-mean error vs spread All forecast systems beat persistence. The multi-model is the most skilful system, with highest deterministic reliability (RMSE~spread), in the first 6 months, while perturbed parameters is as good for longer lead times.

20. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Attributes diagrams for near-surface temperature in Niño3.4 of ENSEMBLES Stream 1 multi-model over 1991-2001. Multi-model matches RMSE and spread, but gives unreliable probabilities: matching RMSE and spread does not imply reliable probability forecasts. RMSE-spread and probabilistic reliability 1-month lead DJF 3-month lead FMA

21. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 ●Reliability is a property of a forecast system that measures the ability to issue trustworthy probabilities oA probabilistic model is required to go from the ensemble members to probability forecasts (using e.g., probabilities as the proportion of ensemble members beyond a threshold, dressing). oForecast reliability measures the conditional biases of the forecast probabilities and is part of the systematic error of the forecast system. oReliability for dichotomous events is best illustrated in an attributes diagram (made of a reliability diagram and a forecast probability histogram). Reliability in probability forecasts Threshold Forecast PDF Actual occurrences

22. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Overconfident Perfect reliability Reliability diagram Forecast reliability measures conditional biases of forecast probabilities; example for event “values above the median” Climatological frequency

23. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Forecast reliability measures conditional biases of forecast probabilities Reliability (the smaller, the better) Reliability diagram Overconfident Perfect reliability

24. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Reliability score (the smaller, the better) Resolution score (the bigger, the better) Reliability diagram Overconfident Perfect reliability

25. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Attributes diagrams for 1-month lead seasonal (JJA) precipitation above the upper tercile over the tropical band for the ENSEMBLES Stream 1 multi-model (left, 45 members), stochastic physics (centre, 9 members) and perturbed parameters (right, 9 members) hindcasts started in May over the period 1991-2001 verified against GPCP. The Brier and ROC skill scores, along with 95% confidence intervals (in brackets) computed using a bootstrap method, are shown on top of each panel. Multi-model 0.129 (0.082,0.178) 0.441 (0.378,0.502) Stochastic physics 0.059 (0.005,0.105) 0.391 (0.322,0.453) Perturbed parameters 0.050 (0.002,0.105) 0.381 (0.329,0.443) Attributes diagram

26. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Reliability diagrams for 1-month lead seasonal (JJA) precipitation above the upper tercile over the tropical band for the ENSEMBLES Stream 1 multi-model (left, 45 members), stochastic physics (centre, 9 members) and perturbed parameters (right, 9 members) hindcasts over the period 1991-2001 verified against GPCP. Direct model output (no bias correction) and threshold (upper tercile) computed from the reference climatology Multi-model Stochastic physicsPerturbed parameters Direct model output

27. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Scores for southern South America precipitation from Stochastic Physics, Perturbed Parameters (both with 9- member ensembles) and Multi-model (5 models, 45 members). Sample values are shown with black dots along with 95% confidence intervals obtained using a bootstrap method (verified against GPCP over 1991-2001). Anomaly correlation coefficient Ratio between spread and RMSE ROCSS for anomalies above the upper tercile Stream 1 seasonal hindcasts

28. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Scores for Mediterranean precipitation from Stochastic Physics, Perturbed Parameters (both with 9-member ensembles) and Multi-model (5 models, 45 members). Sample values are shown with black dots along with 95% confidence intervals obtained using a bootstrap method (verified against GPCP over 1991-2001). Anomaly correlation coefficient Ratio between spread and RMSE ROCSS for anomalies above the upper tercile Stream 1 seasonal hindcasts

29. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 1 seasonal hindcasts 130 27 380 4 27 217 Anomaly correlation coefficient of the three different forecast systems for several regions, lead times (1-1, 1-3, 2-4, 3-5, 4-6 months), start dates (May and November) and variables (near-surface temperature, precipitation, Z500, T850 and MSLP) computed over the period 1991- 2001. The inset numbers indicate the number of cases where a system is superior with 95% confidence.

30. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 1 seasonal hindcasts 500 111 242 277 513 149 Standardized spread of the three different forecast systems for several regions, lead times (1-1, 1-3, 2-4, 3-5, 4-6 months), start dates (May and November) and variables (near-surface temperature, precipitation, Z500, T850 and MSLP) computed over the period 1991-2001. The inset numbers indicate the number of cases where a system is superior with 95% confidence.

31. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 1 seasonal hindcasts 61 171 162 125 75 193 Ratio of spread/RMSE of the three different forecast systems for several regions, lead times (1-1, 1-3, 2-4, 3-5, 4-6 months), start dates (May and November) and variables (near-surface temperature, precipitation, Z500, T850 and MSLP) computed over the period 1991-2001. The inset numbers indicate the number of cases where a system is superior with 95% confidence.

32. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Brier skill score of the three different forecast systems for several regions, lead times (1-1, 1-3, 2-4, 3-5, 4-6 months), events (anomalies above/below the upper/lower tercile and above the median), start dates (May and November) and variables (near-surface temperature, precipitation, Z500, T850 and MSLP) computed over the period 1991- 2001. The inset numbers indicate the number of cases where a system is superior with 95% confidence. Stream 1 seasonal hindcasts 710 34 1369 3 34 458

33. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Brier skill score for infinite ensemble size of the three different forecast systems for several regions, lead times (1-1, 1-3, 2-4, 3-5, 4-6 months), events (anomalies above/below the upper/lower tercile and above the median), start dates (May and November) and variables (near-surface temperature, precipitation, Z500, T850 and MSLP) computed over the period 1991-2001. The inset numbers indicate the number of cases where a system is superior with 95% confidence. Stream 1 seasonal hindcasts 243 132 520 49 39 459

34. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 1 seasonal hindcasts 521 8 952 0 31 315 Reliability skill score of the three different forecast systems for several regions, lead times (1-1, 1-3, 2-4, 3-5, 4-6 months), events (anomalies above/below the upper/lower tercile and above the median), start dates (May and November) and variables (near-surface temperature, precipitation, Z500, T850 and MSLP) computed over the period 1991- 2001. The inset numbers indicate the number of cases where a system is superior with 95% confidence.

35. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 1 seasonal hindcasts 379 62 907 8 26 404 ROC skill score of the three different forecast systems for several regions, lead times (1-1, 1-3, 2-4, 3-5, 4-6 months), events (anomalies above/below the upper/lower tercile and above the median), start dates (May and November) and variables (near-surface temperature, precipitation, Z500, T850 and MSLP) computed over the period 1991- 2001. The inset numbers indicate the number of cases where a system is superior with 95% confidence.

36. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Sea surface temperature RMSE (solid) and spread (dashed) averaged over the Niño3.4 region for the 1 st November (bottom row) start dates over the period 1991-2001. RMSE of simple persistence in dashed black. Multi-model (5 models, 45 members) ECMWF Stochastic Physics (9 members) Perturbed Parameters (9 members) Stream 1 annual hindcasts All forecast systems beat persistence. Multi-model is the most skilful system, with highest reliability (RMSE~spread), in the first 6 months, while the perturbed parameter system is as good for longer lead times.

37. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Reliability diagrams for nine-month lead seasonal (ASO) precipitation above the upper tercile over the tropical band for the ENSEMBLES Stream 1 multi-model (left, 45 members), stochastic physics (centre, 9 members) and perturbed parameters (right, 9 members) hindcasts started in May over the period 1991-2001 verified against ERA40. The Brier and ROC skill scores, along with 95% confidence intervals (in brackets) computed using a bootstrap method, are shown on top of each panel. Multi-model 0.004 (-0.123,0.115) 0.237 (0.052,0.402) Stochastic physics -0.156 (-0.290,-0.030) 0.162 (0.013,0.334) Perturbed parameters 0.008 (-0.121,0.115) 0.286 (0.125,0.430) Stream 1 annual hindcasts

38. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 1 annual hindcasts 44 1 38 8 29 15 Anomaly correlation coefficient of the three different forecast systems for several regions, lead times (6-8, 8-10, 10-12 months), events (anomalies above/below the upper/lower tercile and above the median) and variables (T2m, precipitation, Z500, T850 and MSLP) computed over the period 1991-2001 for the November start dates. The inset numbers indicate the number of cases where a system is superior with 95% confidence.

39. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 1 annual hindcasts 131 53 35 173 171 38 Standardized spread of the three different forecast systems for several regions, lead times (6-8, 8-10, 10-12 months), events (anomalies above/below the upper/lower tercile and above the median) and variables (T2m, precipitation, Z500, T850 and MSLP) computed over the period 1991-2001 for the November start dates. The inset numbers indicate the number of cases where a system is superior with 95% confidence.

40. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 1 annual hindcasts 16 67 76 17 11 92 Ratio spread/RMSE of the three different forecast systems for several regions, lead times (6-8, 8-10, 10-12 months), events (anomalies above/below the upper/lower tercile and above the median) and variables (T2m, precipitation, Z500, T850 and MSLP) computed over the period 1991-2001 for the November start dates. The inset numbers indicate the number of cases where a system is superior with 95% confidence.

41. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 1 annual hindcasts 377 1 205 3 92 15 Brier skill score of the three different forecast systems for several regions, lead times (6-8, 8-10, 10-12 months), events (anomalies above/below the upper/lower tercile and above the median) and variables (T2m, precipitation, Z500, T850 and MSLP) computed over the period 1991-2001 for the November start dates. The inset numbers indicate the number of cases where a system is superior with 95% confidence.

42. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 1 annual hindcasts 70 22 15 82 95 16 Brier skill score for infinite ensembles of the three different forecast systems for several regions, lead times (6-8, 8-10, 10-12 months), events (anomalies above/below the upper/lower tercile and above the median) and variables (T2m, precipitation, Z500, T850 and MSLP) computed over the period 1991-2001 for the November start dates. The inset numbers indicate the number of cases where a system is superior with 95% confidence.

43. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 1 annual hindcasts 267 1 180 4 76 10 Reliability Skill Score of the three different forecast systems for several regions, lead times (6-8, 8-10, 10-12 months), events (anomalies above/below the upper/lower tercile and above the median) and variables (T2m, precipitation, Z500, T850 and MSLP) computed over the period 1991-2001 for the November start dates. The inset numbers indicate the number of cases where a system is superior with 95% confidence.

44. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 1 annual hindcasts 127 9 68 18 64 21 ROC skill score of the three different forecast systems for several regions, lead times (6-8, 8-10, 10-12 months), events (anomalies above/below the upper/lower tercile and above the median) and variables (T2m, precipitation, Z500, T850 and MSLP) computed over the period 1991-2001 for the November start dates. The inset numbers indicate the number of cases where a system is superior with 95% confidence.

45. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Scores for Northern extratropics T2m (left) and tropical band precipitation (right) from ECMWF, Météo-France, INGV, IfM, Met Office, Perturbed Parameters, reduced multi-model (nine-member ensembles) and multi- model (5 models, 45 members). Temperature verified with ERA40/ERA-Int over 1960-2005 and precipitation verified against GPCP over 1980-2005. Stream 2 seasonal hindcasts Ensemble-mean anomaly correlation Ratio between spread and RMSE RelSS for anomalies above the upper tercile

46. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 2 seasonal hindcasts 10 101 56 Anomaly correlation coefficient of the two different forecast systems for several regions (Northern Hemisphere, Tropics, Southern Hemisphere), lead times (2-4, 5-7 months), start dates (Feb, May, Aug and Nov) and variables (near-surface temperature, precipitation, Z500, T850 and MSLP) computed over the period 1960-2005. The inset numbers indicate the number of cases where a system is superior. 64

47. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 2 seasonal hindcasts 47 73 48 Standardized spread of the two different forecast systems for several regions (Northern Hemisphere, Tropics, Southern Hemisphere), lead times (2-4, 5-7 months), start dates (Feb, May, Aug and Nov) and variables (near-surface temperature, precipitation, Z500, T850 and MSLP) computed over the period 1960-2005. The inset numbers indicate the number of cases where a system is superior. 72

48. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 2 seasonal hindcasts 54 66 64 Ratio spread/RMSE of the two different forecast systems for several regions (Northern Hemisphere, Tropics, Southern Hemisphere), lead times (2-4, 5-7 months), start dates (Feb, May, Aug and Nov) and variables (near-surface temperature, precipitation, Z500, T850 and MSLP) computed over the period 1960-2005. The inset numbers indicate the number of cases where a system is superior. 56

49. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 2 seasonal hindcasts 3 237 64 Brier skill score for several regions (Northern Hemisphere, Tropics, Southern Hemisphere), events (anomalies above/below the upper/lower tercile), lead times (2-4, 5-7 months), start dates (Feb, May, Aug and Nov) and variables (near-surface temperature, precipitation, Z500, T850 and MSLP) computed over the period 1960-2005. The inset numbers indicate the number of cases where a system is superior. 176

50. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 2 seasonal hindcasts 123 117 64 Debiased Brier skill score for several regions (Northern Hemisphere, Tropics, Southern Hemisphere), events (anomalies above/below the upper/lower tercile), lead times (2-4, 5-7 months), start dates (Feb, May, Aug and Nov) and variables (near-surface temperature, precip, Z500, T850 and MSLP) computed over the period 1960-2005. The inset numbers indicate the number of cases where a system is superior. 176

51. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 2 seasonal hindcasts 65 175 56 Brier skill score (infinite ensemble size) for several regions (Northern Hemisphere, Tropics, Southern Hemisphere), events (anomalies above/below the upper/lower tercile), lead times (2-4, 5-7 months), start dates (Feb, May, Aug and Nov) and variables (near-surface temperature, precip, Z500, T850 and MSLP) computed over the period 1960-2005. The inset numbers indicate the number of cases where a system is superior. 184

52. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 2 seasonal hindcasts 0 240 40 Reliability skill score for several regions (Northern Hemisphere, Tropics, Southern Hemisphere), events (anomalies above/below the upper/lower tercile), lead times (2-4, 5-7 months), start dates (Feb, May, Aug and Nov) and variables (near-surface temperature, precip, Z500, T850 and MSLP) computed over the period 1960-2005. The inset numbers indicate the number of cases where a system is superior. 200

53. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Stream 2 seasonal hindcasts 20 220 124 ROC skill score for several regions (Northern Hemisphere, Tropics, Southern Hemisphere), events (anomalies above/below the upper/lower tercile), lead times (2-4, 5-7 months), start dates (Feb, May, Aug and Nov) and variables (near-surface temperature, precip, Z500, T850 and MSLP) computed over the period 1960-2005. The inset numbers indicate the number of cases where a system is superior. 116

54. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 DEMETER vs ENSEMBLES Brier skill score for Niño3 SST re-forecasts in DEMETER, ENSEMBLES and DEMETER+ENSEMBLES using all start dates over the period 1980-2001. Forecast period 2-4 months Forecast period 4-6 months x’>upper tercilex’<lower tercile

55. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 DEMETER vs ENSEMBLES Brier skill score for re-forecasts of near-surface temperature and precipitation for different land regions over the period 1980-2001. DEMETER ENSEMBLES DEMETER+ENSEMBLES

56. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 EUROBRISA at IC3 ●The Catalan Institute for Climate Sciences (IC3, Barcelona, Spain) will start this winter a new group on seasonal and interannual climate forecasting. ●The main goals are to develop a capability to perform research on climate forecasting and to work on methods that provide useful climate information; the target regions are the Mediterranean area, South America and Africa. ●A solid link to EUROBRISA is expected at IC3. Two members of the group will work on seasonal forecasting calibration and combination with a focus on the Mediterranean region and the calibration of forecasts using non-stationary series (in the presence of trends and low- frequency variability).

57. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009 Summary ●Substantial systematic error, including lack of reliability, is still a fundamental problem in dynamical seasonal and interannual forecasting and forces a posteriori corrections to obtain useful predictions. ●Comprehensive assessments of the forecast quality measures (including estimates of their standard error) are indispensable in forecast system comparisons. ●Perturbed-parameter ensembles are competitive with multi-model ensembles. ●The ENSEMBLES multi-model is marginally better than the DEMETER multi-model; much is still to be gained from robust calibration and combination.

58. Seasonal forecasting from DEMETER to ENSEMBLES21 July 2009