1. Deterministic vs Probabilistic Forecast J.P. Céron – Météo-France

2. The Predictability « a Thunderstorm will be observed next Sunday over the Toulouse « Météopole » between 15h and 16h »  Irrealistic, the confidence that one can have in this forecast is very low « a rainy system will cross the Toulouse region Sunday afternoon »  realistic, one can be quite confident in this forecast

3. The Predictability The predictability depends on : The scale of the forecasted phenomenum (Thunderstorm, Easterly Wave, Blocking situation, ENSO, …) The Range of the forecast (NowCasting, Short, Medium, Seasonnal, Climatic) Deterministic formulation  error or lost of informations Probabilistic formulation  more possibilities in the forecast but interpretation problems

4. The different Forecasts How do you play to Horse races? Informations about the form of horses, trainers, jockey, race conditions (type of soil, weather, …), predictions, … Synthesis then making a bet on horses (in a more or less subjective choice) Use of seasonnal forecasts. Informations about the states of atmosphere, continental surfaces, oceanic system and its evolutions, the different forecasts, … Synthesis and decision/action (i.e. make a bet on the real solution in a more or less objective way)

5. The different Forecasts The horse n°5 will win the race. It will rain 650 mm at Niamey during the next rainy season. Consequences : gain or lost depending of the success or not of the forecast.

6. The different Forecasts Deterministic forecast

7. The different Forecasts

8. Deterministic forecast

9. Limits of Numerical Forecasting The forecast errors can come from different part of the forexast system : ANALYSIS - errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions But also... Network density of surface observations Over the whole globe

10. Limits of Numerical Forecasting MODELS LIMITS - mesh, number of vertical levels (resolution of the model) - Equations, parameterisations - small scale phenomena (under the mesh or badly know) But also... etc... Equations are generally “simplifyed” and one calibrate “parametrisations” in the model, that is to say that one use data computed in an approximate form (even sometime as “constant”). ANALYSIS - errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions

11. Limits of Numerical Forecasting INTERPRETATION - Models generally provide “raw data” and consequently the interpretation is often difficult) And finally... MODELS LIMITS - mesh, number of vertical levels (resolution of the model) - Equations, parameterisations - small scale phenomena (under the mesh or badly know) ANALYSIS - errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions

12. Limits of Numerical Forecasting COMMUNICATION - misfitted vocabulary - misreading of users’ needs - dissemination without feedbacks from the user MODELS LIMITS - mesh, number of vertical levels (resolution of the model) - Equations, parameterisations - small scale phenomena (under the mesh or badly know) ANALYSIS - errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions INTERPRETATION - Models generally provide “raw data” and consequently the interpretation is often difficult)

13. Limits of Numerical Forecasting Natural variability of the Ocean/Atmosphere system and the Atmospheric respons to external forcing

14. Limits of Numerical Forecasting Uncertainties of the initial state of the Climatic system Modelisation Error (both Oceanic and Atmosphéric) Natural varibility of Atmosphere and its respons to external forcing Interpretation of the forecast To sample the initial state uncertainty  disturbances of the analysis To sample the model uncertainty  distubances of the model To sample in the forecast all the possible solutions of the Ocean/Atmosphere system

15. Limits of Numerical Forecasting The numerical forecast using "ARPEGE« model at short range is a « déterministic » forecast. It uses the initial state description of the climate system and, using model’s equation, allow to perform the time evolution of the atmospheric state. Major errors of this type of forecasts mainly come from initial state errors intoduced inside the model. The uncertainty spread increase with the range of the forecast. error at time t 0 (initial uncertainty) error at time t 0 + range

16. Limits of Numerical Forecasting To take into account the probability of the deterministic evolution, one perform several forecasts starting from the same initial time but using slightly modified values of the parameters of the simulation (namely inside the probable range of errors introduced at the initial time). Deterministic forecast At this range: Strong probability region Forecast range value of the parameter Initial time

17. Limits of Numerical Forecasting Because of computer ressources, one use a larger mesh model (to limit the computation time) and one perform, typically, around thirty forecast using different conditions for the model. 850 hPa Temperature plumes (Toulouse) base 21 september 1999 at 12h UTC 1 25 - 50% 6 - 25% 0 - 6% 50 - 75% 75 - 100% Deterministic model verification

18. Limits of Numerical Forecasting One can look at the parameter’s dispersion as a function of the time intergration and describe the encountered value distribution rath rather than the values themselves. One can also give a confidence indice with a more or less high value (the larger the dispersion of the forecasts, the lower the indice that is to say the lower the confidence). Temperature dispersion plumes as a function of the time integration 850 hPa Temperature plumes (Toulouse) base 21 september 1999 at 12h UTC 1 25 - 50% 6 - 25% 0 - 6% 50 - 75% 75 - 100% Deterministic model verification This method is named “Ensemble forecast”

19. Limits of Numerical Forecasting

20. The ensemble forecast (Resume) To sample the initial state uncertainty  disturbances of the analysis To sample the model uncertainty  distubances of the model To sample in the forecast all the possible solutions of the Ocean/Atmosphere system Several forecasts – trying to get the distribution of the possible solutions instead of a single value Several models – How can we merge the informations coming from many different models – empirical, AGCM, COAGCM ? Multimodel approach.

21. Limites of Statistical Forecasting Uncertainty already included inside the statistical tools

23. The different Forecasts The horse n°5 will win the race. It will rain 650 mm at Niamey during the next rainy season. The horse n°5 has good chances to be in the firsts in this race. The situation of the Ocean/Atmosphere system and its probable evolutions indicate that the next rainy season in Niger has a good probability to be « above the Normal ». Consequences : gain or lost depending of the success or not of the forecast

24. The different Forecasts Probabilistic Forecast

25. The different Forecasts Probabilistic Forecast

26. The different Forecasts Probabilistic Forecast

27. The different Forecasts Probabilistic Forecast

28. The different Forecasts Analogue technics

29. Probabilistic Forecast : formulation 1 model et n members p models et n members Gaussian : mean + standard deviation Analogues Statistical Methods (Discriminant Analysis, Multiple Regression, Probabilistic Regression, …)

30. Catégories Forecast Probabilistic Forecast

31. Catégories Forecast How to define the categories? Number Categories Limits Needs of user? How to evaluate the forecasted probabilities for each category? Frequency/Probability, Climatological Probabilities, Conditionnal probabilities, Confidence Indice Statistical Models Numerical Models How to transform the forecast in « readable and comprehensive » form for the user?

32. Quadratic Scores (Brier, RPS, …) Relative Operating Characteristic (FA vs ND) Cost/Lost ratio approach Deux categories: + dry / + wet et Ratio c/L=0.5 C1= averaged cost using a climatological forecast C2 = averaged cost using a perfect forecast C3= averaged cost using a the model forecast Probabilistic Forecast: verification obsnon obs prev cc non prev L0 obsnon obs prev n 11 n 12 non prev n 21 n 22

33. Brier Scores : BrSc = 1/N  (p i – o i ) 2 p i probabilité prévue pour l’événement oi variable indicatrice de l’observation de l’événement 1 BrSc=   o(p)  1-p  2 +  1-o(p)  p 2  g(p) dp 0 g(p) Fréquence relative avec laquelle l’événement est prévu avec une probabilité comprise entre p et p+dp 1 1 1 BrSc=  f  1- f  +   p  1-o(p)  2 -   f-o(p)  2  g(p) dp 0 0 0 Uncertainty Reliability Resolution Pour un système qui aurait toujours prévu la probabilité climatique d’occurrence ( p  f lt ) LCBrSc= f  1- f  + f  1- f lt  [L/S]CBrSkSc = 1 – BrSC / [L/S]CBrSc Brier Skill Score BSSREL = 1 – BSREL // [L/S]CBrSc Brier Reliability SC BSSRSL = BSRSL / Uncertainty Brier Resolution SC

34. Probabilistic Forecast: verification Reliability Diagrams

35. Probabilistic Forecast Cost/Lost ratio approach

36. Probabilistic Forecast Differents users

37. Probabilistic Forecast Comparison between Deterministic and Probabilistic formulation Probabiliste Déterministe

38. Probabilistic Forecast Resume