Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Improving COSMO-LEPS forecasts of extreme events with reforecasts F. Fundel, A. Walser, M. Liniger, C. Appenzeller

2 Improving CLEPS forecasts | COSMO GM |

3

4 How much is it going to rain? What is the probability of such an event to happen? Are there systematic model errors? Do model errors vary in space, time? Did the model ever forecast a such an event? Should a warning be given?

5 Improving CLEPS forecasts | COSMO GM | Model Obs 25. Jun. +-14d Why can reforecasts help to improve meteorological warnings?

6 Improving CLEPS forecasts | COSMO GM | Spatial variation of model bias Difference of CDF of observations and COSMO-LEPS 24h total precipitation 10/ /2006 Model too wet, worse in southern Switzerland

7 Improving CLEPS forecasts | COSMO GM | “However, the improved skill from calibration using large datasets is equivalent to the skill increases afforded by perhaps 5–10 yr of numerical modeling system development and model resolution increases.” (Wilks and Hamill, Mon. Wea. Rev. 2007) “Use of reforecasts improved probabilistic precipitation forecasts dramatically, aided the diagnosis of model biases, and provided enough forecast samples to answer some interesting questions about predictability in the forecast model.” (Hamill et. al, BAMS 2006) “…reforecast data sets may be particularly helpful in the improvement of probabilistic forecasts of the variables that are most directly relevant to many forecast users…” (Hamill and Whitaker, subm. to Mon. Wea. Rev 2006) Proven use of reforecasts

8 Improving CLEPS forecasts | COSMO GM | COSMO-LEPS Model Climatology Setup Reforecasts over a period of 30 years ( ) Deterministic run of COSMO-LEPS (1 member) (convective scheme = tiedtke) ERA40 Reanalysis as Initial/Boundary 42h lead time, 12:00 Initial time Calculated on hpce at ECMWF Archived on Mars at ECMWF (surf (30 parameters), 4 plev (8 parameters); 3h step) Post processing at CSCS Limitations Reforecasts with lead time of 42h are used to calibrate forecasts of up to 132h Only one convection scheme (COSMO-LEPS uses 2) New climatology needed with each model version change Building a climatology is slow and costly Currently only a monthly subset of the climatology is used for calibration (warning indices need to be interpreted with respect to the actual month)

9 Improving CLEPS forecasts | COSMO GM | x Model Climate Ensemble Forecast Calibrating an EPS

10 Improving CLEPS forecasts | COSMO GM | Extreme Forecast Index EFI (ECMWF) -1 1 EFI = -1 : All Forecast are below the climatology EFI = 1 : All Forecast are above the climatology F(p) p F(p) = proportion of EPS members below the p percentile

11 Improving CLEPS forecasts | COSMO GM | Extreme Forecast Index EFI (ECMWF) EFI for 24h total precipitation UTC – UTC UTC – UTC COSMO-LEPS ECMWF 0.8???

12 Improving CLEPS forecasts | COSMO GM | Extreme Forecast Index EFI (ECMWF) EFI properties (desired?) Combines properties of two CDFs in one number Forecast and climatology spread influence the EFI Ambiguous interpretation without further information EFI for varying forecast mean and standard deviation constant climatology with mean=0 and  =1

13 Improving CLEPS forecasts | COSMO GM | Approach: fit a distribution function to the model climate (e.g. Gamma for precipitation) find the return levels according to a given return period find the number of forecasts exceeding the return level of a given return period Advantages: calibrated forecast probabilistic forecast straight forward to interpret return periods are a often related to warning levels (favorably for forecasters) Limitation: Not applicable on extreme (rare) events Return Periods

14 Improving CLEPS forecasts | COSMO GM | New index Probability of Return Period exceedance PRP Dependent on the climatology used to calculate return levels/periods Here, a monthly subset of the climatology is used (e.g. only data from September ) PRP 1 = Event that happens once per September PRP 100 = Event that happens in one out of 100 Septembers

15 Improving CLEPS forecasts | COSMO GM | Probability of Return Period exceedance once in 6 Septembers once in 2 Septembers each Septemberstwice per September COSMO-PRP 1/2 COSMO-PRP 1 COSMO-PRP 2 COSMO-PRP 6

16 Improving CLEPS forecasts | COSMO GM | 24h total precipitation UTC VT: UTC – UTC Probability of Return Period exceedance EFI COSMO-PRP 2

17 Improving CLEPS forecasts | COSMO GM | PRP based Warngramms twice per September (15.8 mm/24h) once per September (21 mm/24h) once in 3 Septembers (26.3 mm/24h) once in 6 Septembers (34.8 mm/24h)

18 Improving CLEPS forecasts | COSMO GM | PRP with Extreme Value Analysis Extremal types Theorem: Maxima of a large number of independent random data of the same distribution function follow the Generalized Extreme Value distribution (GEV)  → 0 : Gumbel  > 0 : Frechet  < 0 : Weibull  =position;  =scale;  =shape C. Frei, Introduction to EVA

19 Improving CLEPS forecasts | COSMO GM | The underlying distribution function of extreme values y=x-u above a threshold u is the Generalized Pareto Distribution (GPD) (a special case of the GEV)  =scale;  =shape C. Frei, Introduction to EVA PRP with Extreme Value Analysis

20 Improving CLEPS forecasts | COSMO GM | Steps towards a GPD based probabilistic forecast of extreme events Find an eligible threshold for the detection of extreme events (97.5% percentile of the climatology) Fit the GPD to the found extreme values Calculate return levels for chosen return periods Find the proportion of forecast members exceeding a return level PRP with Extreme Value Analysis

21 Improving CLEPS forecasts | COSMO GM | Return Period [days] Return Level [mm/24h] GPD fit to extreme values (>97.5 %-ile i.e. top 25) of COSMO-LEPS 24h precipitation (1 grid point only) and 5%,95% confidence intervals PRP with Extreme Value Analysis

22 Improving CLEPS forecasts | COSMO GM | COSMO-PRP 2 COSMO-PRP 2 (GPD) PRP with Extreme Value Analysis

23 Improving CLEPS forecasts | COSMO GM | COSMO-PRP 60 (GPD)COSMO-PRP 12 (GPD) PRP with Extreme Value Analysis

24 Improving CLEPS forecasts | COSMO GM | Difficulties of GPD based warning products In case of precipitation very dry regions sometimes do not have enough days of precipitation (solution: extend reforecasts/mask regions) A low number of extreme events increases the uncertainty of the GPD fit (solution: extend reforecasts) Verification of extreme events is difficult due to the low number of events available. PRP with Extreme Value Analysis

25 Improving CLEPS forecasts | COSMO GM | Next Steps Extend the model climate used for calibration and extreme value statistics Probabilistic verification of the calibrated COSMO-LEPS forecast Translate model output to real atmospheric values

26 Improving CLEPS forecasts | COSMO GM | Conclusion A 30-years COSMO-LEPS climatology is about to being completed New probabilistic, calibrated forecasts of extreme events are in quasi operational use An objective verification is necessary Extreme events might only be verified with case studies Forecaster feedback is necessary