Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss WG4 Activities Priority project « Advanced interpretation and verification of very high resolution models » Pierre Eckert MeteoSwiss, Geneva WG4 coordinator

2 WG4 reporting Gust diagnostics Jan-Peter Schulz Deutscher Wetterdienst

3 Diagnosing turbulent gusts In the COSMO model the maximum gusts at 10 m above the ground are estimated from the absolute speed of the near-surface mean wind V m and its standard deviation σ : α = 3 : Tuning parameter : Friction velocity C D : Drag coefficient for momentum following Panofsky and Dutton (1984)

4 Diagnosing turbulent gusts In the original version of the COSMO model as of 1999 (35 levels, lowest one about 30 m above the ground) the absolute mean wind speed at the lowest level was taken for V m in the formula above. When introducing the 40 vertical levels in 2005 (lowest one about 10 m above the ground) the formulation was adapted, in order to keep the tuning, by interpolating V m at 30 m from the two lowest model levels (while computing the friction velocity by definition from the speed at the lowest model level). This formulation leads to the overestimation of the gusts. In the new version the wind speed at 10 m above the ground is taken for V m, leading to more realistic gust estimates. α is kept at a value of 3.

5 Verification Period: Jan Mean Gust (observed) [m/s] Mean Gust/Mean Wind (observed) Mean Wind [m/s] x old + new

6 WG4 reporting Gust diagnostics Recommandation WG4 recommends that the formulation of wind gusts of the COMSO reference version is adapted so that the gusts are reduced. Could be affected by the choice of the vertical discretisation  Poster

Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Thunderstorm Prediction with Boosting: Verification and Implementation of a new Base Classifier André Walser (MeteoSwiss) Martin Kohli (ETH Zürich, Semester Thesis)

8 Andre Walser Output of the Learn process M base classifier Threshold classifier:

9 Andre Walser AdaBoost Algorithm Input Weighted learn samples Number of base classifier M Iteration 1 determine base classifier G 2 calculate error, weights w 3 adapt the weights of falsely classified samples Classifier:

10 Andre Walser C_TSTORM MAPS 17 UTC 18 UTC 19 UTC

11 Andre Walser July 2006 ~7% events Random forecast

The COSMO-LEPS system: getting close to the 5-year milestone Andrea Montani, Chiara Marsigli and Tiziana Paccagnella ARPA-SIM Hydrometeorological service of Emilia-Romagna, Italy IX General COSMO meeting Athens,18-21 September 2007

The new COSMO-LEPS ECMWF since February 2006 d-1 dd+5 d+1d+2 d+4d+3 older EPS younger EPS clustering period Cluster Analysis and RM identification 4 variables Z U V Q 3 levels hPa 2 time steps Cluster Analysis and RM identification European area Complete Linkage COSMO- LEPS Integratio n Domain 16 Representative Members driving the 16 COSMO-model integrations (weighted according to the cluster populations) employing either Tiedtke or Kain- Fristch convection scheme (randomly choosen) COSMO- LEPS clustering area suite running as a ”time- critical application” managed by ARPA-SIM; Δx ~ 10 km; 40 ML; COSM0-LM 3.20 since Nov06; fc length: 132h; Computer time (4.3 million BU for 2007) provided by the COSMO partners which are ECMWF member states.

Dissemination  probabilistic products  deterministic products (individual COSMO-LEPS runs)  derived probability products (EM, ES)  meteograms over station points products delivered at about 1UTC to the COSMO weather services, to Hungary (case studies) and to the MAP D- PHASE and COPS communities (field campaign).

Time series of Brier Skill Score  BSS is written as 1-BS/BS ref. Sample climate is the reference system. Useful forecast systems if BSS > 0.  BS measures the mean squared difference between forecast and observation in probability space.  Equivalent to MSE for deterministic forecast. BSS  improvement of performance detectable for all thresholds along the years;  still problems with high thresholds, but good trend in fc step: 30-42h Jun04: 5m  10m Feb06: 10m  16m; 32ML  40 ML

Main results COSMO-LEPS system runs on a daily basis since November 2002 (6 “failures” in almost 5 years of activity) and it has become a “member-state time-critical application” at ECMWF (  ECMWF operators involved in the suite monitoring). COSMO-LEPS products used in EC Projects (e.g. PREVIEW), field campaigns (e.g. COPS, MAP D-PHASE) and met-ops rooms across COSMO community. Nevertheless, positive trends can be identified: increase in ROC area scores and reduction in outliers percentages; positive impact of increasing the population from 5 to 10 members (June 2004); although some deficiency in the skill of the system were identified after the system upgrades occurred on February 2006 (from 10 to 16 members; from 32 to 40 model levels + EPS upgrade!!!), scores are encouraging throughout Time series scores cannot easily disentangle improvements related to COSMO-LEPS itself from those due to better boundaries by ECMWF EPS. 2 more features: marked semi-diurnal cycle in COSMO-LEPS scores (better skill for “night-time” forecasts); better scores over the Alpine area rather than over the full domain (to be confirmed).

Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Improving COSMO-LEPS forecasts of extreme events with reforecasts F. Fundel, A. Walser, M. Liniger, C. Appenzeller  Poster

18 Improving CLEPS forecasts | COSMO GM | Model Obs 25. Jun. +-14d Why can reforecasts help to improve meteorological warnings?

19 Improving CLEPS forecasts | COSMO GM | Spatial variation of model bias Difference of CDF of observations and COSMO-LEPS 24h total precipitation 10/ /2006 Model too wet, worse in southern Switzerland

20 Improving CLEPS forecasts | COSMO GM | COSMO-LEPS Model Climatology Setup Reforecasts over a period of 30 years ( ) Deterministic run of COSMO-LEPS (1 member) (convective scheme = tiedtke) ERA40 Reanalysis as Initial/Boundary 42h lead time, 12:00 Initial time Calculated on hpce at ECMWF Archived on Mars at ECMWF (surf (30 parameters), 4 plev (8 parameters); 3h step) Post processing at CSCS

21 Improving CLEPS forecasts | COSMO GM | x Model Climate Ensemble Forecast Calibrating an EPS

22 Improving CLEPS forecasts | COSMO GM | New index Probability of Return Period exceedance PRP Dependent on the climatology used to calculate return levels/periods Here, a monthly subset of the climatology is used (e.g. only data from September ) PRP 1 = Event that happens once per September PRP 100 = Event that happens in one out of 100 Septembers

23 Improving CLEPS forecasts | COSMO GM | Probability of Return Period exceedance once in 6 Septembers once in 2 Septembers each Septemberstwice per September COSMO-PRP 1/2 COSMO-PRP 1 COSMO-PRP 2 COSMO-PRP 6

24 Improving CLEPS forecasts | COSMO GM | PRP based Warngramms twice per September (15.8 mm/24h) once per September (21 mm/24h) once in 3 Septembers (26.3 mm/24h) once in 6 Septembers (34.8 mm/24h)

25 Improving CLEPS forecasts | COSMO GM | The underlying distribution function of extreme values y=x-u above a threshold u is the Generalized Pareto Distribution (GPD) (a special case of the GEV)  =scale;  =shape C. Frei, Introduction to EVA PRP with Extreme Value Analysis

26 Improving CLEPS forecasts | COSMO GM | COSMO-PRP 60 (GPD)COSMO-PRP 12 (GPD) PRP with Extreme Value Analysis

Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Priority project « Verification of very high resolution models » Slides from Felix Ament  Poster Ulrich Damrath Carlo Cacciamani Pirmin Kaufmann  Poster

28 WG4 reporting Which rain forecast would you rather use? Mesoscale model (5 km) 21 Mar 2004 Sydney Global model (100 km) 21 Mar 2004 Sydney Motivation for new scores Observed 24h rain RMS=13.0 RMS=4.6

29 WG4 reporting Fine scale verification: Fuzzy Methods observation forecast xxx xx xxx x XX XX XX xX X X x x x Intensity Scale Evaluate box statistics (Choose a threshold to define event and non-event) define scales of interest consider statistics at these scales for verification General Recipe “… do not evaluate a point by point match!”  score depends on spatial scale and intensity

30 WG4 reporting A Fuzzy Verification Toolbox Ebert, E.E., 2007: Fuzzy verification of high resolution gridded forecasts: A review and proposed framework. Meteorol. Appls., submitted. Toolbox available at Fuzzy methodDecision model for useful forecast Upscaling (Zepeda-Arce et al. 2000; Weygandt et al. 2004)Resembles obs when averaged to coarser scales Anywhere in window (Damrath 2004), 50% coveragePredicts event over minimum fraction of region Fuzzy logic (Damrath 2004), Joint probability (Ebert 2002)More correct than incorrect Multi-event contingency table (Atger 2001)Predicts at least one event close to observed event Intensity-scale (Casati et al. 2004)Lower error than random arrangement of obs Fractions skill score (Roberts and Lean 2005)Similar frequency of forecast and observed events Practically perfect hindcast (Brooks et al. 1998)Resembles forecast based on perfect knowledge of observations Pragmatic (Theis et al. 2005)Can distinguish events and non-events CSRR (Germann and Zawadzki 2004)High probability of matching observed value Area-related RMSE (Rezacova et al. 2005)Similar intensity distribution as observed

31 WG4 reporting A Fuzzy Verification testbed Perturbation Generator Analyzer Fuzzy Verification Toolbox Virtual truth (Radar data, model data, synthetic field) Realizations of virtual erroneous model forecasts Realizations of verification results Assessment of sensitivity (mean) [reliability (STD)] Two ingredients: 1.Reference fields: Hourly radar derived rain fields, August 2005 flood event, 19 time stamps (Frei et al., 2005) 2.Perturbations:  next slide

32 WG4 reporting Perturbations PerturbationType of forecast errorAlgorithm PERFECTNo error – perfect forecast!- XSHIFTHorizontal translation Horizontal translation (10 grid points) BROWNIANNo small scale skill Random exchange of neighboring points (Brownian motion) LS_NOISEWrong large scale forcing Multiplication with a disturbance factor generated by large scale 2d Gaussian kernels. SMOOTH High horizontal diffusion (or coarse scale model) Moving window arithmetic average DRIZZLE Overestimation of low intensity precipitation Moving Window filter setting each point below average point to the mean value

33 WG4 reporting Perfect forecast All scores should equal ! But, in fact, 5 out of 12 do not!

34 WG4 reporting Expected response to perturbations XSHIFTBROWNIANLS_NOISESMOOTHDRIZZLE Sensitivity: expected (=0.0); not expected (=1.0) Contrast := mean( ) – mean( ) Summary in terms of contrast: low high intensity coarse fine spatial scale

35 WG4 reporting Summary real Up- scaling Any- where in Window 50% cover- age Fuzzy Logig Joint Prob. Multi event cont. tab. Intensity Scale Fraction Skill Score Prag- matic Appr. Practic. Perf. Hindcast CSSR Area related RMSE Leaking Scores XSHIFT BROWNIANSMOOTH LS_NOISEDRIZZLE Contrast STD good Leaking scores show an overall poor performance “Intensity scale” and “Practically Perfect Hindcast” perform in general well, but … Many score have problem to detect large scale noise (LS_NOISE); “Upscaling” and “50% coverage” are beneficial in this respect Leaking scores show an overall poor performance “Intensity scale” and “Practically Perfect Hindcast” perform in general well, but … Many score have problem to detect large scale noise (LS_NOISE); “Upscaling” and “50% coverage” are beneficial in this respect

36 WG4 reporting August 2005 flood event Precipitation sum : Mean: 106.2mm Mean: 43.2mm Mean: 73.1mm Mean: 62.8mm (Hourly radar data calibrated using rain gauges (Frei et al., 2005))

37 WG4 reporting Fuzzy Verification of August 2005 flood Based on 3 hourly accumulations during August 2005 flood period ( ) bad good Intensity threshold (mm/3h) Scale (7km gridpoints) COSMO-7 COSMO-2

38 WG4 reporting Fuzzy Verification of August 2005 flood COSMO-7 better COSMO-2 better neutral Difference of Fuzzy Scores Intensity threshold (mm/3h) Scale (7km gridpoints)

39 WG4 reporting D-PHASE: August 2007 Intensity Scale score (preliminary), 3h accumulation COSMO-7 COSMO-2 COSMO-DE COSMO-EU

U. Damrath, COSMO GM, Athens „Fuzzy“-type verification for 12 h forecasts (vv=06 till vv=18) starting at 00 UTC August 2007 (fraction skill score)

In box of different size (what is the best size ?) alert warning areas (Emilia-Romagna) First simple approach: averaging QPF

42 Sensitivity to box size and precipitation threshold Positive impact of larger box is more visible at higher precipitation thresholds

43 Best result box = 0.5 deg ? (7 * 7 grid points …) Sensitivity to box size and precipitation threshold

44 Sensitivity to box size and precipitation threshold Best result box = 0.5 deg ? (7 * 7 grid points …)

Some preliminary conclusions QPF spatial averaging over box or alert areas produces a more usable QPF field for applications. Space-time localisation errors are minimised Box or alert areas with size of 5-6 times the grid resolution gives the best results Positive impact of larger box is more visible at higher precipitation thresholds The gain of HRLam with respect to GCMs is greater for high thresholds and for precipitation maxima Better results increasing time averaging (problems with 6 hours accumulation period, much better with 24 hours cumulated period !

46 COSMO General Meeting 2007, Athens, Greece (presented by (Case L) Temporal radius r t =3 obsr t =1 r t =6

47 COSMO General Meeting 2007, Athens, Greece (presented by (Case L) Spatial radius obsr xy =5 r xy =15r xy =10

COSMO general meeting – Athens September 2007 – WG5 48 Italian COSMO Models implementations cross-verifications

COSMO general meeting – Athens September 2007 – WG5 49 Comparison between COSMO-ME and COSMO-IT (with upscaling)

Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Verification of very high resolution (precipitation) « Optimal » scale: 0.5° : 50 km 5 x grid (7km) : 35 km 30 x 2.2 km: 70 km Some signals that 2 km models better than 7 km I would like to generate smothed products Material starts to be collected: MAP D-PHASE, 2km models Work has to continue Exchange of experience with other consortia

Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Verification of COSMO-LEPS and coupling with a hydrologic model André Walser 1) and Simon Jaun 2) 1) MeteoSwiss 2) Institute for Atmospheric and Climate Science, ETH

52 A. Walser Data flow for MAP D-PHASE Main partner WSL: Swiss Federal Institute for Forest, Snow and Landscape Research

53 A. Walser Comparison different models August 2007 event Linth at Mollis, initial time