1. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Medium-range flood forecasting and warning X General COSMO meeting Cracow, 15-19 September 2008 Basin-oriented verification of COSMO-LEPS system Andrea Montani ARPA-SIM Hydrometeorological service of Emilia-Romagna, Italy Thanks to H. Asensio, R. Buizza, F. Pappenberger, B. Ritter, J.W. Schipper

2. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Workpackage Medium-Range Plain Flood of PREVIEW project Aims: 1.Set-up and validation of the probabilistic medium-range flood forecasting using the meteorological products for the Upper- Danube in the hydrological year 2002, which includes the large flood of August 2002; to achieve this, reruns of a number of state-of-the-art of atmospheric models were performed, “convenient” (basin-oriented) scoring techniques for probabilistic forecasts were developed. 2.In terms of operational applications, the goal is to demonstrate the usefulness of probabilistic medium-range flood forecasting as a sound basis for early warning and decision making.

3. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 The region of interest

4. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Verification methodology MAIN FEATURES: Verification performed in terms of 24-hour precipitation (from 6UTC to 6UTC); fcst ranges: 18-42h, 42-66h, 66-90h, 90-114h, …; observations: gridded observations (about 5 km of horizontal resolution) provided by JRC; verification domain: full upper-Danube river basin and 3 (out of 19) sub-basins; verification period: 20 July - 31 August 2002.

5. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Sub-basin verification IDname of Basin area (km 2 )no. of gridded obs 1Bratislava 131.9785278 13Wiblingen 2.24780 16Passau-Ingling 25.9771045 20Hofkirchen 47.5341896 For each gridded observation, it was identified the sub-basin it belonged to.

6. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Deterministic measures for verification Catchment-based measures centre of gravity: it is a specific point at which the system's mass behaves as if it were concentrated. The centre of mass/gravity is a function only of the positions and masses of the particles of the system.

7. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Ensemble systems in PREVIEW – – VarEPS by ECMWF (global, ensemble) – COSMO-LEPS by ARPA-SIM (limited-area, ensemble)

8. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 CentreofGravity distance  Aim: quantify the "location error" of the precipitation forecast (by the ensemble mean);  the CoG is scaled with the "mean distance" in a catchment  a value of cog=0.3 for the catchment "Passau-Ingling" (mean distance =113 km) indicates a location error of the forecasted centre of gravity of precipitation in a catchment to the observerd centre of gravity of precipitation of about 30 km.  with the scaling, we get a dimensionless number (…the lower, the better…) to compare the absolute location error with respect to the catchment size. CoG  The absolute figures are very low (distances of the order of a few tens of km).  Little dependence of the distance from the forecast range.  Slightly higher distances for the Passau-Ingling basin (possibly related to observation problems in the Alpine region).  Best results for the smallest basin. CoG distance

9. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Brier Score for VarEPS and COSMO-LEPS BS  ECMWF : solid.  COSMO-LEPS: dotted.  Low values for both models: GOOD!  COSMO-LEPS performs better over the smaller basins (not well resolved by VarEPS).  Similar performance of the two systems over the largest basin.  BS measures the mean squared difference between forecast and observation in probability space.  equivalent to MSE for deterministic forecast.  BS between 0 and 1; the lower the better ….  the largest (Bratislava) and the smallest (Wiblingen) basins are considered (tp > 80% of obs distribution).

10. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 For the other results, come and see the poster!

11. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Thank you !

12. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 The 32-day unified VarEPS at ECMWF Unification of the 15d VarEPS (50+1 members) and the 32d monthly forecast (MOFC) systems into the unified 32d VarEPS: +768 15d and 32d VAREPS T0 +240 +360 +768 NEW SYSTEM (since 11/3/2008) Twice-a-day (at 00 and 12 UTC): d 0-10: T L 399L62 uncoupled d 10-15: T L 255L62 coupled at 00 Once a week: d 0-10: T L 399L62 uncoupled d 10-32: T L 255L62 coupled old system Twice-a-day (at 00 and 12 UTC): d 0-10: T L 399L62 uncoupled d 10-15: T L 255L62 uncoupled Once a week: –d 0-32: T L 159L62 coupled 15d VAREPS T0 +240 +360 T0 32d MOFC

13. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Skill of ECMWF predictions for hydrological modelling  Nash_Sutcliffe efficiency coefficient: normally used to assess the predictive power of hydrological models.  … the higher, the better (ranges from –  to 1)  Timing of forecasts is good, although underestimation occurs.  The ensemble spread brackets observations only partially.

14. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 17 th of July 2007

15. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 7 Panel version

16. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Atmospheric models in PREVIEW – – COSMO-LEPS by ARPA-SIM (limited-area, ensemble)

17. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 COSMO-LEPS (developed at ARPA-SIM) MAIN FEATURES: initial time: 12 UTC (i.e. once a day); bc and ic: “selected” VarEPS members; COSMO-LEPS configuration – 10 members; – hor. res. = 10 km; 32 vertical levels; – forecast length: 132h; – archived variables: surf and plev up to +132h, every 3h; – output fields archived at ECMWF; rerun period: 20 July – 31 August 2002. COSMO-LEPS: the Limited-area Ensemble Prediction System (LEPS), based on COSMO-model and implemented within COSMO (COnsortium for Small-scale MOdelling, including Germany, Greece, Italy, Poland, Romania, Switzerland). integration domain

18. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Brier Skill Score vs forecast range  BSS is written as 1-BS/BS ref. Sample climate is the reference system. Useful forecast systems if BSS > 0.  BS measures the mean squared difference between forecast and observation in probability space.  Equivalent to MSE for deterministic forecast. BSS  For low thresholds, better performance over the smallest basin.  For higher thresholds, more difficult to assess a clear trend (possible sampling problems over small basins).  Worse than climatology only for d+5 range (10 and 20 mm threshold).

19. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Brier Skill Score vs increasing threshold  BSS is plotted vs increasing threshold to assess the COSMO-LEPS skill as a function of rainfall intensity.  BSS positive for all thresholds at both forecast ranges.  At low thresholds, it is confirmed the higher skill of COSMO-LEPS over the smallest basin.  Stable skill of the system for intermediate basins.  For higher thresholds, more difficult to assess a clear trend (sampling problems over the small basin). BSS +90hBSS +42h

20. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Ranked Probability Skill Score  A sort of BSS, but “cumulated” over all thresholds.  Useful forecast system, if RPSS > 0. RPSS  RPSS always positive.  Better (worse) performance of the system over the smallest (largest) basin.  Almost identical scores using either nearest grid-point (NGP) or bilinear interpolation (BILIN).

21. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 ROC area vs increasing threshold  ROC area is plotted vs increasing threshold to assess the dependence of COSMO-LEPS HIT/FAR diagram on rainfall intensity.  ROC area always above 0.7 for all thresholds at both forecast ranges.  At low thresholds, it is confirmed the higher skill of COSMO-LEPS over the smallest basin (the same as BSS).  Scores increase with thresholds for intermediate and large basins.  For higher thresholds, more difficult to assess a clear trend (sampling problems over the small basin). ROC +42hROC +90h

22. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 ROC area vs forecast range  Area under the curve in the HIT rate vs FAR diagram.  Valuable forecast systems have ROC area values > 0.6. ROC  ROC area always above 0.6;  at low thresholds, better COSMO- LEPS performance over the smallest basin;  possible sampling problems at 20 mm thresholds.

23. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Outliers  How many times the analysis is outside the forecast interval spanned by the ensemble members.  … the lower the better … OUTL  The absolute figures are quite large (about 20% at d+5 range).  Lower outliers for the smallest basin.  Similar percentages for all basins at the longest forecast range.

24. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Main results and future plans Output files are of COSMO-LEPS, for the period 20/7/ to 31/8/2002, are archived on ECFS (retrieval script disseminated) and ready to be used for hydrological purposes; for any kind of help with grib files, just ask. Verification against gridded observations (about 5300 in the full upper-Danube basin) indicates better performance of the system over the smallest basin, especially for low thresholds. OLD RESULT (Offenbach, 25-26-/9/2007): Verification against SYNOP stations (about 100 in the full upper-Danube basin) indicates slightly better performance of the system over larger basins, although results are not statistically robust. Difficulty to understand and use the CoG measure for a probabilistic system. Finish the work on the verification report. Provide COSMO-LEPS fields for 2-month real-time testing (no “bureaucratic” problems envisaged). Real-time testing of services will be done using the “improved” COSMO–LEPS (16 members; 40 vertical levels; physics perturbations).

25. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Main results During the hydrological year 2002, reruns of a number of NWP systems, both global-scale and limited-area, both deterministic and probabilistic. Development of new –basin-oriented scores (e.g. centre of gravity); –hydrological-oriented products (e.g. rivergrams); All systems seem to provide useful guidance for the possible occurrence of flood events also for forecast ranges up to 7 (5) days for global (limited-area) systems. Verification vs gridded observations (about 5300 in the full upper-Danube basin) indicates better performance of the high-res system over the smaller basin. The verification period is probably too short to draw general (and statistically solid) conclusions about the overall skill of the different forecast systems; the “Plain-flood campaign” clearly shows the potential of state-of-the-art NWP systems in the field of weather forecasting for river flooding.

26. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Probabilistic measures for verification Brier Score The BS is the mean-squared error of the probability forecasts: where: the observation is either o k = 1 (the event occurs) or o k = 0 (the event does not occur); f k is the fraction of ensemble members which forecast a precipitation amount exceeding that threshold k denotes a numbering of the N forecast/event pairs. BS ranges from 0 to 1, the perfect forecast having BS = 0. BS is computed for a fixed precipitation threshold. ROC area It is the area under the Relative Operating Characteristics curve in the HIT rate vs FAR diagram The integral under the curve is used to indicate the skill of the forecast. ROC area ranges from 0 to 1 … the higher the better … Useful forecast systems have ROC area values greater than 0.6

27. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Summary of measures Deterministic measures: Root Mean Squared Error Mean Error Probability of detection Probability of false detection True Skill Statistics Centre of gravity Coverage Probabilistic measures: Brier score ROC area

28. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008

29. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Performance measures 1)continuous measures (RMSE and MAE for the ensemble mean); 2)catchment-based measures (CentreofGravity for the ensemble mean); 3)probabilistic measures (Brier Skill Score, ROC area, Percentage of Outliers, …). Centre of Gravity: the centre of gravity of a system of raster cells is a specific point at which, for many purposes, the system's mass behaves as if it were concentrated. The centre of mass is a function only of the positions and masses of the particles that comprise the system. CoG:Centre of Gravity measure c:Centre of gravity dist:Distance f:forecast distmean: mean distance of grid elements to catchment outlet obs:observed p: precipitation c= this score indicates a location error of the forecast, a perfect score having the value of 0.

30. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Catchment Information System River-grams Sylvie Lamy-Thepaut, Enrico Fucile

31. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Main results During the hydrological year 2002 (1/10/2001 to 30/9/2002), reruns of a number of NWP systems, both global-scale and limited-area, both deterministic and probabilistic. Coordinated efforts to provide state-of-the-art weather forecasts over the Danube river basin. Development of basin-oriented scores Development of hydrological-oriented products (e.g. rivergrams) All systems seem to provide useful guidance for the occurrence of flood events.  increase in ROC area scores and reduction in outliers percentages;  positive impact of increasing the population from 5 to 10 members (June 2004);  some deficiency in the skill of the system can be identified after the system upgrades occurred on February 2006 (from 10 to 16 members; from 32 to 40 model levels), BUT  scores are encouraging throughout DPHASE Operations Period. Time-series verification scores indicate the following trends:

32. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 “Close the gap” between hydrological and meteorological communities. Learn from MAP D-PHASE experience Exploit the wealth of information provided by probabilistic forecasts: –assess performance over different domains (North and South of the Alps), –study individual case studies, –consider basin-by-basin performance. “Think about” increasing horizontal resolution to 7 km. Calibrate COSMO-LEPS fcsts using reforecasts (F. Fundel, Meteoswiss). What is left?

33. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Sub-basin verification IDname of Basin area (km 2 )no. of “stations”approx no of grid points 1Bratislava 131.97852781320 13Wiblingen 2.2478020 16Passau-Ingling 25.9771045260 20Hofkirchen 47.5341896470 For each gridded observation, it was identified the sub-basin it belonged to.

34. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Ranked Probability Skill Score  A sort of BSS, but “cumulated” over all thresholds.  Useful forecast system, if RPSS > 0. RPSS  RPSS always positive.  Better (worse) performance of the system over the smallest (largest) basin.  Almost identical scores using either nearest grid-point (NGP) or bilinear interpolation (BILIN).

35. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 CentreOfGravity distance  Scaled (that is, dimensionless) distance between predicted (by COSMO-LEPS ensemble mean) and observed centre of gravity for each catchment.  the "cog" is an attempt to quantify the "location error" of the precipitation forecast; we have scaled the "cog" with the "mean distance" in a catchment - so a value of cog=0.3 for the catchment "Passau-Ingling" with a mean distance distmean=113 km indicates a location error of the forecasted centre of gravity of precipitation in a catchment to the observerd centre of gravity of precipitation of about 30 km. With the scaling we get a (dimensonless) number to compare the absolute location error with respect to the catchment size.  … the lower the better … CoG  The absolute figures are very low (distances of the order of a few tens of km).  Little dependence of the distance from the forecast range.  Slightly higher distances for the Passau-Ingling basin.  Best results for the Wiblingen basin (the smallest one)  Similar distances for the other basins. CoG distance

36. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Catchment Information System River-grams Proto type, currently implemented operational. Database of catchments will be extended to more catchments in Europe and to include all major World catchments. Variables are currently static (always the same for all catchments), but will be ‘dynamic’ – reflecting the usage of catchments.

37. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Outliers  How many times the analysis is outside the forecast interval spanned by the ensemble members.  … the lower the better … OUTL  The absolute figures are quite large (about 20% at d+5 range).  Lower outliers for the smallest basin.  Similar percentages for all basins at the longest forecast range.

38. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Verification of GME forecasts against rain gauge data from high density observation network for the four sub-catchments of the Danube with different catchment sizes: increasing centre of gravity score („location error“) with increasing forecast time; a positive mean coverage error for this period. Catchment-based verification of GME (2) period 2002-01-01 – 2002-09-30

39. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 the verification period might be too short for scores to be statistically significant. Verification of GME and COSMO-EU forecasts against rain gauge observations for the catchment Hofkirchen (upper Danube): increasing centre of gravity score („location error“) with increasing forecast time mostly a negative mean coverage error for this period Catchment-based verification of DWD models (2) period 2002-07-19 – 2002-08-20

40. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 COSMO-1km – cont. measures A.Montani; Novelties in weather forecasting PREVIEW training workshop - Mosonmagyaróvár - 19-20 June 2008

41. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Dim 2 Initial conditions Dim 1 Dim 2 Possible evolution scenarios Dim 1 Initial conditions ensemble size reduction Cluster members chosen as representative members (RMs) LAM integrations driven by RMs LAM scenario COSMO-LEPS methodology

42. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 COSMO-LEPS (developed at ARPA-SIM) What is it? It is a Limited-area Ensemble Prediction System (LEPS), based on COSMO-model and implemented within COSMO (COnsortium for Small-scale MOdelling, which includes Germany, Greece, Italy, Poland, Romania, Switzerland). Why? It was developed to combine the advantages of global-model ensembles with the high-resolution details gained by the LAMs, so as to identify the possible occurrence of severe and localised weather events (heavy rainfall, strong winds, temperature anomalies, snowfall, …) generation of COSMO-LEPS to improve the Late-Short (48hr) to Early-Medium (132hr) range forecast of severe weather events.

43. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 ROC area vs forecast range  Area under the curve in the HIT rate vs FAR diagram.  Valuable forecast systems have ROC area values > 0.6. ROC  ROC area always above 0.6.  Similar results to those obtained in terms of BSS: at low thresholds, better COSMO-LEPS performance over the smallest basin.  Possible sampling problems at 20 mm thresholds.

44. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Probabilistic measures for verification (1) The probabilistic measures we used, are the Brier Score and the ROC area. The thresholds for their computation are based on both absolute values (>1mm) and on the percentiles of the observed cumulative precipitation distribution (> 80% of observed). Brier Score The BS is the mean-squared error of the probability forecasts; it averages the squared difference between pairs of forecast probabilities and the correspondent binary observations: where: the observation is either o k = 1 (the event occurs) or o k = 0 (the event does not occur); k denotes a numbering of the N forecast/event pairs. BS ranges from 0 to 1, the perfect forecast having BS = 0. BS is computed for a fixed precipitation threshold and f k is the fraction of the ensemble members which forecast a precipitation amount exceeding that threshold.

45. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Probabilistic measures for verification (2) ROC area It is the area under the Relative Operating Characteristics curve in the HIT rate vs FAR diagram Hit rate (HR) and false alarm rate (FAR) are computed for each probability class k: where: the verification sample is subdivided into subsamples of size N k, according to the probability with which the event is forecasted, o k is the frequency with which the event is observed, being forecasted with a given probability and is the sample climatology. The cumulative HR k are plotted against the corresponding cumulative FAR k, generating the ROC curve. The integral under the curve is used to indicate the skill of the forecast. ROC area ranges from 0 to 1, the higher the better. Useful forecast systems have ROC area values greater than 0.6

46. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 COSMO-1km – Skill Scores (IMK) A.Montani; Novelties in weather forecasting PREVIEW training workshop - Mosonmagyaróvár - 19-20 June 2008

47. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 17 th of July 2007

48. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Outline Introduction Measure for model performance (precipitation only) Atmospheric models re-run for PREVIEW: – GME by DWD (global, deterministic), – COSMO-EU by DWD (limited-area, deterministic), – VarEPS by ECMWF (global, ensemble), – COSMO-LEPS by ARPA-SIM (limited-area, ensemble), – COSMO-1km by IMK (limited-area, deterministic) (tomorrow’s talk by J.W. Schipper) Application to the Danube sub-basins Lesson learnt

49. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Deterministic measures for verification (1) Continuous measures The Mean Error (ME, bias) of the precipitation forecast against the measurement (F: Forecast; O: Observations; N: sample size): ME ranges from –infinity to infinity; the closer to zero, the better … Categorical measures Consider thresholds (e.g. tp >1mm and tp > 80% of observed mean). Scores are generated with the help of a contingency table:

50. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Atmospheric models in PREVIEW – GME by DWD (global, deterministic) – COSMO-EU by DWD (limited-area, deterministic) –

51. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 GME: grid spacing: 40 km 368642 * 40 grid elements time step : 133 sec. forecasts up to 7 days COSMO-EU: grid spacing: 7 km 665*657 * 40 grid elements time step : 40 sec. forecasts up to 78 hours COSMO-DE: grid spacing: 2.8 km 421*461 * 50 grid elements time step: 25 sec forecasts up to 21 hours DWD operational NWP models

52. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 Example: DWD forecast of 24h accumulated precipitation with GME (global) and COSMO-EU (regional) for 2002-08-12 06 UTC to 2002-08-13 06 UTC, the forecast start time is 2002-08-11 12 UTC. forecasts with the global model GME for the hydrological year 2002 (2001-10-01  2002-09-30). forecasts with the high-resolution regional model COSMO-EU for the PREVIEW special period (2002-07-19  2002-08-20). DWD meteorological forecasts (1)

53. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 24h catchment mean precipitation for the Hofkirchen catchment (upper Danube) forecasts with GME and COSMO-EU and corresponding adjusted radar data and high-density network rain gauge observations in the period 2002-08-01 to 2002-08-20. There are differences between the two observing systems  the overall evolution for the flooding event is well captured by both forecast models (COSMO-EU slightly better). DWD meterorological forecasts (2)

54. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 GME forecasts vs rain-gauge data from high-density network for the four sub- catchments of the Danube with different catchment sizes: positive bias for all four catchments in this period (the model “rains” too much); positive TSS (true skill statistics), i.e. the forecasts are skilful. Catchment-based verification of GME period 2002-01-01 – 2002-09-30

55. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 the verification period might be too short for scores to be statistically significant. GME and COSMO-EU forecasts vs rain-gauge observations for the catchment Hofkirchen: negative bias for both models (the models “rain” too little); GME slightly better for low thresholds positive TSS (true skill statistics), i.e. the forecasts are skilful. Catchment-based verification of DWD models period 2002-07-19 – 2002-08-20

56. A.Montani; Basin-oriented verification of COSMO-LEPS X COSMO meeting – Cracow – 15-19 September 2008 “Close the gap” between hydrological and meteorological communities. Consider the outcome of MAP D-PHASE experimentation (model performance may vary considerably from basin to basin). Develop and exploit “basin-oriented” calibration of NWP systems (being implemented in these months for ECMWF varEPS and COSMO-LEPS). Exploit the wealth of information provided by probabilistic forecasts: –get more and more acquainted with the concept of probability; –consider the “gain” (€, $, Ft,…)  presentation by D. Richardson. What is left?