1. Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Quantitative precipitation forecast in the Alps Verification results DOP MAP-DPHASE June-November 2007 July 2008 COSMO GM, PP INTERP, 16.09.2008 Felix Ament 1,2,Tanja Weusthoff 1, Matteo Buzzi 1 1 MeteoSwiss 2 Universität Hamburg

2. 2 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch Quantitative precipitation forecast with NWP Intensity Scale difficult good difficult good

3. 3 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch Quantitative precipitation forecast with NWP 1. How bad is COSMO-2? Traditional verification $ 2. How useful can be a model forecast? Cost loss analysis 3. At which scale and intensity we perform better? Fuzzy verification July 2008 4. COSMO-2: Convection resolving or convection inhibiting model? Turbulent lenght scale

4. 4 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch MAP D-PHASE Demonstration of Probabilistic Hydrological and Atmospheric Simulation of flood Events in the Alpine region 2nd WWRP FDP (forecast demonstration experiment) after Sydney 2000 and before Beijing 2008 Focuses on heavy precipitation, hydrology, high-resolution numerical modeling and ensembles Huge number of participants: 30 atmospheric models, 17 hydrological models and more than 35 end users. Establishes an end-to-end forecasting system, which is operated from June until November 2007 Data archive >10 TB model data in a unified GRIB format for scientific evaluation www.d-phase.info Real-time warnings of all models forecast chart

5. 5 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch Observational data Swiss Radar composite 3 Radar stations 5 min scans accumulated to hourly estimates 1km resolution Gridded rain gauge data Statistical interpolation + elevation correction Daily accumulations Calibrated radar time series Warn regions averages Hourly accumulations Daily sums equivalent to gridded gauge data No alert 10 times a year Twice a year Every 10 years 65 warn regions domain averages 3 warnlevels

6. 6 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch Alerts – level „yellow“, 3h int., JJA 2007 (Alert level yellow = return frequency of 6 times per year) Relative frequency of an alert (frequency bias) 0.85 Probability to detect an event (probability of detection) 46% Probability to issue a false alarm (false alarm ratio) 46% and but COSMO-2

7. 7 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch Daily cycle, JJA 2007 Deep convection resolving models Parameterized convection Significantly better timing!

8. 8 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch Contingency table COSMO-2, JJA 2007 Alert level yellow? YesNo Yes1.51%1.76% No1.29%95.44% Observation frequency Model 10 log ( Rain / Rain Threshold )

9. 9 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch Concept of “Relative Value” Economic point of view: Precautions causes Costs Having no protection results in Losses YesNo YesCC NoL0 Event Precaution Relative Value no forecast Total Cost 1.0 0.0 0.25 0.5 0.75 useful + - useless real forecast perfect forecast $

10. 10 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch $ Relative value – Alert level „yellow“ useful + - useless … against false alarms insensitive … sensitive … (03h, 06h and 12h accumulations, cut-off +03h) JJA 2007

11. 11 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch Advantage of convection resolving models useful + - useless … against false alarms insensitive … sensitive … (03h, 06h and 12h accumulations, cut-off +03h) global model resolved conv. all models param. conv. $ JJA 2007

12. 12 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch Impact of rapid update cycle all initial times only starting at 00UTC (03h, 06h and 12h accumulations, cut-off +03h) $ JJA 2007

13. 13 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch Fuzzy Verification Verification on coarser scales than model scale: “Do not require a point wise match!“ XX XX XX xX X X x MethodRaw DataFuzzyficationScoreExample result Upscaling Average Equitable threat score Fraction Skill Score (Roberts and Lean, 2005) Fractional coverage Skill score with reference to worst forecast XX XX XX xX X X x Radar composite

14. 14 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch Upscaling (2007) JJA SONJJASON COSMO-2 COSMO-7 good bad

15. 15 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch Fraction skill score (2007) JJA SONJJASON COSMO-2 COSMO-7 good bad

16. 16 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch Score vs intensity, whole dop COSMO-2 COSMO-7 JJASON 2007

17. 17 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch COSMO-2 – COSMO-7 (2007) JJA SONJJASON COSMO-7 better COSMO-2 better

18. 18 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch COSMO-EU - COSMO-7 (2007) JJASON COSMO-7 better COSMO-EU better JJASONJJASON

19. 19 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch COSMO-DE - COSMO-2 (2007) COSMO-2 better COSMO-DE better JJASONJJASON

20. 20 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch COSMO-2: Convection resolving or convection inhibiting model? COSMO-2 RADAR mm/24h

21. 21 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch Parallel test-suite - July 2008 Motivation: Try something in order to save model reputation Past tests: COSMO-2 model is very sensitive to changes of the turbulence scheme parameter tur_len Testsuite: Change of settings in the statistical cloud scheme Modified asymptotic turbulent length scale (tur_len) Fuzzy verification for 00 and 12 UTC runs separately Fuzzy verification for optimal time-serie considering all the available runs with a cut-off of 3h

22. 22 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch July 2008 – 00 UTC RUNS COSMO-2 Operational COSMO-2 testDifference COSMO-2 operational better COSMO-2 test better COSMO-2 operational better COSMO-2 test better

23. 23 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch July 2008 – 12 UTC RUNS COSMO-2 Operational COSMO-2 testDifference COSMO-2 operational better COSMO-2 test better COSMO-2 operational better COSMO-2 test better

24. 24 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch July 2008 – all runs with 3h cut-off COSMO-2 Operational COSMO-2 testDifference COSMO-2 operational better COSMO-2 test better COSMO-2 operational better COSMO-2 test better

25. 25 COSMO GM, PP INTERP, Quantitative precipitation verification in the Alps Matteo.Buzzi[at]meteoswiss.ch Conclusions Quality of precipitation warnings certainly does not satisfy all demands (D-PHASE set-up: ~50% POD, ~50% FAR). However COSMO-2 gives very good results compared to other D- PHASE models. NWP forecasts are not perfect but they can be very useful depending on the user sensitivity to losses and costs. Customer specific verification. Convection resolving models perform better. In particular, COSMO-2 outperforms its host models: COSMO-7 and the European center model IFS. Performance is clear better at larger scales and for small thresholds. COSMO-EU is slightly better than COSMO-7, COSMO-DE and COSMO-2 are nearly equivalent A reduction of the asymptotic turbulent length scale gives a small improvement in COSMO-2 (above all runs started in the SBL) $

26. Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Thank you for your attention!