1. Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Verification results at MeteoSwiss in the year 2010 Francis Schubiger, Vanessa Stauch, Tanja Weusthoff, Pirmin Kaufmann, Christophe Hug MeteoSwiss COSMO General Meeting 2010 WG5 Parallel Session 6 September 2010

2. 2 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Operational (COSMO-2, COSMO-7, COSMO-LEPS [mean], IFS): Surface (3h-steps SYNOP and 1h-steps SMN [Swiss Met Net]) psred, ps, T2m, Td2m, 10m-wind, cloud cover, 12h-precipitation, gusts Upper-air (TEMP) T, RH, wind,  Quasi-operational (COSMO-2, COSMO-7, IFS): precipitation with RADAR (neighborhood/fuzzy verification, Ebert’s package) also weather-type dependant Monitoring: SMN (swiss SYNOPs), RADAR, windprofilers Ongoing in experimental modus: windprofiler over Switzerland Fluxes at Payerne New / Next developments: Global radiation at swiss stations and with gridded data from CM-SAF VAD (locations of swiss radars) => All these packages will be replaced by VERSUS 2 Verification of COSMO-2, COSMO-7, COSMO-LEPS and IFS

3. 3 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 T2m: mean diurnal cycle (first 24h forecasts) domain Switzerland (hourly SYNOP‘s) OBS COSMO-7 COSMO-2 Summer 2009 Autumn 2009 Winter 2009/2010 Spring 2010 P. Kaufmann, V. Stauch

4. 4 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 T2m: mean diurnal cycle (first 24h forecasts) domain Switzerland (hourly SYNOP‘s) Autumn 2009 Winter 2009/2010 Spring 2010 P. Kaufmann, V. Stauch OBS COSMO-7 COSMO-2 Summer 2010

5. 5 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 T2m: mean diurnal cycle Spring 2010 COSMO-7 vs IFS SYNOP‘s over COSMO-7 domain OBS COSMO-7 IFS P. Kaufmann, V. Stauch

6. 6 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 T2m: mean diurnal cycle Spring 2010 COSMO-7 vs COSMO-LEPS SYNOP‘s over COSMO-7 domain OBS COSMO-7 COSMO-LEPS P. Kaufmann, V. Stauch

7. 7 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 10m windspeed: comparison IFS & COSMO-7 Autumn 2009, bias 10m windspeed for forecast +39 to +48h IFSCOSMO-7 P. Kaufmann, V. Stauch

8. 8 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Precipitation: Spring 2010 geographical distribution precipitation 12h sums frequency bias (1 mm/12h) Spring 2010 COSMO-7 P. Kaufmann, V. Stauch

9. 9 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Precipitation: Summer 2010 geographical distribution P. Kaufmann, V. Stauch precipitation 12h sums frequency bias (1 mm/12h) Summer 2010 COSMO-7

10. 10 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 precipitation 12h sums frequency bias (10 mm/12h) Spring 2010 COSMO-7 Precipitation: Spring 2010 geographical distribution P. Kaufmann, V. Stauch

11. 11 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Precipitation: Summer 2010 geographical distribution P. Kaufmann, V. Stauch precipitation 12h sums frequency bias (10 mm/12h) Summer 2010 COSMO-7

12. 12 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 observed frequency probability of detection frequency bias (mm) False alarm ratio SYNOP: comparison IFS & COSMO-7 precipitation 12h sums threshold (0.1 mm/12h) Spring 2010 domain Switzerland IFS COSMO-7 P. Kaufmann, V. Stauch

13. 13 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 observed frequency probability of detection frequency bias (mm) False alarm ratio SYNOP: comparison IFS & COSMO-7 precipitation 12h sums threshold (1 mm/12h) Spring 2010 domain Switzerland IFS COSMO-7 P. Kaufmann, V. Stauch

14. 14 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 observed frequency probability of detection frequency bias (mm) False alarm ratio SYNOP: comparison IFS & COSMO-7 precipitation 12h sums threshold (10 mm/12h) Spring 2010 domain Switzerland IFS COSMO-7 P. Kaufmann, V. Stauch

15. 15 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 observed frequency probability of detection frequency bias (mm) False alarm ratio SYNOP: comparison IFS & COSMO-7 precipitation 12h sums threshold (10 mm/12h) Spring 2010 domain Europe IFS COSMO-7 P. Kaufmann, V. Stauch

16. 16 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 P. Kaufmann, V. Stauch SYNOP: comparison IFS & COSMO-7 precipitation 12h sums mean error (mm/12h) Spring 2010 domain Europe IFS COSMO-7

17. 17 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 P. Kaufmann, V. Stauch SYNOP: comparison COSMO-7 & COSMO-LEPS precipitation 12h sums mean error (mm/12h) Spring 2010 domain Europe COSMO- LEPS COSMO-7

18. 18 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Precipitation (12h-sums +12 to +24h): Spring 2010 over Switzerland (SYNOP‘s) frequency bias: COSMO-7 & COSMO-2 observed frequency V. Stauch

19. 19 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Precipitation (12h-sums +12 to +24h): Spring 2010 over Switzerland (SYNOP‘s) frequency bias: COSMO-7 & IFS observed frequency V. Stauch

20. 20 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Precipitation (12h-sums +12 to +24h): Spring 2010 over Switzerland (SYNOP‘s) observed frequency nn: nearest c5: 5 points s9: 9 points d3: 3x3 points d9: 9x9 points COSMO-2 for different gridpoint selections/areas V. Stauch

21. 21 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Precipitation (12h-sums +12 to +24h): Spring 2010 over Switzerland (SYNOP‘s) COSMO-2 for different gridpoint selections/areas nn: nearest c5: 5 points s9: 9 points d3: 3x3 points d9: 9x9 points V. Stauch

22. 22 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Precipitation (12h-sums +12 to +24h): Spring 2010 over Switzerland (SYNOP‘s) COSMO-7 & COSMO-2 V. Stauch

23. 23 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Precipitation (12h-sums +12 to +24h): Spring 2010 over Switzerland (SYNOP‘s) COSMO-7 & IFS V. Stauch

24. 24 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Precipitation (12h-sums +36 to +48h): Spring 2010 over Switzerland (SYNOP‘s) COSMO-7 & IFS V. Stauch

25. 25 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Precipitation [mm/12h]: frequency bias [in %] (1) threshold mod/domain 09S309S410S110S210S3 7_all1039812511798 7_alps113116154140112 0.1 mm7_CH120122164148124 2_alps 94112144130107 2_CH109132151141120 7_all 919411610585 7_alps 9911314913298 1.0 mm7_CH 99115172142107 2_alps 89114142128100 2_CH101131166142111

26. 26 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 threshold mod/domain 09S309S410S110S210S3 7_all65749283 67 7_alps709312311485 5 mm7_CH709413612691 2_alps8212810512894 2_CH89120140139103 7_all5466807362 7_alps60839910680 10 mm7_CH6311011511686 2_alps85119106137100 2_CH95138121135102 Precipitation [mm/12h]: frequency bias [in %] (2)

27. 27 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 threshold mod/domain 09S309S410S110S210S3 7_all3440395243 7_alps4154346454 30 mm7_CH4464-7578 2_alps877466140118 2_CH8693(20)106130 7_alps739612912187 2_alps88108133136104 2_CH99131147142113 Precipitation [mm/12h]: frequency bias [in %] (3) Summary: mean bias

28. 28 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 2m-temperature: cold bias in winter positive bias during nighttime in Summer (for COSMO-2 also Spring and Autumn) COSMO-2 ~0.3 K warmer (higher values) than COSMO-7 2m-dewpoint: negative bias (~1 K) in the 6-month summer period over COSMO-2 domain 10m-windspeed: negative bias along the coast and on mountains, positive bias inland total cloudiness: mean daily cycle not well represented precipitation: underestimation in Summer (25% COSMO-7, 10% COSMO-2) and overestimation in Winter/Spring (25% COSMO-7, 35% COSMO-2) higher amounts in COSMO-2 / higher amounts over Alpine area low amounts (0.1 mm/12h): overestimated in Winter (over the Alps) high amounts (10 mm/12h): underestimated in COSMO-7 (over full domain by ~30% in summer, 10% in winter) Verification of surface weather parameters

29. 29 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Windspeed: Payerne Winter 09/10 (24h fcst) COSMO-7 COSMO-2 Ch. Hug

30. 30 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 IFS COSMO-7 Windspeed: Payerne Winter 09/10 (24h fcst) Ch. Hug

31. 31 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 temperature: cold bias (~ 0.5 K) in winter and warm bias (~ 0.5 K) in summer from ground up to 300 hPa windspeed: positive bias in PBL (up to +1 m/s in winter) and slight negative bias above 400 hPa Verification with vertical profiles: main results

32. 32 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Neighborhood (fuzzy) verification for precipitation verification on lower resolution: avoid the „double penalty problem“ XX XX XX xX X X x methodOriginal data“fuzzyfication”scoreexample Upscaling Average Equitable threat score Fractions Skill Score (Roberts and Lean, 2005) Fractional coverage Skill score with reference to worst forecast XX XX XX xX X X x T. Weusthoff

33. 33 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 How to read „fuzzy grafics“ increasing window size Increasing threshold numbers / colours  value of the score, here: FSS good bad numbers in bold  useful scales (only for FSS) low skill for high precipitation on small scales best skill for low precipitation on large scales T. Weusthoff

34. 34 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 results for 2009 3h accumulated precipitation sums over the domain of the swiss radar composit models: COSMO-2 and COSMO-7 leadtimes 04 – 07h for all 8 daily forecast runs obervation precipitation estimates of the swiss radar composit in case of a missing value, the full date will not be evaluated (total of 28 days) Neighborhood verification for precipitation

35. 35 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 COSMO-2COSMO-7 COSMO-2 - COSMO-7 -= -= good bad COSMO-7 better COSMO-2 better Fractions Skill Score Upscaling Neighborhood (fuzzy) verification 2009, FSS and UP T. Weusthoff

36. 36 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Neighborhood (fuzzy) verification: Spring 2010 COSMO-2/COSMO-7: 3h acc, leadtime +4 to +6 for all models Fractions Skill Score (top), Upscaling (bottom) COSMO-2 COSMO-7 COSMO-2 – COSMO-7 T. Weusthoff

37. 37 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Fractions Skill Score FSS Upscaling ETS Upscaling freq. bias FBI Neighborhood (fuzzy) verification: Spring 2010 COSMO-2/COSMO-7: 3h acc, leadtime +4 to +6 for all models COSMO-2 COSMO-7 IFS T. Weusthoff

38. 38 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Neighborhood (fuzzy) verification: Spring 2010 COSMO-2/COSMO-7: 12h acc, leadtime +13 to +24 for all models only 00 and 12 UTC runs COSMO-2 COSMO-7 COSMO-2 – COSMO-7 T. Weusthoff

39. 39 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Fractions Skill Score FSS Upscaling ETS Upscaling freq. bias FBI Neighborhood (fuzzy) verification: Spring 2010 COSMO-2/COSMO-7: 12h acc, leadtime +13 to +24 for all models COSMO-2 COSMO-7 IFS T. Weusthoff

40. 40 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Neighborhood (fuzzy) verification: Spring 2010 mean precipitation bias 12h sums leadtime+13 to +24 for all models 3h sums leadtime: +4 to +6 for COSMO models +4 to +15 for IFS T. Weusthoff

41. 41 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Neighborhood 3h precip sums COSMO-7 01.07-27.07.0905.12-31.2109 Differences winter to summer period: - not significant in Fractions Skill Score and Upscaling with ETS - but significant in the bias, see the Upscaling with frequency bias bias with SYNOP stations in Switzerland: summer 09: 25% underest. winter 09/10: 46% overest. T. Weusthoff

42. 42 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 3h sums; leadtime: +4 to +6 for COSMO models, +4 to +15 for IFS Neighborhood (fuzzy) verification: Spring 2010 precipitation T. Weusthoff

43. 43 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 12h sums; leadtime+13 to +24 for all models Neighborhood (fuzzy) verification: Spring 2010 T. Weusthoff

44. 44 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 NE N NW SE S SW E W F H L Differences in Fractions Skill score for weather-type dependant verif COSMO-2 minus COSMO-7 COSMO-7 better COSMO-2 better YEAR 2009  greatest adavantage for COSMO-2 for weather types N, NW, SW and „flat“, „high“ and „low“ mainly for higher thresholds

45. 45 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Summary of neighborhood verification What did we learn from neighborhood verification? COSMO-2, COSMO-7 and IFS have skill best forecast of the spatial structure on larger scales (higher FSS values) skill of the models strongly varies for different weather types and also the difference COSMO-2 to COSMO-7 is differently best skill: early summer and autumn, resp. south and westerly flow greatest difference COSMO-2 minus COSMO-7: summer (May to September) resp. for northern and westerly flow and in convective situations What remains unclear ? interpretation of the results for intensity scale: is there a spatial shift in COSMO-2 ? T. Weusthoff

46. 46 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Mögliche Erklärungen für das Minimum an Skill bei einer räumlichen Skala von 16 gridpoints: eine Verschiebung in der Grössenordnung der entsprechenden räumlichen Skala Unterschiede in der räumlichen Ausdehnung der Niederschlags - Muster good bad Fuzzy Verifikation 2009, IS  Weitere Untersuchungen notwendig … COSMO-2COSMO-7 T. Weusthoff

47. 47 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Monitoring COSMO With: radar (intranet: for COSMO-2) swiss SYNOP‘s (intranet: for COSMO-2 & COSMO-7) windprofiler (intranet: for COSMO-2) Example of ARPA Veneto (Andrea Ross) with: PBL height

48. 48 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 D. Leuenberger

49. 49 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Monitoring COSMO: Windprofiler D. Ruffieux, P. Huguenin (Payerne)

50. 50 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Boundary layer height monitoring preliminary results over Veneto (21-22.01.2010) red: TEMP green: radiometer black/white: COSMO-2 fcst A. Rossa (ARPA Veneto)

51. 51 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Temperature profile Padova (I) Jan – Feb 2010 Height dependent cold bias increasing in the first 6h and then relatively constant A. Rossa (ARPA Veneto)

52. 52 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010

53. 53 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010

54. 54 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010

55. 55 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010

56. 56 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Niederschlags-Bias für 2009 basierend auf 3h acc. List of good / bad cases (MO Intranet)  mehrere Unterschätzungs-Meldungen durch Prognostiker im Juli… Generell: Überschätzung durch beide Modelle in Bezug auf das Radar. Sommer (JJA): Unterschätzung durch COSMO-2.

57. 57 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Neue Wetterlagenklassifizierung für die Verifikation Bisher: manuelle Klassifizierung by E. Zala NEU: automatisierte Klassifizierung basierend auf IFS Analysen (Praktikum D. Jean-Mairet)  11 Klassen: 8 Windrichtungen + Flat, High, Low pressure  63 % Übereinstimmungen mit Zala Klassifizierung für die Jahre 2002- 2008 1 GWT Methode (GrossWetterTypes, aus COST 733), drei vordefinierte Druckmuster (= Prototypen) zonal meridional vorticity

58. 58 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, 06.09.2010 Jährliche Wetterlagen 2002-2009 im Vergleich (alt und neu) automatisierte Klassifikation gibt mehr N, weniger H