Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Verification results at MeteoSwiss in the year 2010 Francis Schubiger, Vanessa Stauch, Tanja Weusthoff, Pirmin Kaufmann, Christophe Hug MeteoSwiss COSMO General Meeting 2010 WG5 Parallel Session 6 September 2010

2 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Operational (COSMO-2, COSMO-7, COSMO-LEPS [mean], IFS): Surface (3h-steps SYNOP and 1h-steps SMN [Swiss Met Net]) psred, ps, T2m, Td2m, 10m-wind, cloud cover, 12h-precipitation, gusts Upper-air (TEMP) T, RH, wind,  Quasi-operational (COSMO-2, COSMO-7, IFS): precipitation with RADAR (neighborhood/fuzzy verification, Ebert’s package) also weather-type dependant Monitoring: SMN (swiss SYNOPs), RADAR, windprofilers Ongoing in experimental modus: windprofiler over Switzerland Fluxes at Payerne New / Next developments: Global radiation at swiss stations and with gridded data from CM-SAF VAD (locations of swiss radars) => All these packages will be replaced by VERSUS 2 Verification of COSMO-2, COSMO-7, COSMO-LEPS and IFS

3 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, T2m: mean diurnal cycle (first 24h forecasts) domain Switzerland (hourly SYNOP‘s) OBS COSMO-7 COSMO-2 Summer 2009 Autumn 2009 Winter 2009/2010 Spring 2010 P. Kaufmann, V. Stauch

4 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, T2m: mean diurnal cycle (first 24h forecasts) domain Switzerland (hourly SYNOP‘s) Autumn 2009 Winter 2009/2010 Spring 2010 P. Kaufmann, V. Stauch OBS COSMO-7 COSMO-2 Summer 2010

5 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, T2m: mean diurnal cycle Spring 2010 COSMO-7 vs IFS SYNOP‘s over COSMO-7 domain OBS COSMO-7 IFS P. Kaufmann, V. Stauch

6 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, T2m: mean diurnal cycle Spring 2010 COSMO-7 vs COSMO-LEPS SYNOP‘s over COSMO-7 domain OBS COSMO-7 COSMO-LEPS P. Kaufmann, V. Stauch

7 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, m windspeed: comparison IFS & COSMO-7 Autumn 2009, bias 10m windspeed for forecast +39 to +48h IFSCOSMO-7 P. Kaufmann, V. Stauch

8 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Precipitation: Spring 2010 geographical distribution precipitation 12h sums frequency bias (1 mm/12h) Spring 2010 COSMO-7 P. Kaufmann, V. Stauch

9 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Precipitation: Summer 2010 geographical distribution P. Kaufmann, V. Stauch precipitation 12h sums frequency bias (1 mm/12h) Summer 2010 COSMO-7

10 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, precipitation 12h sums frequency bias (10 mm/12h) Spring 2010 COSMO-7 Precipitation: Spring 2010 geographical distribution P. Kaufmann, V. Stauch

11 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Precipitation: Summer 2010 geographical distribution P. Kaufmann, V. Stauch precipitation 12h sums frequency bias (10 mm/12h) Summer 2010 COSMO-7

12 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, observed frequency probability of detection frequency bias (mm) False alarm ratio SYNOP: comparison IFS & COSMO-7 precipitation 12h sums threshold (0.1 mm/12h) Spring 2010 domain Switzerland IFS COSMO-7 P. Kaufmann, V. Stauch

13 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, observed frequency probability of detection frequency bias (mm) False alarm ratio SYNOP: comparison IFS & COSMO-7 precipitation 12h sums threshold (1 mm/12h) Spring 2010 domain Switzerland IFS COSMO-7 P. Kaufmann, V. Stauch

14 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, observed frequency probability of detection frequency bias (mm) False alarm ratio SYNOP: comparison IFS & COSMO-7 precipitation 12h sums threshold (10 mm/12h) Spring 2010 domain Switzerland IFS COSMO-7 P. Kaufmann, V. Stauch

15 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, observed frequency probability of detection frequency bias (mm) False alarm ratio SYNOP: comparison IFS & COSMO-7 precipitation 12h sums threshold (10 mm/12h) Spring 2010 domain Europe IFS COSMO-7 P. Kaufmann, V. Stauch

16 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, P. Kaufmann, V. Stauch SYNOP: comparison IFS & COSMO-7 precipitation 12h sums mean error (mm/12h) Spring 2010 domain Europe IFS COSMO-7

17 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, P. Kaufmann, V. Stauch SYNOP: comparison COSMO-7 & COSMO-LEPS precipitation 12h sums mean error (mm/12h) Spring 2010 domain Europe COSMO- LEPS COSMO-7

18 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Precipitation (12h-sums +12 to +24h): Spring 2010 over Switzerland (SYNOP‘s) frequency bias: COSMO-7 & COSMO-2 observed frequency V. Stauch

19 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Precipitation (12h-sums +12 to +24h): Spring 2010 over Switzerland (SYNOP‘s) frequency bias: COSMO-7 & IFS observed frequency V. Stauch

20 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Precipitation (12h-sums +12 to +24h): Spring 2010 over Switzerland (SYNOP‘s) observed frequency nn: nearest c5: 5 points s9: 9 points d3: 3x3 points d9: 9x9 points COSMO-2 for different gridpoint selections/areas V. Stauch

21 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Precipitation (12h-sums +12 to +24h): Spring 2010 over Switzerland (SYNOP‘s) COSMO-2 for different gridpoint selections/areas nn: nearest c5: 5 points s9: 9 points d3: 3x3 points d9: 9x9 points V. Stauch

22 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Precipitation (12h-sums +12 to +24h): Spring 2010 over Switzerland (SYNOP‘s) COSMO-7 & COSMO-2 V. Stauch

23 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Precipitation (12h-sums +12 to +24h): Spring 2010 over Switzerland (SYNOP‘s) COSMO-7 & IFS V. Stauch

24 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Precipitation (12h-sums +36 to +48h): Spring 2010 over Switzerland (SYNOP‘s) COSMO-7 & IFS V. Stauch

25 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Precipitation [mm/12h]: frequency bias [in %] (1) threshold mod/domain 09S309S410S110S210S3 7_all _alps mm7_CH _alps _CH _all _alps mm7_CH _alps _CH

26 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, threshold mod/domain 09S309S410S110S210S3 7_all _alps mm7_CH _alps _CH _all _alps mm7_CH _alps _CH Precipitation [mm/12h]: frequency bias [in %] (2)

27 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, threshold mod/domain 09S309S410S110S210S3 7_all _alps mm7_CH _alps _CH8693(20) _alps _alps _CH Precipitation [mm/12h]: frequency bias [in %] (3) Summary: mean bias

28 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, m-temperature: cold bias in winter positive bias during nighttime in Summer (for COSMO-2 also Spring and Autumn) COSMO-2 ~0.3 K warmer (higher values) than COSMO-7 2m-dewpoint: negative bias (~1 K) in the 6-month summer period over COSMO-2 domain 10m-windspeed: negative bias along the coast and on mountains, positive bias inland total cloudiness: mean daily cycle not well represented precipitation: underestimation in Summer (25% COSMO-7, 10% COSMO-2) and overestimation in Winter/Spring (25% COSMO-7, 35% COSMO-2) higher amounts in COSMO-2 / higher amounts over Alpine area low amounts (0.1 mm/12h): overestimated in Winter (over the Alps) high amounts (10 mm/12h): underestimated in COSMO-7 (over full domain by ~30% in summer, 10% in winter) Verification of surface weather parameters

29 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Windspeed: Payerne Winter 09/10 (24h fcst) COSMO-7 COSMO-2 Ch. Hug

30 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, IFS COSMO-7 Windspeed: Payerne Winter 09/10 (24h fcst) Ch. Hug

31 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, temperature: cold bias (~ 0.5 K) in winter and warm bias (~ 0.5 K) in summer from ground up to 300 hPa windspeed: positive bias in PBL (up to +1 m/s in winter) and slight negative bias above 400 hPa Verification with vertical profiles: main results

32 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Neighborhood (fuzzy) verification for precipitation verification on lower resolution: avoid the „double penalty problem“ XX XX XX xX X X x methodOriginal data“fuzzyfication”scoreexample Upscaling Average Equitable threat score Fractions Skill Score (Roberts and Lean, 2005) Fractional coverage Skill score with reference to worst forecast XX XX XX xX X X x T. Weusthoff

33 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, How to read „fuzzy grafics“ increasing window size Increasing threshold numbers / colours  value of the score, here: FSS good bad numbers in bold  useful scales (only for FSS) low skill for high precipitation on small scales best skill for low precipitation on large scales T. Weusthoff

34 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, results for h accumulated precipitation sums over the domain of the swiss radar composit models: COSMO-2 and COSMO-7 leadtimes 04 – 07h for all 8 daily forecast runs obervation precipitation estimates of the swiss radar composit in case of a missing value, the full date will not be evaluated (total of 28 days) Neighborhood verification for precipitation

35 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, COSMO-2COSMO-7 COSMO-2 - COSMO-7 -= -= good bad COSMO-7 better COSMO-2 better Fractions Skill Score Upscaling Neighborhood (fuzzy) verification 2009, FSS and UP T. Weusthoff

36 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Neighborhood (fuzzy) verification: Spring 2010 COSMO-2/COSMO-7: 3h acc, leadtime +4 to +6 for all models Fractions Skill Score (top), Upscaling (bottom) COSMO-2 COSMO-7 COSMO-2 – COSMO-7 T. Weusthoff

37 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Fractions Skill Score FSS Upscaling ETS Upscaling freq. bias FBI Neighborhood (fuzzy) verification: Spring 2010 COSMO-2/COSMO-7: 3h acc, leadtime +4 to +6 for all models COSMO-2 COSMO-7 IFS T. Weusthoff

38 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Neighborhood (fuzzy) verification: Spring 2010 COSMO-2/COSMO-7: 12h acc, leadtime +13 to +24 for all models only 00 and 12 UTC runs COSMO-2 COSMO-7 COSMO-2 – COSMO-7 T. Weusthoff

39 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Fractions Skill Score FSS Upscaling ETS Upscaling freq. bias FBI Neighborhood (fuzzy) verification: Spring 2010 COSMO-2/COSMO-7: 12h acc, leadtime +13 to +24 for all models COSMO-2 COSMO-7 IFS T. Weusthoff

40 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Neighborhood (fuzzy) verification: Spring 2010 mean precipitation bias 12h sums leadtime+13 to +24 for all models 3h sums leadtime: +4 to +6 for COSMO models +4 to +15 for IFS T. Weusthoff

41 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Neighborhood 3h precip sums COSMO Differences winter to summer period: - not significant in Fractions Skill Score and Upscaling with ETS - but significant in the bias, see the Upscaling with frequency bias bias with SYNOP stations in Switzerland: summer 09: 25% underest. winter 09/10: 46% overest. T. Weusthoff

42 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, h sums; leadtime: +4 to +6 for COSMO models, +4 to +15 for IFS Neighborhood (fuzzy) verification: Spring 2010 precipitation T. Weusthoff

43 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, h sums; leadtime+13 to +24 for all models Neighborhood (fuzzy) verification: Spring 2010 T. Weusthoff

44 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, NE N NW SE S SW E W F H L Differences in Fractions Skill score for weather-type dependant verif COSMO-2 minus COSMO-7 COSMO-7 better COSMO-2 better YEAR 2009  greatest adavantage for COSMO-2 for weather types N, NW, SW and „flat“, „high“ and „low“ mainly for higher thresholds

45 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Summary of neighborhood verification What did we learn from neighborhood verification? COSMO-2, COSMO-7 and IFS have skill best forecast of the spatial structure on larger scales (higher FSS values) skill of the models strongly varies for different weather types and also the difference COSMO-2 to COSMO-7 is differently best skill: early summer and autumn, resp. south and westerly flow greatest difference COSMO-2 minus COSMO-7: summer (May to September) resp. for northern and westerly flow and in convective situations What remains unclear ? interpretation of the results for intensity scale: is there a spatial shift in COSMO-2 ? T. Weusthoff

46 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Mögliche Erklärungen für das Minimum an Skill bei einer räumlichen Skala von 16 gridpoints: eine Verschiebung in der Grössenordnung der entsprechenden räumlichen Skala Unterschiede in der räumlichen Ausdehnung der Niederschlags - Muster good bad Fuzzy Verifikation 2009, IS  Weitere Untersuchungen notwendig … COSMO-2COSMO-7 T. Weusthoff

47 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Monitoring COSMO With: radar (intranet: for COSMO-2) swiss SYNOP‘s (intranet: for COSMO-2 & COSMO-7) windprofiler (intranet: for COSMO-2) Example of ARPA Veneto (Andrea Ross) with: PBL height

48 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, D. Leuenberger

49 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Monitoring COSMO: Windprofiler D. Ruffieux, P. Huguenin (Payerne)

50 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Boundary layer height monitoring preliminary results over Veneto ( ) red: TEMP green: radiometer black/white: COSMO-2 fcst A. Rossa (ARPA Veneto)

51 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Temperature profile Padova (I) Jan – Feb 2010 Height dependent cold bias increasing in the first 6h and then relatively constant A. Rossa (ARPA Veneto)

52 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session,

53 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session,

54 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session,

55 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session,

56 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Niederschlags-Bias für 2009 basierend auf 3h acc. List of good / bad cases (MO Intranet)  mehrere Unterschätzungs-Meldungen durch Prognostiker im Juli… Generell: Überschätzung durch beide Modelle in Bezug auf das Radar. Sommer (JJA): Unterschätzung durch COSMO-2.

57 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Neue Wetterlagenklassifizierung für die Verifikation Bisher: manuelle Klassifizierung by E. Zala NEU: automatisierte Klassifizierung basierend auf IFS Analysen (Praktikum D. Jean-Mairet)  11 Klassen: 8 Windrichtungen + Flat, High, Low pressure  63 % Übereinstimmungen mit Zala Klassifizierung für die Jahre GWT Methode (GrossWetterTypes, aus COST 733), drei vordefinierte Druckmuster (= Prototypen) zonal meridional vorticity

58 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Jährliche Wetterlagen im Vergleich (alt und neu) automatisierte Klassifikation gibt mehr N, weniger H