VERIFICATION Highligths by WG5

2 Outlook Some focus on Temperature with common plots and Conditional Verification Some Fuzzy verification Long trends

SON 2009

DJF

5 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, T2m: mean diurnal cycle (first 24h forecasts) domain Switzerland (hourly SYNOP‘s) Autumn 2009 Winter 2009/2010 Spring 2010 P. Kaufmann, V. Stauch OBS COSMO-7 COSMO-2 Summer 2010

T2m COSMO-I7 00UTC: LAST YEAR

WAM WG5 COSMO General Meeting, Moscow 2010 Conditional Verification Extracting information for relevant performance of weather parameters The input from modelers and forecasters is necessary for identifying and testing hypotheses. F. Gofa - HNMS

Temp in overcast conditions FallWinter SpringSummer F. Gofa - HNMS

COSMO General Meeting – Moscow Sept 2010 Conditional Verification Temp – TCC obs >=75% SON MAM DJF Better behaviour for all the seasons Compare to no condition model

Temp in clear sky conditions FallWinter SpringSummer F. Gofa - HNMS

Conditional Verification Temp – TCC obs <=35% Worse behaviour for all the seasons Compare to no condition model SON MAM DJF

Temp in ‘calm’ conditions (<2 m/s) FallWinter SpringSummer WG5 COSMO General Meeting, Moscow 2010 F. Gofa - HNMS

WG5 COSMO General Meeting, Moscow 2010 FallWinter SpringSummer Temp in ‘high wind’ conditions >10m/s F. Gofa - HNMS

14 Some conclusion A problem with Temp is clear. RMSE between 2-3 °C it is not so small. Diurnal cycle too cold during the day and too warm during the night Clear different behaviour with conditions on TCC and with different wind conditions

15 Outlook Some focus on Temperature with common plots and Conditional Verification Some Fuzzy verification Long trends

16 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, results for h accumulated precipitation sums over the domain of the swiss radar composit models: COSMO-2 and COSMO-7 leadtimes 04 – 07h for all 8 daily forecast runs obervation precipitation estimates of the swiss radar composit in case of a missing value, the full date will not be evaluated (total of 28 days) Neighborhood verification for precipitation

17 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, COSMO-2COSMO-7 COSMO-2 - COSMO-7 -= -= good bad COSMO-7 better COSMO-2 better Fractions Skill Score Upscaling Neighborhood (fuzzy) verification 2009, FSS and UP T. Weusthoff

18 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Fractions Skill Score FSS Upscaling ETS Upscaling freq. bias FBI Neighborhood (fuzzy) verification: Spring 2010 COSMO-2/COSMO-7: 3h acc, leadtime +4 to +6 for all models COSMO-2 COSMO-7 IFS T. Weusthoff

19 Outlook Some Common Plots (Task 6 Versus) Conditional Verification Some Fuzzy verification Long trends verification

Introduction of new Z 0

Total cloud cover _____ Cloud cover above 2 Octa (Cl.1) Cloud cover above 6 Octa (Cl.2) Valid time 00 UTC Cloud cover of low clouds because incorporation of AWS

Stand Mai 2010 Time series of the COSI: State May 2010

Stand Mai 2010 Time series of the COSI: State May 2010

Time series of the COSI: Temperature day 1

Stand Mai 2010 Time series of the COSI: Temperature day 2

Stand Mai 2010 Time series of the COSI: Temperature day 3

Stand Mai 2010 Time series of the COSI: State May 2010 (STDV used for T2m instead of RMSE)

Long period verification (seasonal trend) (from djf’04 to mam’10) 1.Some Statistical indices for low thres (0.2mm/24h) 2.Some Statistical indices for high thres (20mm/24h) Verification ovest last year (DJF 2009-MAM2010) 1. Driving model comparison: ecmwf/Cosmo-I7/Cosmo-I2 2.Driving model comparison: ecmwf/Cosmo-ME/Cosmo-IT

COSMO General Meeting – Moscow Sept 2010 All the versions present a seasonal cycle with an overestimation during summertime (except COSMO-7 and I2) COSMO-7 and I2 underestimate Overestimation error decreases in D+2 (spin-up effect vanished) QPF verification of the 4 model versions at 7 km res. (COSMO-I7, COSMO-7, COSMO-EU, COSMO-ME) with the 2 model versions at 2.8 km res. (COSMO- I2, COSMO-IT) Dataset: high resolution network of rain gauges coming from COSMO dataset and Civil Protection Department  1300 stations Method: 24h/6h averaged cumulated precipitation value over 90 meteo-hydrological basins Seasonal trend - low thresholds

COSMO General Meeting – Moscow Sept 2010 Very light improvement in trend Seasonal error cycle: lower ets during winter and summertime no significant differences between D+1 and D+2 Last winter (very snowy particularly in Northern Italy): low ets value (D+1 and D+2)  model error or lack of representativeness of the rain gauges over the plain during snowfall ? Seasonal trend - low thresholds

COSMO General Meeting – Moscow Sept 2010 Driving model comparison: ECMWF/COSMO- ME/COSMO-IT, low thresholds ECMWF tendency to forecast low rainfall amounts  big overestimation, big false alarms, very low ets, quite good pod Better prediction for COSMO-models (no strong differences between ME and IT) Seasons DJF2009 – MAM2010

COSMO General Meeting – Moscow Sept 2010 ECMWF tendency to forecast low rainfall amounts  big overestimation, big false alarms, very low ets, quite good pod Better prediction for COSMO-models BUT bad performance during summertime Seasons DJF2009 – MAM2010 Driving model comparison: ECMWF/COSMO- I7/COSMO-I2, low thresholds

37 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Precipitation (12h-sums +12 to +24h): Spring 2010 over Switzerland (SYNOP‘s) COSMO-7 & COSMO-2 V. Stauch

38 Verification results at MeteoSwiss in 2010 COSMO GM / WG5 Parallel Session, Precipitation (12h-sums +12 to +24h): Spring 2010 over Switzerland (SYNOP‘s) COSMO-7 & IFS V. Stauch

COSMO General Meeting – Moscow Sept 2010 Slight bias reduction during latest seasons Last winter: all the versions overestimate (probably due to lack of representativeness of the rain gauges over the plain during snowfall) Strong COSMO-7 underestimation BUT slight improvement during latest seasons Seasonal trend - high thresholds

COSMO General Meeting – Moscow Sept 2010 Low values during summertime In general, quite stationary error since son2008 up to now All the versions present a jump around son2008: ets increases from up to Skill decreases with forecast time Seasonal trend - high thresholds

COSMO General Meeting – Moscow Sept 2010 ECMWF difficulty to forecast high rainfall amounts  bias around 1 BUT big false alarms, very low ets and pod Better prediction for COSMO-models Seasons DJF2009 – MAM2010 Driving model comparison: ECMWF/COSMO-ME/COSMO-IT, high thresholds

COSMO General Meeting – Moscow Sept 2010 ECMWF difficulty to forecast high rainfall amounts  bias around 1 BUT big false alarms, very low ets and pod Better prediction for COSMO-models Seasons DJF2009 – MAM2010 Driving model comparison: ECMWF/COSMO- I7/COSMO-I2, high thresholds

12h Precipitation – Sep2009-Aug2010 WG5 COSMO General Meeting, Moscow 2010 COSMOECMWF Really strong overestimation of lower preci amounts up to 3mm and lower ETS scores for ECMWF F. Gofa - HNMS

44 Some conclusion Long term trends show a general (sometimes light) improvements for all the considered models Comparison between COSMO models and IFS shows a general clear better behaviour for COSMO implementations