VERIFICATION Highligths by WG5

2 Outlook The COSMO-Index COSI at DWD Time series of the index and its DWD 2003

U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach Approach Application of a score like the UK-index - basic values Continuous elements Wind Skill score for wind and continuous elements related to persistence Skill score for categorical elements related to chance

U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach Approach Application of a score like the UK-index - final score

U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach Scores in MOVI and UKMO Continuous parameters: Reduction of variance RV = 1 – (RMSE prog / RMSE ref) 2 where ref = persistence Categorical parameters: ETS –ETS = (R – „chance“) / (T –“chance“) R= number of obs events correctly forecast T = number of events which were either observed or forecasted  global score S like COSMO-index COSI = S/S 0 x100

U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach Parameters total cloud amount [threshold: 0-2, 3-6, 7-8 temperature [t2m, later: tmin, tmax] 10m- windvector precipitation [thresholds: 0.2, 2, 10 mm/6h]

U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach Verification frequency Every 3h –T2m, 10m-wind and 00, 03,…, 18, 21 UTC later on: tmin & tmax over 12h 6h-sums: precipitation

U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach T 2m V 3.22 V3.19 COSMO-EU SSO DAY 1 DAY 2 DAY 3

U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach Scores for day 1

U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach Scores for day 2

U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach Scores for day 3

U. Damrath: Long term time series of forecast quality of COSMO-EU - COSMO GM, Offenbach Scores for different forecast times FS

Long period verification (seasonal trend) (from djf’04 to mam’09)  1.Statistical indices for low thres (0.2mm/24h) 2.Statistical indices for high thres (20mm/24h) Verification ovest last year ( )  1. Driving model comparison: ecmwf/Cosmo-I7/Cosmo-I2 2.Driving model comparison: ecmwf/Cosmo-ME/Cosmo-IT 3.Diurnal cycle for all the model

Seasonal trend (low thres) Bias reduction trend Seasonal cycle: big peak during summertime Biggest overestimation peak for cosmo-I7 Underestimation for cosmo-7 during latest seasons

Seasonal trend (low thres) Stable trend/slightly worsening in time Best performance during spring/summertime Worsening for Cosmo-7 during latest seasons

Seasonal trend (low thres) False alarm number reduction esp. in wintertime Worse performance during summertime, esp. for Cosmo-I7

Seasonal trend (low thres) Slightly improvement trend Seasonal cycle: better during moist seasons, worse during dry seasons

Seasonal trend (high thres) Bias reduction trend, at least during last year Seasonal cycle: big peak during spring-summertime (convective period)  seems to disappear during last summer (… why?) General good performance during last year Pronounced underestimation for Cosmo-7 during last seasons

Seasonal trend (high thres) Slightly improvement in time Worse performance during summertime (except 2007) Worsening for Cosmo-7 during last seasons

Seasonal trend (high thres) False alarm number reduction Worse performance during summertime

Seasonal trend (high thres) Slightly improvement trend Worse performance during summertime (except 2007)

Ecmwf vs Cosmo-I7 vs Cosmo-I2 Big gap between Ecmwf and Cosmo-model Cosmo-I7  slightly overestimation; Cosmo-I2  underestimation; Ecmwf  overest. for low thres/ underest. for high thres. I7 equivalent or slightly better then I2 (even if less false alarm for I2) THRES

Ecmwf vs Cosmo-I7 vs Cosmo-I2 fixed thres. - seasonal Big gap between Ecmwf and Cosmo-model Positive trend for both I7 and I2 I2  Underestimation tendency I7 is generally better JJA SON DJF MAM JJA SON DJF MAM

COSMOME vs ECMWF Temperature SON 2008 DJF MAM 2009 JJA

COSMOME vs ECMWF Dew Point Temperature SON 2008 DJF MAM 2009 JJA

COSMOME vs ECMWF – Wind Speed SON 2008 DJF MAM 2009 JJA

COSMOME vs ECMWF – Precipitation SON - FBI 0,2 mm/122h 2 mm/122h 10 mm/122h 1 1 1

COSMOME vs ECMWF – Precipitation DJF - FBI 0,2 mm/122h 2 mm/122h 10 mm/122h 1 1 1

COSMOME vs ECMWF – Precipitation MAM - FBI 0,2 mm/122h 2 mm/122h 10 mm/122h 11 1

Big gap between Ecmwf and Cosmo-model IT and ME –> quite similar, it is difficult to decide the winner IT  tendency to overestimation Ecmwf vs Cosmo-Me vs Cosmo-IT THRES

Big gap between Ecmwf and Cosmo-model IT and ME –> quite similar, it is difficult to decide the winner IT  tendency to overestimation Ecmwf vs Cosmo-Me vs Cosmo-IT fixed thres. - seasonal JJA SON DJF MAM JJA SON DJF MAM

diurnal cycle for all the versions Low thres High thres Bias overestimation peak during midday Spin-up problem for all the models especially COSMO-I7 and COSMO-I2 General worsening with forecast time The spin-up seems to disappear, underestimation during the first 6h Bias overestimation peak during midday COSMO-7 underestimates In general: slight improvement with respect to the previous year

To sum up(1) Long period verification (seasonal trend) (from djf’04 to mam’09)  1.General improvement trend 2.For low thres (rain/no rain): overestimation during spring/summertime with more probability of detection but also more false alarms 3.For high thres: the worse skills during spring/summertime (convective period) 4.General Cosmo-7 worsening during last year

To sum up (2) Verification over last year ( )  1.Good performance for Cosmo-EU 2.Similar performance Cosmo-I7/ Cosmo-ME (slightly better Cosmo- ME) 3. Cosmo-I7/ Cosmo-I2 comparison: similar skill, underestimation I2 4.Cosmo-ME/Cosmo-IT comparison: similar skill, overestimation IT 5.Diurnal cycle: bias overestimation peak during midday, in general slight improvement with respect to previous year

Some Conditional Verification using VERSUS -Verification of 2mT in Clear Sky Condition from the Model -Verification of 2mT in Clear Sky Condition from the Obs -Verification of 2mT in TCC Condition from the Model -Verification of 2mT in TCC Condition from the Obs Compared with “no condition” verification

Temperature CM SON 2008 MAM 2009 DJF

Conditional Verification Temper. with SC from Model SON 2008 MAM 2009 DJF

Conditional Verification Temper. with SC from Obs SON 2008 MAM 2009 DJF

Temperature CM SON 2008 MAM 2009 DJF

Conditional Verification Temper. with TCC from Model SON 2008 MAM 2009 DJF

Conditional Verification Temper. with TCC from Obs SON 2008 MAM 2009 DJF

Temperature CM SON 2008 MAM 2009 DJF