1. Page 1© Crown copyright 2005 SRNWP – Revised Verification Proposal Clive Wilson, COSMO Annual Meeting September 18-21, 2007

2. Page 2© Crown copyright 2005 Original Draft Proposal (not submitted to EUMETNET)  2 aims  Development of a common verification package  Realization of an operational model intercomparison  Further aims  Provide new methods (fuzzy etc)  Allow non-GTS observation data  Radar composites (esp. OPERA)  Responsible Member would:  Write & maintain code of package  Compute comparison scores, website & archive  Find NMS(or ECMWF) to host non-GTS data hub  Motivate NMSs to contribute non-GTS data to hub

3. Page 3© Crown copyright 2005 Not submitted because:  Not fully agreed  Too ambitious- underestimate time/effort needed to develop new code & package  No one indicated wish to be responsible member  Major centres and consortia already had most of proposed functionality in own packages  EUMETNET reluctant to agree proposed cost  (1FTE scientist + travel +25% programme manager)

4. Page 4© Crown copyright 2005 New (draft) proposal – staged aims 1.Inititiate a “realistic” intercomparison based on  Exchange of forecasts from main models at 3-4 centres (format – GRIB then interoperability to define)  Met Office NAE - 12km  Hirlam reference - 15km  Aladin France - 10km  COSMO-EU - 7km  Use existing packages  Accept different station selection, QC (difficult to mandate/change at op. centres)  Verify common scores for same parameters over common areas  Compare and contrast results to reach “consensus”  Extension of existing precip. verification done by Met Office

5. Page 5© Crown copyright 2005 Current Intercomparison of precipitation forecasts

6. Page 6© Crown copyright 2005 External Met Office website- European Precipitation comparison  http://www.metoffice.gov.uk/research/nwp/numeric al/precipitation/emip.html  user : viewprecip  pwd: rain4you  Models:  Hirlam reference  COSMO-EU (DWD lokall)  Aladin (MeteoFrance)  UM – North Atlantic European  COSMO-7 (MeteoSwiss)  Verified against British Isles Nimrod radar composite

7. Page 7© Crown copyright 2005

8. Page 8© Crown copyright 2005 External Met Office website- European Precipitation comparison  24h accumulations thresholds :  0.125, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0, 12.0, 16.0, 20.0, 24.0, 32.0 and 48.0 mm  Scores  frequency bias  Equitable Threat Score (ETS)  log-odds ratio  Extreme Dependency Score  00 UTC run of each model  since the beginning of January 2004  Means, timeseries & ascii contingency tables

9. Page 9© Crown copyright 2005 Some Example results

10. Page 10© Crown copyright 2005 Latest results

11. Page 11© Crown copyright 2005 New (draft) proposal – later staged aims 2.Add higher resolution forecasts to intercomparison 3.Methods/code for high resolution forecasts  Collaborate on investigation of new methods  Intercomparison studies for set of forecasts from single model(s) (cf NCAR project with WRF)  Provide code for new methods  Enable access to radar composites (OPERA) 4.Non-GTS data & hub  Much greater financial and staff resources

12. Page 12© Crown copyright 2005 Why new draft may be acceptable  Practical & pragmatic  Stage 1 does not involve large cost or require extensive code changes  Stage 1 addresses primary concern (of EUMETNET directors) for meaningful verification of operational models  Stage 2 addresses new challenge of high resolution – still active research topic  Stage 3 will be necessary to evaluate and assess future operational high resolution models

13. Page 13© Crown copyright 2005 Questions & Answers