1. 1 On the use of radar data to verify mesoscale model precipitation forecasts Martin Goeber and Sean Milton Model Diagnostics and Validation group Numerical Weather Prediction Division Met Office, Bracknell, U.K. contributions from Clive Wilson, Dawn Harrison, Dave Futyan and Glen Harris

2. 2 Outline of talk Importance of precipitation verification Verifying observations and ‘wet’ model Statistical methods and interpretation Examples from operational mesoscale model forecasts from the wet autumn 2000

3. 3 12 km 5km gauge Model 5km Nimrod Rain gauge representative of an area of about 20 km 2 on mesoscale timescales (Kitchen and Blackall 1992) # observations per model grid box 0.1:1 5:1

4. 4 Characteristics of the Nimrod data Ground clutter removal fixed Z-R conversion attenuation correction removal of corrupt images removal of anaprop accounting for variations in the vertical reflectivity profile gauge adjustment

5. 5 ‘Wet’ Forecast system characteristics 3D-Var latent heat nudging, 3D-cloud from MOPS cloud microphysics with ice and explicit calculation of transfer between phases prognostic cloud liquid water and ice penetrative convection scheme based on an ensemble of buoyant entraining plumes with a treatment of downdraughts

6. 6 Categorical statistics Count concurrent event/no-event, e.g. precipitation > 2 mm / 3 hours

7. 7 Categorical measures (1) Hit rate False alarm rate

8. 8 Which measures for categorical statistics ? complete description of 2*2 contingency table description of different aspects of relationship between forecasts and observations, e.g. independence from marginal distributions confidence intervals interpretation relationship to value

9. 9 Categorical measures (2) Frequency bias Hansen-Kuipers score Odds ratio

10. 10 Categorical measures (3) Confidence intervals

11. 11 Autumn 2000 Orography Accumulation

12. 12 Accumulation autumn (SON) 2000 12km resolution, 18-24 h forecasts Model Nimrod-obs Bias

13. 13 Rain/no-rain (>0.4 mm/6hrs) 12km resolution, 12-18 h forecasts Bias HKS Odds ratio

14. 14 Heavy precipitation (>4mm/6hrs) 12km resolution, 12-18 h forecasts Bias HKS Odds ratio

15. 15 Heavy precipitation (>4mm/6hrs) 36km resolution, 12-18 h forecasts Bias HKS Odds ratio

16. 16 Estimates of confidence intervals Minimum(a,b,c,d) Error(HKS) Error(log(OR))

17. 17 Frequency bias Hansen-Kuipers score Odds ratio Summary (6h)

18. 18 Frequency bias Hansen-Kuipers score Odds ratio Summary (3h)

19. 19 Regionally integrated statistics a) observed area, b) hourly accumulation c) wet area, d) maximum A) Obs b) 2-3hr c) 8-9hr d) 14-15 e) 20-21 f) 26-27

20. 20 Regionally integrated statistics

21. 21 Regionally integrated statistics Probability for rain in one hour Nimrod (obs) 6-12hrs. forecast 18-24 hrs. forecast

22. 22 Regionally integrated statistics Brier skill score for p(rain in one hour) 6-12 hrs. forecasts 18-24 hrs. forecasts

23. 23 Summary Nimrod (radar) data are relatively great for verifying mesoscale quantitative precipitation forecasts, because of their spatial and temporal resolution and near real time availability. Last autumn’s extreme precipitation in England and Wales was relatively well forecasted. Time series of regionally integrated statistics and categorical data analysis provide scientifically based, yet customer friendly, measures to verify quantitative precipitation forecasts.

24. 24 Future developments Application and development of tests on significance of difference between two samples (e.g. convective vs. frontal precipitation, orographic enhancement, spinup, dependency on resolution, new model formulations) Extension of investigations on catchment scale Comparison of spatio-temporal spectral characteristics of model and observations Lagrangian (event based) statistics