1. Systematic timing errors in km-scale NWP precipitation forecasts
Marion Mittermaier
Model diagnostics and novel verification methods, Weather Science
© Crown copyright Met Office

2. Outline Introduction Method and rationale
Offsets as described by correlations
Impact of offsets on frequency bias and skill scores
Conclusions
© Crown copyright Met Office

3. Introduction
© Crown copyright Met Office

4. Introduction Spatial offsets can have a temporal origin
Double penalty effect
Errors are counted as false alarms and misses.
Detail penalised, closeness not rewarded
2. Unskilful scales
Grid-scale detail should not be believed
Lorenz (1969) argued that the ability to resolve smaller scales would result in forecast errors growing more rapidly -> more noise
© Crown copyright Met Office

5. Methodology
© Crown copyright Met Office

6. Data and method
Consider 18 months of 36h forecasts from the km Met Office Unified Model (UKV); compared to 2 km rainfall accumulations from the UK radar network.
Filter the noisy unskilful spatial scales out of the field through smoothing/upscaling to 12 km.
Correlate the hourly forecast accumulation to the adjacent hourly radar accumulations, up to 3h hours either side  Which offset has the largest correlation?
Compute the frequency bias and skill scores for a range of thresholds at all offset times  Where is the bias and the score optimised?
© Crown copyright Met Office

7. Caveats
Correlation measures the pattern association between two fields. It is insensitive to the intensity or bias and therefore, to some extent to the imperfections in the observations.
The categorical stats on the other hand are sensitive to the observations characteristics, which are known to be influential in this case.
Upscaling to 8 times the native grid resolution (from km to 12 km) may still not be enough for the traditional categorical statistics.
© Crown copyright Met Office

8. Two-dimensional distribution of maximum correlations as a function of offset
July 2012
January 2012
May 2012
Hourly forecast correlations are separated by initialisation time
Showing the distribution of hourly maximum correlations aggregated over months as a function of initialisation and time offset.
Evidence of offsets at early lead times for more convective months (both summer and winter)
Less evidence of offsets during autumn (dominated by frontal rain)
SLOW
FAST
© Crown copyright Met Office

9. Evolution of correlation coefficients separated by forecast run
June 2011-Dec 2012
Hourly forecast correlations are separated by initialisation time.
Average correlations as a function of lead time and offset.
Skewed towards model being too slow.
Later lead times often tending towards flat. Forecast error/loss of predictability dominating.
SLOW
FAST
© Crown copyright Met Office

10. Frequency bias as a function of lead time and offsets
Focus on 12-18Z for each model initialisation for afternoon convection.
Hourly forecast frequency bias for 4 mm/h separated by initialisation time.
Generally an over-forecast bias; here slow offsets appear to have larger biases.
0-offset does not necessarily have best bias.
NAE
4 km
© Crown copyright Met Office

11. ETS and SEDS as a function of lead time and offsets
+ve = offset better
Focus on 12-18Z for each model initialisation for afternoon convection.
Hourly ETS or SEDS for 4 mm/h relative to 0-offset, separated by initialisation time.
0-offset is not necessarily the most skilful based on the SEDS or ETS; over-forecast bias does not necessarily translate to maximised scores.
There is no clear signal for any particular offset to be more skilful overall.
© Crown copyright Met Office

12. Concluding remarks
© Crown copyright Met Office

13. Preliminary findings
Start time of the run is influential irrespective of time of year with large variations between different initialisations. Influence of data being assimilated?
Seasonal variations have a strong influence, especially autumn stands out as being different.
Months dominated by more convective rain appear to show a delay in onset of precipitation for the afternoon hours.
Drop in correlation coefficient beyond the first few hours can be marked. Average correlation coefficients show some skewness towards the model being too slow.
Very occasionally better predictability in terms of the pattern is maintained beyond the first 6 hours.
Skill scores show more response to timing errors for larger accumulation thresholds.
Model tends to over-forecast larger accumulations.
© Crown copyright Met Office

14. Thanks for listening!
© Crown copyright Met Office