1. Performance of the MOGREPS Regional Ensemble
Sarah Beare (nee John)
Neill Bowler, Marie Dando, Anette Van der Wal, Rob Darvell
© Crown copyright

2. Outline Overview of MOGREPS Configuration
Overview of probabilistic verification
Results
Summary and future work
© Crown copyright

3. MOGREPS-Regional Regional Ensemble
Regional ensemble over N. Atlantic and Europe (NAE)
23 perturbed members + control
IC Perturbations from regional ETKF
LBC from global ensemble
Stochastic physics
- random parameters
24km resolution, 38 levels
Run at 6Z & 18Z to T+54
Our current suite of operational NWP models are shown here along with future configurations. The numbers refer to typical horizontal grid spacings in mid-latitudes. This grid spacing is often referred to as resolution.
© Crown copyright

4. MOGREPS Operational System diagram
New global analysis
Global ensemble forecast using stochastic physics
LBCs
Perturbations mixed and scaled by ETKF
New NAE analysis
New NAE analysis 0Z
06Z
12Z
18Z
© Crown copyright

5. MOGREPS Verification

6. Verification Aim to verify products issued from MOGREPS system
Two systems available
Focus on surface parameters and higher thresholds
Further results available in verification report recently published on met office website.
© Crown copyright

7. Site-specific Verification system
- 79 sites across the UK and Europe.
Probabilistic forecast data binned into 10% categories
Ability to compare with ECMWF forecasts
- Ability to compare RAW model forecasts and post-processed data
Can calculate and plot :
Reliability diagrams,
Empirical ROC Curves,
Brier Score and decompositions
Brier Skill Score
CRPS in the process of being added
© Crown copyright

8. Area Based Verification System
Can verify against observations or analyses
- Forecast probabilities not binned
Can calculate and plot :
Reliability and attributes diagrams,
Empirical and parametric ROC,
Economic value,
Brier Score and decompositions
Brier Skill Score
Spread and Skill based measures
Would like to include the RPS, CRPS
Can compare Global v Regional ensemble.
THORPEX group introducing ECMWF EPS data for comparison purposes (no facility to account for different ensemble sizes).
© Crown copyright

9. Performance of ensemble and high-res (12km)
Ensemble mean has lower RMSE than high-res for day 2 forecasts, indicating presence of substantial uncertainties
10m w/s
2m Temp
Verification for UK stations for Dec 2006 to Feb T+48 data only available after 6 Feb
© Crown copyright

10. Spread-skill relationship – wind speed T+30
Spread-skill for wind-speed binned into equal population bins by spread
Skill corrected for observation error
Blue – MOGREPS
Pink – No Skill
Green – Perfect
DJF 06/07
JJA 06
© Crown copyright

11. Spread-skill relationship – temperature
Spread-skill for temperature binned into equal population bins by spread
Skill corrected for observation error
Blue – MOGREPS
Pink – No Skill
Green – Perfect
DJF 06/07
JJA 06
© Crown copyright

12. Temperature

13. Temp DJF06/07 (79 sites UK & Europe)
Temp < 0oC T+36
Temp < -2oC T+36
© Crown copyright

14. Brier Skill Score components - Temperature > 10oC 6 Nov 2006 - 28 Feb 2007
Temp>10C
79 sites UK & Europe
6 Nov 2006 – 28 Feb 2007
Resolution
Brier Skill score
Reliability
© Crown copyright

15. Reliability diagram for surface temperature
Reliability diagram for Temp>10C
79 sites UK & Europe
6 Nov 2006 – 28 Feb 2007
© Crown copyright

16. Temperature > 10oC KFMOS
Kalman-filter bias-corrected forecasts.
Resolution
Brier Skill score
Reliability
© Crown copyright

17. Precipitation

18. Brier Skill Score components for Precipitation
12-hour precip>0.5mm
79 sites UK & Europe
1 July 2006 – 31 March 2007
12-hour precip>5mm
© Crown copyright

19. Reliability diagram for precipitation
Reliability diagram for 12h precip>5mm
79 sites UK & Europe
1 July 2006 – 31 March 2007
© Crown copyright

20. 6hr precip > 0.3mm against gridded analysis
Reliability and sharpness diagram for T+36 forecast.
6h precip > 0.3mm
Verification against Nimrod Analysis over the UK at 1.0 degree resolution.
1st Jan 06 – 28th Feb 07
© Crown copyright

21. 6hr precip 5mm against gridded analysis
Reliability and sharpness diagram for T+36 forecast.
6h precip > 5mm
Verification against Nimrod Analysis over the UK at 1.0 degree resolution.
1st Jan 06 – 28th Feb 07
© Crown copyright

22. 6hr precip 25mm against gridded analysis
Reliability and sharpness diagram for T+36 forecast.
6h precip > 25mm
Verification against Nimrod Analysis over the UK at 1.0 degree resolution.
1st Jan 06 – 28th Feb 07
© Crown copyright

23. Wind speed

24. Reliability diagram for wind speed
Reliability diagram for wind speed >F5 at T+30
79 sites UK & Europe
6 Nov 2006 – 31 March 2007
© Crown copyright

25. Wind speed at least gale force 8
Reliability and sharpness diagram for T+36 forecast.
10m Wind > F8
Verification against surface obs over UK and Europe.
1 Jan 06 – 28 Feb 07
© Crown copyright

26. Wind speed at least severe gale force 9
Reliability and sharpness diagram for T+36 forecast.
10m Wind > F9
Verification against surface obs over UK and Europe.
1 Jan 06 – 28 Feb 07
© Crown copyright

27. Wind speed at least storm force 10
Reliability and sharpness diagram for T+36 forecast.
10m Wind > F10
Verification against surface obs over UK and Europe.
1 Jan 06 – 28 Feb 07
© Crown copyright

28. Summary of verification
Ensemble mean beats control and hi-res from ~18h
Very good reliability for non-extreme events
Useful skill for some more severe events
T<-2C
Windspeed > F9
6h Precip > 5mm/12h
KFMOS bias correction effective in reducing many biases
Better forecasts from NAE than ECMWF, even after KFMOS for some parameters
Combination of resolution and perturbation strategy
Useful spread-skill in many cases
Exception is temperature in summer
© Crown copyright

29. Future Plans Fully operational early 2008
SKEB2 scheme including convection for improved stochastic physics
Vertical localization of ETKF to improve near-surface perturbation scaling
Perturbations to lower boundary (eg soil moisture, SST)
Enhanced resolution:
Global 60km from 2009
Regional 16km from 2009 and 12 from 2011
Small convection-resolving ensemble from 2011
~1.5km over UK
~6 members
Nested on members selected by clustering
© Crown copyright

30. Any Questions?
© Crown copyright

31. More Information…. Technical Reports
The MOGREPS short-range ensemble prediction system: Verification
report - Trial Performance of MOGREPS January March 2007
N Bowler, M Dando, S Beare and K Mylne (2007)
497 The MOGREPS short-range ensemble prediction system Part 1: System Description
N Bowler, A Arribas, K Mylne and K Robertson, (2007)
Comparison of error breeding, singular vectors,random perturbations and
ensemble Kalman filter perturbation strategies on a simple model.
N Bowler (2004)
452 Results of an initial stochastic physics scheme for the Met Office Unified
Model. A. Arribas (2004)
© Crown copyright

32. NEW! Regional ETKF
Until end of May MOGREPS Perturbations for the NAE were interpolated from the global at T+6
Large scale perturbations
Spread too large (especially at upper levels)
Since beginning of June perturbations to regional ensemble generated by a regional ETKF.
Smaller, more realistic perturbations
Better representation of important short length scales
But surface perturbations may be too small
© Crown copyright

33. Subjective Evaluation of Severe Weather
Forecaster comments/feedback
Severe Gale Event – 02/09/06
…. the NAE MOGREPS output clearly captured the strength and gave a good forecast for potential severe gale gusts…. (Paul Davies, Chief Forecaster)
Intense rainfall - 20/07/07
… The risk of very high totals is confirmed by NAE MOGREPS members and also other models… (Steve Willington, Chief Forecaster)
© Crown copyright