1. AMV impact studies at the Met Office
James Cotton, Mary Forsythe, Francis Warrick, Richard Marriot
12th International Winds Workshop, Copenhagen, June 2014
© Crown copyright Met Office

2. Overview Forecast Sensitivity to Observations MSG low level winds
Filling the gap
IODC investigation
© Crown copyright Met Office

3. Forecast Sensitivity to Observations (FSO)

4. Impact on 24-hr forecast error - FSO
Global FSO impacts now being calculated on routine basis in NRT
Example from May 2014
Moist total energy norm
Surface to 150 hPa
8.6%
5.5%
Impact of observations on forecast error can be measured by the adjoint-based Forecast error Sensitivity to Observations (FSO) method.
This works by calculating a single, scalar measure of the reduction in forecast error due to the assimilation of observations
We are now able to calculate these impacts on a routine basis in NRT.
Example shown from May 2014 and shows the impact on a 24-hour total (moist) energy norm, from surface to 150 hPa.
Geo AMVs make a significant contribution at 8.6%, Scatterometers 5.5 % and polar winds around 0.5%
AVHRR > MODIS
© Crown copyright Met Office

5. Impact on 24-hr forecast error - FSO
Global FSO impacts now being calculated on routine basis in NRT
Example from May 2014
Moist total energy norm
Surface to 150 hPa
9.1%
If we now combine the AMV impacts then we see that, for May 2014, the AMV impact actually overtakes the Sondes.
For years the AMVs have always been "the icing on the cake" in terms of impact but now they are more substantive which is good.
This is a pretty impressive result for the AMVs, so its worth revisiting how we have got to this point..
© Crown copyright Met Office

6. Impact on 24-hr forecast error - FSO
PS30-like
Jan-Mar 2012
PS32-like
Apr-July 2013
6.7%
4.3%
+Metop-B
The figure on the left shows the impacts from around the time of the last Winds Workshop in 2012.
The Geo AMVs sit in 8th place on 4.3%, behind the scatterometers.
By mid 2013, the Geo AMVs have jumped up to 5th (6.7%) ahead of Scat, AIRS and SYNOP
AVHRR slightly ahead of MODIS winds due to addition of Metop-B (note: have also since lost Terra WV channel winds)
Contributions to the total observation impact on a moist 24-hour forecast-error energy-norm (courtesy of Richard Marriot)
© Crown copyright Met Office

7. Number of ‘soundings’
PS30-like
Jan-Mar 2012
PS32-like
Apr-July 2013
The reason for the large increase in AMV impact is largely due to the increase in the number of AMVs assimilated.
GEO AMVs now equal with Scatwind in terms of number of ‘soundings’ i.e. wind vectors (u/v pair)
NB. Scatwind will have ~2x number of 'observations' due to the ambiguous u/v wind components
i.e. num wind vectors (u/v pair) for AMV and Scatterometers
© Crown copyright Met Office

8. Implementation of temporal thinning
Previously used one wind in each spatial box in the 6 hour window
From PS31 (January 2013) assimilate in 2-hourly time slots
2-3x number of winds
Time relative to Analysis (hrs) -3 -2 -1 1 2 3
2hr Thinning Boxes
Meteosat-7
Meteosat-9
Implementation of 2-hourly thinning leads to 2-3x increase in number of winds
MTSAT-1R/2
GOES-15/13
Polar winds

9. Change in MSG impact AMV impact by satellite
PS30-like
Jan-Mar 2012
PS32-like
Apr-July 2013
Amongst the large improvement in AMV impact from 2012 to 2013, we also observed a large decrease in the impact from MSG.
In mid 2013, MSG actually contributing less than Meteosat-7.
This occurred because we had taken the decision to blacklist the low level IR/VIS winds following the implementation of the CCC scheme at EUMETSAT..
Observed significant drop in relative impact of MSG following blacklist of low level IR and visible channel winds
Contribution similar to Meteosat-7

10. EUMETSAT CCC winds

11. EUMETSAT CCC winds Main changes
Cross Correlation Contribution (CCC) method (Borde and Oyama, 2008)
Maintain closer link between the pixels used in the height assignment with those that dominate in the tracking
Makes direct use of pixel-based cloud top pressures from CLA product rather than generating AMV CTPs
Pre-operational monitoring showed
Significant improvements e.g. high level in the jet regions.
Increase in RMSVD of ~0.6 m/s (20%) for IR and VIS winds at low levels
CCC data became operational on 5 September 2012
Decided to blacklist the low level MSG data (16 Oct 2012)
EUMETSAT ‘fix’ for low level data implemented 16 Apr Impact?
© Crown copyright Met Office

12. Parallel monitoring - June 2012
VIS 0.8, Oper (pre-CCC), QI2>80
VIS 0.8, CCC, QI2>80
CCC higher
Positive speed bias 0-30°S
Negative bias mid latitudes/ Europe
Assigned minus best-fit pressure
Negative, ‘high’ height bias
Shift to higher levels not in agreement with model
This shows some comparisons for the parallel period of data we had in June The operational data at that time (left) and CCC data (right)
With CCC scheme, wind distribution is generally shifted upwards.
This led to
a positive speed bias 0-30S over Atlantic.
A larger negative speed bias over Europe
If we compare to model best-fit pressure we see the CCC data has negative ‘high’ height bias, confirming that the shift to higher levels is not in agreement with the model.
© Crown copyright Met Office

13. Impact experiment at ECMWF
Courtesy Kirsti Salonen (ECMWF)
Significant difference in mean wind analysis at low levels (850 hPa)
Negative impact on forecasts
-> blacklist the low level MSG winds (Sept 2012)
At the time we weren’t able to run an assimilation experiment ourselves, so we relied to some extent on results from ECMWF.
These are some results from Kirsti Salonen.
Assimilating the CCC winds leads to a significant difference in the mean wind analysis at low levels
Strengthening the mean wind field in the same region where we saw an increase in the positive bias
This leads to a negative impact on forecasts (red shading on plot on right)
This was why we decided to blacklist the data
CCC AMVs tends to strengthen the mean wind field
Same region as the increase in positive speed bias for CCC winds
© Crown copyright Met Office

14. Update for low level HA
Height difference mainly occurs in regions with a low level temperature inversion (Borde et al, 2013).
Due to difference in the way the inversion correction is applied in CLA algorithm compared to the pre-CCC AMV algorithm
Pre-CCC
If final EBBT height below 650 hPa and above an inversion then corrected to base of inversion layer
CCC
If EBBT CTH below 650 hPa + temperature inversion is found then height is corrected to 1/3 way above base of inversion layer
CTH can be corrected upwards as well as downwards
Derivation updated on 16 April 2013 so more consistent with old inversion scheme
© Crown copyright Met Office

15. CGMS Time series Met-9 -> Met-9 CCC -> Met-10 CCC Tropics
VIS 0.8
IR 10.8
Best way to analyse impact of the low level update is from long time series.
This shows the time series of Met-9 data (black), Met-9 after the CCC change (blue) and Met-10 CCC (purple)
Solid line is RMSVD, dashed is speed bias
The orange line shows when the low level update was applied
This is for the tropics
Clear increase in RMSVD and bias when CCC introduced (black/blue overlap)
Following the update the stats look much better for the IR (particularly last 6 months or so)
VIS still positive bias, RMS not quite back to pre-CCC levels
Met-9 -> Met-9 CCC -> Met-10 CCC
QI1 > 80 (IR) or QI1 > 65 (vis)
© Crown copyright Met Office

16. CGMS Time series Met-9 -> Met-9 CCC -> Met-10 CCC
Southern hemisphere extra-tropics
VIS 0.8
IR 10.8
In the SH
Bias looks good
RMSVD still not back to pre-CCC level (particularly for VIS)
Met-9 -> Met-9 CCC -> Met-10 CCC
QI1 > 80 (IR) or QI1 > 65 (vis)
© Crown copyright Met Office

17. Assimilation Experiment
45-day experiment to assess impact of reintroducing low level MSG winds to operations
10 July 2013 to 25 Aug 2013
Control: No Met-10 AMVs below 700 hPa
Trial: Assimilate Met-10 IR, VIS, HRVIS below 700 hPa
PS32 configuration, N320 L70, 4D-Var (N108-N216)
We conducted a 45-day assimilation experiment to see the impact of reinstating the low level winds
The period covered 10 July to 25 Aug 2013
We ran a control experiment, as per operations, with the Met-10 data below 700 hPa removed
And then a trial experiment with the data reinstated
© Crown copyright Met Office

18. Number of AMVs assimilated
Control
Trial
In all these plots, blue is the control and red is the trial experiment
20% increase in AMV observations assimilated (from avg. 24,800 to 30,000)
© Crown copyright Met Office

19. AMV zonal wind component O-B
Mean u-wind O-B
RMS u-wind O-B
Control
Trial
RMS O-B ~ 0.2 m/s (6%) lower when the low level MSG winds are included. Mean O-B also shows a small improvement
Similar results for O-A
© Crown copyright Met Office

20. Scatterometer zonal wind O-B
Mean u-wind O-B
RMS u-wind O-B
Control
Trial
RMS O-B for scatterometer winds also slightly lower in the trial (0.5% reduction for U and V)
© Crown copyright Met Office

21. Impact on wind analysis
850 hPa wind field, averaged over trial period
Diff: Green-blue indicates wind speeds are slower in the trial, yellow-red where they are faster.
This shows the impact on the mean wind analysis at 850 hPa
Trade winds in tropical South Atlantic increase by up to 0.7 m/s
Slower in North Atlantic by up to 0.9 m/s
© Crown copyright Met Office

22. Impact on 850 hPa wind forecasts
Zonal wind: T+48 RMS error
blue: positive impact
yellow-red: negative impact
This shows the impact on low level wind forecasts
Areas shaded blue show positive impact, yellow/red shading shows negative impact
Each experiment verified against own analysis
Small region of negative impact ~ 20°S
North Atlantic shows a widespread region of positive impact
© Crown copyright Met Office

23. Change in forecast RMS (NWP Index)
Standard (-0.11)
Alternative (-0.13)
Mixed impacts for PMSL in NH
Tropical winds rather neutral
Improvements for most forecast ranges in SH
AMDAR
sonde
AMDAR
sonde
AMV
sonde
These plots show the change in forecast RMS (verified against obs) for the parameters that make up our Global NWP Index
The different colours represent different forecast ranges
Here we show 2 plots, “standard” and “alternative”
AMDAR
sonde
Positive impact
Positive impact
© Crown copyright Met Office

24. Change in MSG impact (II)
AMV impact by satellite
PS32-like
Apr-July 2013
May 2014
Meteosat-10 impact restored following reintroduction of low level winds in January 2014

25. Filling the gap

26. LeoGeo mixed satellite winds
LeoGeo provides less complete coverage than seen in previous analyses
Loss of GOES-12 (South America) and removal of Met-7, FY-2E (?)
Assimilation trials ongoing with BUFR data

27. EUMETSAT Metop winds EUMETSAT data reaches as far as 50° N/S.
CIMSS data is polewards of ~65°
CIMSS Metop-A/B
EUMETSAT vs NoPolar
CIMSS vs NoPolar
-0.06
-0.15
EUM Metop-A/B
Positive impact
Positive impact

28. Dual Metop-A/B winds Collocation with Meteosat-10
Good vector agreement
More differences in height (HA limitations for AVHRR)

29. Indian Ocean intercomparison

30. Indian Ocean inter-comparison
Report available from NWP SAF investigation web page

31. Summary
Latest FSO statistics show AMVs are having a substantial positive impact on Met Office global NWP
Low level MSG winds show some improvement following update in April O-B’s perhaps not back to pre-CCC levels.
Reinstating the data gives some improvement in forecast RMS scores
AMV O-B and O-A fit improved and small benefit for scatterometer O-B
Reinstated to operations 21 January 2014
Trials with LeoGeo and EUMETSAT Metop winds ongoing
IODC inter-comparison report published on NWP SAF website
Would be good to understand why stats not quite back to pre-CCC levels

32. Thank You
Questions? 32

33. Met Office NWP model suites
Global and MOGREPS-G
25-km in mid latitudes (17-km from July 2014)
70 levels (80-km model top)
Hybrid 4D-Var (60-km)
Analysis times: 0,6,12,18 Z
T+67 forecast twice/day
T+168 (7 day) forecast twice/day
12-member EPS km 4x/day T+168
Euro4
4.4 km, 70 levels (40-km model top)
Global downscaler
T+60 forecast twice/day
T+120 (5 day) forecast twice/day
UKV and MOGREPS-UK
1.5-km, 70 levels (40-km model top)
3D-Var (3 hourly)
T+36 hr forecast 8/day
12-member EPS km 4x/day 36h
Global model
25-km but will be increasing resolution to 17-km from July 2014
European model at 4-km, downscaled from the global
High resolution UK regional model
1.5-km
Short range forecasts over the UK at convection resolving resolution
MOGREPS = Met Office Global Regional Ensemble Prediction System

34. Current operational use of AMVs
Using data from 5 geostationary and 7 polar platforms
Using data from standard set of 5 Geo platforms
For polar winds, using MODIS winds from NESDIS and AVHRR winds from CIMSS (including Metop-A and Metop-B).
NOAA-16 mission ended 6 June 2014.
EUMETSAT
Met-10, Met-7
JMA
MTSAT-2
NESDIS
GOES-13/15, Aqua/Terra MODIS
CIMSS
NOAA-15/16/18/19 AVHRR, Metop-A/B AVHRR
© Crown copyright Met Office

35. EBBT CTH above inversion
CCC
Pre-CCC
Courtesy Regis Borde (EUMETSAT)
© Crown copyright Met Office

36. Impact of low level update
Met-10 visible 0.8, QI2 >80
CCC: month prior
updated-CCC: month after
Updated AMVs assigned lower in atmosphere
Reduced positive speed bias in tropical Atlantic.
Negative speed bias in mid-latitudes also improved.
We can then try and judge the impact by comparing the data before (left) and after the update (right).
As expected the updated AMVs are generally assigned lower
Impact on speed bias looks rather good
Reduced the positive bias in the tropical Atlantic
Negative speed boas in mid latitudes also improved
© Crown copyright Met Office

37. CGMS Time series Met-9 -> Met-9 CCC -> Met-10 CCC
Northern hemisphere extra-tropics
VIS 0.8
IR 10.8
Northern Hemisphere:
Negative bias improved
RMSDV not quite back to pre-CCC (e.g. IR)
Met-9 -> Met-9 CCC -> Met-10 CCC
QI1 > 80 (IR) or QI1 > 65 (vis)
© Crown copyright Met Office

38. Wind verification in VER
Global NWP index verification
250 hPa wind against AMDARS
850 hPa wind against AMVs (tropics only)
Other heights in extended index verified against sondes
AMV observations used to verify the forecasts are those that were assimilated in operations at the time.
The low level MSG winds are blacklisted so have fewer observations to verify against in the region where we are adding AMVs back in.
Still some coverage in this region from GOES-13 and Met-7.
Alternatively, the VER system also allows the option to verify all wind scores against Sondes
© Crown copyright Met Office

39. EBBT CTH below inversion
CCC
Pre-CCC
Red line shows final pressure.
With pre-CCC (right), when we have a CTH below an inversion there is no adjustment
With CCC the pressure is shifted up to 1/3 way up the inversion layer
Courtesy Regis Borde (EUMETSAT)
© Crown copyright Met Office

40. Scatwind FSO
Impact by satellite
January 2014
May 2014

41. Operational changes 2013 Emergency: GOES-13/14 Seasonal: MTSAT
Derivation updates: [GOES hourly product implementation delayed]
New satellites/transitions: Meteosat-10, Metop-B, MTSAT-1R/2
Assimilation changes: temporal thinning, QC, blacklisting
© Crown copyright Met Office