Testing the assimilation of highly sampled aircraft wind observations (GADS) in the ECMWF global model Workshop on Wind Profiles and Mesoscale Data Assimilation, Ljubljana, 19-20 September 2016 Michael Rennie, ECMWF In collaboration with Joel Tenenbaum (State University of New York (SUNY)) and Lars Isaksen (ECMWF)

Outline What is GADS? Assimilation tests and initial OSEs OSEs with improved QC Latest OSE results

What is GADS? Global Aircraft Data Set Flight data recordings of British Airways Boeing 747 and 777 aircraft Long-haul flights Most data at cruise level – but also ascent/descent Data available mostly in Northern Hemisphere, but some flights to Australia, S. America and S. Africa Different modes: Mode B: Wind reported every 4 s (~0.9 km horizontal sampling) Mode C: every 32 s (~7 km) 82% is mode B data Provides wind and temperature “point” observations < 4 km res., 1 m/s accuracy, see Rukhovets and Tenenbaum (1998) With “pressure altitude”, lat, lon, date/time Basically the same observation method as Aircraft Meteorological DAta Relay (AMDAR) but AMDAR typical sampling ~100 km

Where is the data from? Provided by Joel Tenenbaum (SUNY) to ECMWF for a scientific research collaboration project Only a limited subset is available to ECMWF December 2010, January and February 2011 provided Data is in a bespoke ASCII format Geolocation, temperature, wind speed, wind direction, turbulence information Investigations have focussed on using wind observations from GADS Test impact of assimilating GADS without any thinning or superobbing Assess what scales the model can represent

Section of example flight: GADS wind is provided as speed and direction 50 km Can see rounding precision error in wind speed (nearest knot) Wind speed / ms-1 Wind variability at small horizontal scales evident: thought to be real wind variability, rather than noise To the nearest knot

Wind direction 370 Wind direction/ degrees

GADS: example coverage in 12 hour period Typically ~125,000 records (u, v and T) in a 12 hours period Very high density, but only BA flight paths

Try assimilating GADS winds as if Aeolus HLOS winds Convert GADS winds to HLOS wind u-component to HLOS wind in +ve u direction Technically easier to use as-if Aeolus data Similar thing done with conventional winds in e.g. Horanyi et al. 2014 Observing System Experiment: Control: Normal IFS setup Experiment: control + assimilate GADS HLOS wind (zonal) Jan-Feb 2011 dataset No thinning or superobbing of GADS 4D-Var, global Climatological B-matrix; EDA first implemented in June 2011

Typical vertical sampling with GADS as zonal HLOS wind during 4D-Var window

Typical O-B of GADS zonal HLOS wind m/s

Need for bespoke QC of GADS data Early experiments gave negative impact After investigating found some gross errors in dataset e.g. values out of physical range, wrong date-times, spurious repeated values So implemented a careful pre-assimilation QC of the data

OSE in late 2015: GADS OSE ECMWF CY42R1 T511 outer loop (~40 km grid) T255 (~80 km grid) final of three inner loops (analysis increments) Bespoke GADS QC applied to reject any suspicious data; prior to creation of ODB Also, reject observations with p > 500 hPa since often find spurious data recorded when aircraft on ground Assigned observation error 2.5 m/s – similar weight given to AMDAR Global aircraft dataset, BA long-haul flights

Example GADS assimilation results DA during window for cycle 2011/01/04 00Z ~400 km Analysis pulls strongly to GADS, but doesn't fit small-scale variability e.g. < 100 km 16/05/2014 L2B/C PM31 L2B/C update during 4D-Var window

Example: jet stream maximum wind speed underestimated in background forecast ~400 km ~10 m/s during 4D-Var window

Example GADS data with high wind variability ~400 km ~400 km during 4D-Var window

2.5 m/s OSE: Comparing “normal” aircraft to GADS. O-B, O-A statistics Northern Hemisphere stats O-A “Normal” aircraft u-wind GADS O-B st. dev. is similar to “normal” aircraft Analysis pulls more strongly to GADS than “normal” aircraft m/s GADS HLOS wind (in u dir.) m/s

2.5 m/s OSE: fit to other observations; experiment-control; O-B, O-A std. dev. GPSRO AMSU-A O-A O-B Statistically significant degradation in fit of background forecast and analysis to other observation types Not good! Conventional u-wind, not including GADS

2.5 m/s OSE: Forecast impact of GADS Change in RMSE (verified against own analysis), red is worse Wind velocity Very large change in analysis in NH Little statistical significance in longer range i.e. neutral

2.5 m/s GADS OSE summary GADS with 2.5 m/s assigned obs error makes substantial changes to the analysis in Northern Hemisphere Large degradation in O-Bs compared to other important observations seen Pulling strongly to GADS data; noisy analysis in the vicinity Neutral impact at longer-range in 2 month OSE Why negative impact? Suspect too much weight given to GADS, perhaps due to not accounting for correlated representativeness error and/or correlated instrument error

Recent results: GADS OSE in Sep. 2016 Same settings as before; apart from: ECMWF CY43R1 TcO399 outer loop; ~28 km grid T255 inner loop (analysis increment grid); ~80 km grid Try again with significantly less weight given to GADS Assign inflated obs error; 10 m/s to compensate for correlated errors

10 m/s OSE: Comparing “normal” aircraft to GADS. O-B, O-A statistics Northern Hemisphere stats O-A “Normal” aircraft u-wind GADS O-B st. dev. is similar to “normal” aircraft Still pulls to closely to GADS but less so < 300 hPa m/s GADS HLOS wind (in u dir.) m/s

2.5 m/s OSE: Comparing “normal” aircraft to GADS. O-B, O-A statistics Northern Hemisphere stats O-A “Normal” aircraft u-wind m/s GADS HLOS wind (in u dir.) m/s

10 m/s OSE: fit to other observations; experiment-control; O-B, O-A std. dev. GPSRO AMSU-A O-A O-B Conventional wind Fairly neutral impact here; much better than 2.5 m/s OSE

10 m/s OSE: Further obs types Some improvements in fit to other observations in O-B statistics Radiosondes T Radiosondes q AMVs SSMIS

2016 OSE: Forecast impact of GADS Change in RMSE (verified against own analysis), red is worse, blue is good Geopotential height Wind velocity Neutral Need longer period Don’t see overfitting effect at short FC range

Conclusions from GADS OSEs (so far) Bespoke QC of GADS is critical to avoid some gross errors 4D-Var appears to handle dense dataset well Is not trivial to get positive impact from densely sampled and (apparently) high accuracy dataset Assume because of limited geographical coverage over already well observed flight paths and using incorrect R matrix With inflated obs error (10 m/s) get neutral/slightly positive impact Slightly positive against some other obs types Mitigating correlated representativeness error (or even correlated instrument error)? ECMWF OSEs typically need several months to get stat. significant results Just started experiments with modifications in hope of getting a stronger signal: Keep 10 m/s assigned obs error Both u and v wind components  HLOS winds Use 3 rather than 2 month OSE, Dec 2010 – Feb 2011, 47M obs

Any questions?

Assessment of 2014 GADS OSE results Analysis creates reasonable looking wind structures near the GADS data at >100 km scales Issue: O-B std. dev. ~4-5 m/s for GADS AMDAR O-B st. dev. ~2-3 m/s. Suggests problem with GADS, since same basic observation as GADS GADS O-A O-B m/s m/s

2014 OSE impact results Impact from GADS was v. negative Reason: GADS O-B statistics significantly worse than AMDAR data Should be similar, since effectively the same observations just sampled a lot more! On closer inspection of O-B results Turns out there some erroneous data e.g. values out of physical range, wrong date- times, spurious repeated values Hence worked on bespoke pre-assimilation QC to select only the valid data – this was a lot of work!

Late 2015 OSE: Forecast impact of GADS Change in RMSE (verified against own analysis), red is worse Geopotential height Wind velocity Very large change in analysis in NH Little statistical significance in longer range i.e. neutral

Change in RMSE vs forecast range Geopotential height Vector wind