“Where America’s Climate and Weather Services Begin” WMO Aircraft Lead Center Work Dr. Bradley Ballish NOAA/NWS/NCEP/EMC/GCWMB 12 April 2013 “Where America’s Climate and Weather Services Begin”

Outline Aircraft data are important for model forecast skill, but there have been many challenging problems to solve involving help from many people of different countries Aircraft data coverage is getting better NCEP monthly aircraft data monitoring reports Special monitoring reports and data alerts Colleagues supporting aircraft data efforts Data quality and QC problems Aircraft temperature bias problems and correction Reject-list optimization Fast alerts for data problems Conclusions and the way forward

Aircraft data are important impart because of their quality, many Impact of GOS components on 24-h ECMWF Global Forecast skill (courtesy of Erik Andersson, ECMWF) AIREP denotes all aircraft data and has large error reduction even though the coverage is not that good compared to satellite data Aircraft data are important impart because of their quality, many problems getting fixed and from expanding data coverage Forecast error reduction contribution (%) AMSUA, IASA and AIRS satellite data have bigger impact than aircraft partly due to their global coverage. Radiosonde (TEMP) data have less impact than aircraft

But Aircraft Coverage Could be Much Better and is Expanding The following data coverage slides from the Navy Research Lab website http://www.usgodae.org/cgi-bin/cvrg_con.cgi, courtesy of Dr. Patricia Pauley*, shows that although aircraft coverage has increased, it still has large areas with no data These slides are for 18Z for just one case, but other times and days will show similar large data voids Currently some Automatic Dependent Surveillance (ADS) data are being put on the GTS in the North Atlantic, see green dots 3 slides later, but there could be much more if all the ADS data were on the GTS * Patricia Pauley has made many contributions to analyzing aircraft data problems and their quality control

Canadian AMDAR data (red dots)are not yet used in the GFS

Green dots show ADS reports These reports may be due to ADS reports due to high frequency Automatic Dependent Surveillance (ADS) reports are going to aircraft traffic control at many places in the world, but little is on the GTS

Location of Southwest ACARS Moisture Reports in NCEP GDAS run for 18Z 15 November 2012 They have better quality than sonde moisture data – coverage is getting better We also have UPS moisture data, but not as much This side shows that the MDCRS moisture data coverage is getting better, with blue around cruise level and red-orange at lower levels

Data Coverage Conclusions Since aircraft data are important for model forecast skill, getting more and better aircraft data coverage will help forecasts and the airlines The WMO and ICAO are working on getting ADS data on the GTS but there are interagency disagreements If more ADS data are on the GTS, centers like NCEP could report on units with data problems, such as temperatures that are too warm which may impact fuel economy The NWS and or WMO are working on getting Hawaiian ACARS and Mexican AMDAR data and more ACARS moisture data

NCEP Monthly Data Monitoring Reports For the US MDCRS data, also referred to as ACARS data, we have the latest 12 months of statistics versus the NCEP model background at website: http://www.nco.ncep.noaa.gov/pmb/qap/acars/ Non-US AMDAR data reports are at: http://www.nco.ncep.noaa.gov/pmb/qap/amdar/ A track-checking code is run on the whole month’s data to diagnose problems like position errors and gross or stuck data The above AMDAR reports and reports to the US airlines contain easy to understand detailed explanations of data problems

Special Monitoring Reports and Alerts As the key person for the lead center on aircraft data, I have tried to be the leader in issuing many special reports and alerts on data problems However, people from other centers sometimes first find problems, such as: In November 2009 Colin Parrett first reported on slow speed biases with the new Chinese AMDAR data as described later These reports and alerts are very important for NWP centers to take correction action and also to get the problems fixed The next slides give a partial list of people and their country or agency that have been involved with finding, analyzing or fixing many aircraft data problems or helping expand data coverage

Colleagues Involved in Finding, Analyzing and Fixing Aircraft Data Problems Australia: Jeff Stickland, Dean Lockett, Mike Berechree, Kelvin Wong and Doug Body ECMWF: Francois Lalaurette, Antonio Garcia, Erik Andersson, Dick Dee, Lars Isaksen and Drasko Vasiljevic UKMET: Colin Parrett European AMDAR: Stewart Taylor and Stig Carlberg South Africa: Gaborekwe Khambule, Mike Edwards, Kobus Olivier and others Japan: Kazutoshi Onogi, Junichi Ishida and others Netherlands: Jitze van der Meulen and Frank Grooters Germany: Clemens Drüe and Axel Hoff France: Herve Benichou Canada: Charles Anderson, Gilles Verner, Yulia Zaitseva and Gilles Fournier China: Xiang Li, Jiangling Xu and others US military: Patricia Pauley, Doug Stewart and Eric Wise US NOAA: William Moninger*, David Helms, George Schmidt, Stan Benjamin, Krishna Kumar, Curtis Marshall, Carl Weiss and others ARINC: Al Homans, Alan Williard and Jeannine Hendricks US airlines: Many people including Carl Knable, Randy Baker and Rick Curtis * Bill Moninger has contributed many papers and reports on aircraft data, and his website is very useful: http://amdar.noaa.gov/java/

Data Quality and QC Problem Examples The following group of slides show or discuss various problems with aircraft data quality These include: Stuck (spuriously constant data) Groups of aircraft with wrong locations Aircraft with temperatures off by a factor of ten Old data sent as current New data may have problems There have been many more and different serious problems with aircraft data in the past that are not well documented or may not even have been discovered since monitoring tools are much better today than years ago These past problems need to be corrected for model climate studies of the past

NCEP GDAS OB – BG Temperature Increments for Aircraft EU4264 with stuck temperatures and altitude 9-10 February 2013 All reports were at exactly 641.7 hPa! SDM deleted temps but not winds Very few reports were rejected by aircraft QC as it does not use background values in stuck data QC decisions Stuck (spuriously constant data) is a common problem for aircraft and marine reports Green means passes Red means ACQC deletes Black SDM deletes Blue VARQC deletes A bigger QC time-window would help

EU4264 report increments shown here are at spurious 641.7 hPa -1000 The GDAS had little impact from these bad winds, but the NAM had impact

NAM may have more impact due to no VARQC and a smaller time-window

Wind increments in knots to background for winds passing NCEP GSI QC in knots for spurious wind directions stuck at 360 degrees 6 to 20 July 2009 for ACARS unit X1B2MIRA Many more winds failed operational QC Only 12 of these passed NRL QC, but that is too many A fast data alert system would help A bigger QC time-window would help

Group track-check error example The blue numbers are vector wind differences to the background in knots 2 1 Group track-check error example 3 My aircraft QC code has segment QC checks For a period of a few years, the South African AMDAR data had groups of reports at wrong locations like group 2 this example. No center’s QC could handle this, so NCEP had them on the reject-list for a long time

Australian aircraft developed the same problem. Dean Lockett of Australian then helped fix the same African problem 2 Wind increments to background (Winc) are in knots and shown in red. Reports in group 2 are all in the wrong location which is a very difficult QC problem for most NWP centers 1 3 AU0137 Locations and Winc 06Z June 2009

For a period of a few months, some European AMDAR reports were bouncing off the equator until a fix was made All of the spurious reports in the left side of this V pattern passed the NRLACQC in the NCEP GDAS Stewart Taylor took actions to get this problem fixed

From 5 to 10 November 2009, there were large spurious impacts on the NCEP GDAS analysis due to new Chinese AMDAR reports with winds in m/sec versus expected knots – If aircraft QC had a history file and logic to delete suspect new data, this could have been prevented International emails triggered by Colin Parrett’s findings led to a fix in days Winds in knots

Aircraft Temperatures with a Factor of 10 Error For a period from May 2011 to September 2012, typically a few European AMDAR aircraft per month had reports with temperatures that appeared to be one tenth of the correct value such as: If the measured T was -50.3, the report would be -5.0 If the measured T was 4.8, the report was 0.5 When this happened and the true temperature was in a range of roughly -5.0 to +5.0, then the reported error is in a range of -4.5 to 4.5 and the data would sometimes pass NCEP QC and impact the analysis a few degrees in some cases over small areas near airports Again this is a case where better QC logic could delete the bad data – for reanalyses, these should be deleted Stewart Taylor of the EU AMDAR program would shutoff such data until the problem was fixed when informed of the problem

Old Data Transmitted as Current In some past cases, hundreds of past aircraft reports were transmitted by mistake as current data Fortunately, this happens rarely, such as twice per year or less Codes that can find this sort of problem are available but not in operations Next slide shows a case with day-old Australian AMDAR data, where NCEP SDM QC actions coupled with the VARQC prevented bad impact Kelvin Wong of Australia made a quick fix for this There are more cases with small amounts of old ACARS transmitted as current These often have no or small impact, but two slides later shows a case with NAM impact

In this case, several hundred day-old Australian AMDAR reports were sent out as current when after a data server went down. NCEP analysis impact was small helped by SDM QC actions

These aircraft wind increments along the California coast are large as they were duplicates of the previous day’s data Next slide shows analysis minus background changes that were large in NAM. They were small in GFS run where either the aircraft QC or VARQC deleted the data Pressures around 670 hPa from ACARS unit P0IKJHBA

NAM analysis is impacted by spurious 20 knot vector wind changes

Temperature Bias Problems Aircraft with excessive temperature biases has been the most common problem for many years In addition, the following group of slides show that aircraft and sondes have different temperature biases The aircraft biases vary with: Aircraft types Phases of flight such as ascent, descent and level Pressure Individual aircraft of the same type Biases can also change suddenly with time Sonde temperature biases vary with sonde types, pressure and different solar angles GPSRO data can help derive the bias corrections

Aircraft vs Sonde NCEP GSI Draws to Temps between 200-300 mb Temperature differences in tenths of a degree C SOND Tdiff (obs-ges) Aircraft Tdiff (obs-ges) -Sonds colder than bg and anl -aircft warmer than anl and bg at jet stream level -large number of aircraft overwhelm the GSI analysis How this varies with altitude SOND Tdiff (obs-anl) Aircraft Tdiff (obs-anl) # Aircraft >> # Sondes, thus warm aircraft data overwhelms the GSI/GFS system and the model is not truth, Nov 2008

From: Sun, B., A. Reale, S. Schroeder, D. Seidel and B. Ballish (2013), Towards improved correction for radiation-induced biases in radiososonde temperature observations, accepted in J. Geophys. Res. Based on collocations with GPSRO data, the US sondes are too cold around 400-200 hPa so bias corrections should not treat sondes or the background as truth RADCOR is needed for sondes and bias corrections for aircraft temperatures

From Ballish and Kumar BAMS(Nov 2008) Biases also vary with ascent or descent These corrections assumed the sondes were correct, which is not always true

Reject-list Optimization Data with bad stats or problems should be added to the reject-list to prevent negative impact on the analyses When are stats bad enough that data should be on the reject-list? It would be good to have a system where adjoints of the analysis and forecast model could estimate impact forecast skill depending on reject-list criterion How quickly can we add or remove data from the reject-list? The next few slides introduce a new fast alert system for aircraft data developed at NCEP

Bad Data Alert Code System The current version of the new codes produce daily stats on aircraft temperatures and winds in 3 pressure levels This includes biases and RMS differences to the model background plus counts of total, rejected and gross observations There are also wind direction stats for one deep level Another code then takes the latest 7 days worth of daily stats and averages the stats going backward one day at a time from the latest day If any of these stats have a combination of bad enough quality plus enough observations, alerts are made Alerts can be to add or remove the aircraft from the reject-list The daily stats in the latest history file are easy to view for checking on the alert decisions So far, the current code is working well but needs more work

Future Updates to Alert Code System Aircraft track-checking will be run first to: Check for counts of track-check errors and stuck data problems Allow for effective thinning of the data so that aircraft with very high reporting rates are not treated as statistically independent data Checks on track-check errors and stuck data rates will be added to the alert codes Moisture stats will be added More tuning will be performed on the alert logic which is a function of the data quality and counts Eventually ARINC could use the alerts for flagging the MCDRS data The system needs to be automated with results shared with other centers and corrective action performed on the problems

New code alerted early on the 17th

Conclusions and the Way Forward Aircraft data are important for model forecast skill even though their coverage is not as good as desired The data coverage is better and growing More aircraft are getting moisture sensors Lead center work supported by many international colleagues is helping to quickly find data and QC problems and fixes, which helps make aircraft data more important Faster alerts are needed to be shared internationally for data problems More sharing of information on optimal quality control, bias correction and analysis use of aircraft data is needed Continue feedback to data providers to get problems fixed Thorough deletion of problem aircraft data in climate runs