1. Working Group/Observational Data Spring 2015 Co-Chairs: Mr. Dennis Hobson, ACC/A5W Mr. Kyle Rushing, NAVO Exec. Secretary: Mr. Anthony Ramirez, OFCM Mr. Anthony Ramirez, OFCM

2. 2 Overview Scope of Responsibility Scope of Responsibility Members Members Ongoing Activities Ongoing Activities Action Items Action Items Upcoming Activities Upcoming Activities

3. 3 Scope of Responsibility (Modified extract from Terms of Reference) Facilitates the acquisition, processing, and exchange of observational data among the National Operational Processing Centers (OPCs) and other related data centers and supports the Committee for Operational Processing Centers (COPC) through its executive agent, the Working Group for Cooperative Support and Backup (WG/CSAB). The primary focus areas: Facilitate the acquisition, processing and exchange of meteorological, oceanographic, and space environmental data Provide an interface between the OPCs, their research and development partners in the Joint Center for Satellite Data Assimilation (JCSDA), and the other national data and prediction centers for the purpose of coordinating and satisfying national requirements for observational data Coordinate data formatting standards where practical and the implementation of approved data product enhancements and new data products Coordinate observational data issues that overlap related responsibilities of other OFCM committees and COPC groups Ensure the OPCs and related data centers are provided the maximum quality and optimum quantity of observational environmental data streams required for assimilation into their respective processes

4. 4 Member Agencies/Entities NOAA National Environmental Satellite, Data, and Information Service/Office of Satellite and Product Operations (NESDIS/OSPO), Suitland, MD National Centers for Environmental Information (NCEI) Center for Weather and Climate (CWC) (formerly NCDC) National Weather Service (NWS), Silver Spring, MD National Centers for Environmental Prediction (NCEP), College Park, MD o Space Weather Prediction Center (SWPC) o NCEP Central Operations (NCO-Silver Spring) (formerly NWS/Telecommunications Operations Center (TOC)) o Environmental Modeling Center (EMC) National Data Buoy Center (NDBC) Office of Oceanic and Atmospheric Research (OAR)

5. 5 Member Agencies/Entities Air Force HQ AF/A3W, Pentagon, Washington DC HQ Air Combat Command (ACC/A5W OL-P), Offutt AFB, NE 557 th Weather Wing, Offutt AFB, NE 14 th Weather Squadron, Asheville, NC Navy Fleet Numerical Meteorology and Oceanography Center (FNMOC), Monterey, CA. Naval Oceanographic Office (NAVOCEANO), Stennis Space Center, MS Naval Research Laboratory-Marine Meteorology Division Joint Center for Satellite Data Assimilation (JCSDA) (Headquarters), College Park, MD

6. 6 WG/Observational Data Co-Chairs Mr. Dennis Hobson, ACC/A5W Mr. Kyle Rushing, NAVO NCEP Mr. Jeff Ator - EMC Mr. Walter Smith – NCO/SS Air Force Mr. Eric Wise Mr. Dennis Hobson Mr. Al Zamiska 14th WS Mr. Randy Haeberle NESDIS/OSPO Mr. Vincent Tabor FNMOC Mr. James Vermeulen Dr. Justin Reeves -- new hire Dr. Patricia Pauley - NRL NAVO Mr. Kyle Rushing Mr. Keith Willis JCSDA Dr. James Yoe NWS/NDBC Mr. Rex Hervey

7. 7 NOAA/OAR Dr. Stephen Piotrowicz NOAA/IOOS Mr. Derrick Snowden NESDIS/NCEI/CWC (formerly NCDC) Vacant (seeking replacement) NWS/SWPC Dr. Rodney Viereck OFCM Mr. Anthony Ramirez (Exec. Secretary) WG/Observational Data

8. 8 MET-7 Follow-on EUMETSAT Council publicly has stated they do not intend to support IO imaging beyond end of CY 2016 and planned MET-7 end of life/deorbit Mitigation to gap in geostationary imaging of IO region HQ AF directive is to not exploit China/Russia METSAT COMS-1 posn 12 deg further west (128 deg E) than H-8 (140 deg E) Explore quality of INSAT 3-D data received by NOAA on devel sys Potential COA if favorable result: pursue operational feed? Lack of geo imagery will force reliance on polar-orbiting platforms for DoD forecaster application Ongoing Activities

9. 9 Acquisition of INSAT-3D data India National Satellite - 3D is a multi-purpose geostationary satellite with the potential to extend high temporal-resolution geostationary coverage to the Indian Ocean/Arabian Sea region Acquisition effort ongoing (WG/CSAB update in upcoming brief)

10. 10 Ongoing Activities AF/Navy Interest in Operational Feed of NASA Global Precipitation Measurement (GPM) Data Global Precipitation Measurement (GPM) Data GPM Spacecraft launched Feb 27, 2014 AF submitted SDPR Mar 2014 to acquire Level 1b GPM brightness temps and radar enhanced precipitation retrievals; Navy added to SDPR AF using in Climate Prediction Cen Morphing Technique (CMORPH) Replaced TRMM in CMORPH process Navy will use in NAVDAS-AR and tactical imagery applications NOAA partners in JCSDA have access to some GPM/GMI Q3 FY14: GMI L1B, L1C, and L1C-R data had been BUFR-ized Q4 FY14 GMI L1C-R data ingested into GSI, and optimization of GMI for data assimilation is beginning (QC, obs error, bias) Status: NESDIS accelerated CCB process, made GPM available operationally 2 April; TRMM data ceased 9 April AF CMORPH production uninterrupted

11. 11 Ongoing Activities NAVO interest in acquiring Oceansat II data from NOAA for Ocean Color (OCM sensor -- Ocean Color Monitor) NOAA states that the OCM-2 data will not be global Data covers Indian waters only Data covers Indian waters only The onboard recorders are not configured to record globally The onboard recorders are not configured to record globally NAVO to recommend NRL skip incorporation of OCM-2 into AOPS NAVO to recommend NRL skip incorporation of OCM-2 into AOPS Closed Closed

12. 12 DataRequestorProviderStatus GOES-13/15 Level 1c SounderAFWANESDISClosed GCOM-W Lvl 1b and Lvl 2 ProdsAFWA/FNMOCNESDISOpen NASA GPM GMI Lvl 1b Brightness temps AFWA/FNMOCNESDISClosed NASA GPM GMI Lvl 2 Radar Enhanced Precip Retrievals AFWA/FNMOCNESDISClosed MODIS Active Fire ProdAFWANESDISOpen Wildfire Auto Biomass Algorithm (WF-ABBA) AFWA/FNMOCNESDISOpen VIIRS Polar WindsFNMOCNESDISClosed Satellite Data Requests Satellite Data Requests

13. 13 DataRequestorProviderStatus NUCAPS ProductsFNMOCNESDISOpen 1 KM IMSNAVONESDISOpen SMAPAFWANESDISOpen F-19 EDR, SDR, TDR and UPP Products NESDISFNMOCClosed Satellite Data Requests Satellite Data Requests

14. 14. Radar Data Exchange FNMOC interested in acquiring NEXRAD Level II files direct from NOAA vs 557 th WW. FNMOC interested in acquiring NEXRAD Level II files direct from NOAA vs 557 th WW. 557 th WW acquires NEXRAD Level II and III files from NCO SS server using Local Data Manager (LDM); rehosts on 557 th WW server for distribution to mil users which employ Gibson Ridge software. 557 th WW acquires NEXRAD Level II and III files from NCO SS server using Local Data Manager (LDM); rehosts on 557 th WW server for distribution to mil users which employ Gibson Ridge software. Initially, FNMOC was not able to reliably access data via 557 th WW server; 557 th WW identified problem and implemented fix. NEXRAD radar data feed between FNMOC & 557 th WW has been rock-solid and NEXRAD data is being assimilated into FNMOC’s operational COAMPS model. Initially, FNMOC was not able to reliably access data via 557 th WW server; 557 th WW identified problem and implemented fix. NEXRAD radar data feed between FNMOC & 557 th WW has been rock-solid and NEXRAD data is being assimilated into FNMOC’s operational COAMPS model. Reliability/stability of NEXRAD feed from 557 th WW has provided much appreciated “breathing room”, giving IA/IT personnel time to properly engineer a server/storage solution of NEXRAD Level II data at NCO SS. Reliability/stability of NEXRAD feed from 557 th WW has provided much appreciated “breathing room”, giving IA/IT personnel time to properly engineer a server/storage solution of NEXRAD Level II data at NCO SS. FNMOC & NCO SS currently addressing LDM IA concerns and in the process of establishing necessary network protocols via OTN. FNMOC & NCO SS currently addressing LDM IA concerns and in the process of establishing necessary network protocols via OTN. Ongoing Activities

15. 15. Radar Data for Guantanamo Bay Area NCEP/NCO Silver Spring, on behalf of NWS Office of Dissemination, requested radar data for Guantanamo Bay area Gap in the radar mosaic for that area Discoveries: FAA advised that ARSR4 radar at GITMO has a weather channel ARSR4 is FAA ‘owned’ but DOD/DHS funded to keep the systems operational Not Nexrad quality but may be sufficient to cover any gaps in the mosaic Status: NWS/Radar Operations Center coordinating with NCEP/NCO SS to acquire and provide this data stream Action Items

16. 16. Environmental Satellite Data Annex update Background: Annex slightly outdated (5-yr cost projections), GC comments from DAPE MOA, Transition from AFWA IDPS Draft ESDA developed and reviewed by the METSAT members of the WG/OD; submitted to WG/CSAB for the next level of coord Draft ESDA developed and reviewed by the METSAT members of the WG/OD; submitted to WG/CSAB for the next level of coord Several parts of the document (primarily updated funding projections) are yet to be finalized. Several parts of the document (primarily updated funding projections) are yet to be finalized. WG/CSAB update follows this presentation WG/CSAB update follows this presentation Action Items

17. 17. Monitor/coordinate resolution of issues that arise due to WMO text to BUFR transition BACKGROUND: BACKGROUND: WMO is migrating from Traditional Alphanumeric Code (TAC) forms to Binary Universal Form for the Representation of meteorological information (BUFR) WMO is migrating from Traditional Alphanumeric Code (TAC) forms to Binary Universal Form for the Representation of meteorological information (BUFR) OPCs have expected a complicated migration and have been responding to issues as they arise. OPCs have expected a complicated migration and have been responding to issues as they arise. TAC distribution was SCHEDULED to end in November 2014, but… Most (but not all) countries will continue to distribute in BOTH TAC and BUFR for months or years TAC data counts will diminish as countries discontinue TAC distribution If production centers do not have replacement BUFR products, we could experience negative impacts OPCs need to temporarily use BOTH TAC and BUFR data OPCs need to temporarily use BOTH TAC and BUFR data WMO will no longer approve any changes to or additions of TAC code WMO will no longer approve any changes to or additions of TAC code Action Items

18. 18. Several items of concern as the global community transitions to BUFR Several items of concern as the global community transitions to BUFR Regional Telecommunication Hub (RTH) Washington is not in receipt of a significant number of BUFR bulletins from other RTH sites Regional Telecommunication Hub (RTH) Washington is not in receipt of a significant number of BUFR bulletins from other RTH sites Some bulletins not received by RTH Washington since 2011 Some bulletins not received by RTH Washington since 2011 NCO SS extremely responsive in restoring BUFR bulletins onto the GTS once the problem has been identified NCO SS extremely responsive in restoring BUFR bulletins onto the GTS once the problem has been identified As countries transition from TAC to BUFR, transition plans are not well documented nor well distributed. OPCs need: As countries transition from TAC to BUFR, transition plans are not well documented nor well distributed. OPCs need: Transition dates with overlapping TAC and BUFR for a test period Transition dates with overlapping TAC and BUFR for a test period Names of corresponding BUFR bulletin headers to add to GTS subscription Names of corresponding BUFR bulletin headers to add to GTS subscription There is significant concern that OPCs will lose observational data for DA as countries cease transmission of TAC bulletins and fully transition to BUFR unless: There is significant concern that OPCs will lose observational data for DA as countries cease transmission of TAC bulletins and fully transition to BUFR unless: BUFR Transition plans better communicated to OPCs BUFR Transition plans better communicated to OPCs Observational data exchange between RTHs improved Observational data exchange between RTHs improved Items of Concern

19. Other issues related to BUFR migration ICAO has changed the BUFR migration plan ICAO has changed the BUFR migration plan METAR, SPECI, TAF, and SIGMET products will not migrate to BUFR, but rather to XML format METAR, SPECI, TAF, and SIGMET products will not migrate to BUFR, but rather to XML format ICAO allows exchange of XML products starting Nov 2013 through bilateral agreements ICAO allows exchange of XML products starting Nov 2013 through bilateral agreements Mandated exchange in XML format by Nov 2019 Mandated exchange in XML format by Nov 2019 Traditional Alphanumeric Code (TAC) products must continue to be transmitted until 2019 Traditional Alphanumeric Code (TAC) products must continue to be transmitted until 2019 The oceanographic community continues to resist the transition to BUFR The oceanographic community continues to resist the transition to BUFR (as required by WMO for GTS) (as required by WMO for GTS) NetCDF is the long established format for ocean data NetCDF is the long established format for ocean data NDBC is converting multiple user’s Profiling Glider data to BUFR and inserting these data on GTS for the ocean community NDBC is converting multiple user’s Profiling Glider data to BUFR and inserting these data on GTS for the ocean community 19

20. Significant challenges finding BUFR Bulletin listings (especially for US) Significant challenges finding BUFR Bulletin listings (especially for US) Listings for US BUFR bulletins are virtually non-existent through the WMO website Listings for US BUFR bulletins are virtually non-existent through the WMO website WIS/DAR at RTH Washington is confusing and not intuitive for the user for data discovery (i.e., Comprehensive list of Bulletins) WIS/DAR at RTH Washington is confusing and not intuitive for the user for data discovery (i.e., Comprehensive list of Bulletins) There have been random bulletin “drop-outs” where BUFR bulletins mysteriously stop being transmitted on GTS (Drifting Buoy (Global) & Synoptic Land observations (US)) There have been random bulletin “drop-outs” where BUFR bulletins mysteriously stop being transmitted on GTS (Drifting Buoy (Global) & Synoptic Land observations (US)) Outages range from 5-7 days to several weeks Outages range from 5-7 days to several weeks 20

21. 21. Status at FNMOC Status at FNMOC Continuing to add/modify BUFR decoder definition files as required Continuing to add/modify BUFR decoder definition files as required Adapting DA System to exploit additional (resolution, metadata) info Adapting DA System to exploit additional (resolution, metadata) info Identifying data paths (GTS/NOAAPort) for bulletins to ensure receipt of bulletins is via primary data circuit (GTS) Identifying data paths (GTS/NOAAPort) for bulletins to ensure receipt of bulletins is via primary data circuit (GTS) Partnering with NRL and WG/OD members to: Partnering with NRL and WG/OD members to: Identify metadata discrepancies between BUFR (internal) and existing databases Identify metadata discrepancies between BUFR (internal) and existing databases Identify BUFR encoding errors and reporting errors to the international community Identify BUFR encoding errors and reporting errors to the international community

22. 22. Status at NCEP Status at NCEP Negligible disruption for NCEP Central Operations Negligible disruption for NCEP Central Operations NCEP EMC re-directing effort (expected several months) to NCEP EMC re-directing effort (expected several months) to “Homogenize” BUFR data from different international sources; “Homogenize” BUFR data from different international sources; Adapt to higher resolution data available in BUFR than in TAC; Adapt to higher resolution data available in BUFR than in TAC; Perform duplicate checking ACROSS (rather than within) tanks Perform duplicate checking ACROSS (rather than within) tanks Status at NAVO Status at NAVO Comparing ingested BUFR data vs what is ingested at FNMOC Comparing ingested BUFR data vs what is ingested at FNMOC Continuing to enhance BUFR ingest capability for Ocean data Continuing to enhance BUFR ingest capability for Ocean data Searching web for Oceanographic data available real-time (Not on GTS). Continuing to encode releasable data in WMO compliant formats for insertion onto the GTS

23. 23. Status at 557 th Weather Wing Status at 557 th Weather Wing Initially anticipated negligible disruption for 557 th WW NWP production Initially anticipated negligible disruption for 557 th WW NWP production Already processing both BUFR and TAC data Already processing both BUFR and TAC data Crude duplicate elimination has been in place (assimilate the first observation, reject any that follow for the same place and time) Crude duplicate elimination has been in place (assimilate the first observation, reject any that follow for the same place and time) As some countries/organizations transitioned, we found issues with our BUFR decoder. Developing improved decoder now. As some countries/organizations transitioned, we found issues with our BUFR decoder. Developing improved decoder now. More concern for downstream systems who have historically depended on TAC data disseminated from 557 th WW More concern for downstream systems who have historically depended on TAC data disseminated from 557 th WW Assessing potential impact to Joint Environmental Toolkit (JET) at Operational Weather Squadrons Assessing potential impact to Joint Environmental Toolkit (JET) at Operational Weather Squadrons

24. 24. Upper Air BUFR Data Status Migrating U.S. radiosonde data to BUFR Migrating U.S. radiosonde data to BUFR Currently, the TAC TEMP data is converted to BUFR Currently, the TAC TEMP data is converted to BUFR Resulting output is low-resolution and contains numerous conversion and metadata errors, making it of limited benefit to users Resulting output is low-resolution and contains numerous conversion and metadata errors, making it of limited benefit to users High-resolution radiosonde data is available within the RRS (Radiosonde Replacement System) software at each NWS reporting site High-resolution radiosonde data is available within the RRS (Radiosonde Replacement System) software at each NWS reporting site Data is currently only sent to NCEI for archive on delayed schedule Data is currently only sent to NCEI for archive on delayed schedule OPCs have requested access to this data in real-time OPCs have requested access to this data in real-time Expected to be of great benefit for use in NWP Expected to be of great benefit for use in NWP A project is underway within NWS to obtain access to this data in real-time and convert it to a WMO-approved BUFR sequence for dissemination to worldwide OPCs via GTS A project is underway within NWS to obtain access to this data in real-time and convert it to a WMO-approved BUFR sequence for dissemination to worldwide OPCs via GTS Project currently being managed by NCEP Central Operations Project currently being managed by NCEP Central Operations Converted data will meet WMO as well as OPC requirements Converted data will meet WMO as well as OPC requirements Project won’t include data from U.S. military (block 74) radiosonde sites, since those sites don’t have RRS installed Project won’t include data from U.S. military (block 74) radiosonde sites, since those sites don’t have RRS installed

25. 25 The Promise of BUFR Non-operational high-resolution sounding for 72305 (Newport, NC) at 2015010800 BUFR: 1-sec (~5 m) data with balloon drift, all variables at all levels 6373 levels from 1020 mb to 9.8 mb Similar to operational soundings from Europe TAC (TEMP): mix of mandatory and significant mass/wind levels 16 mandatory, 120 significant mass, 36 significant wind levels Launch 500 mb 100 mb 9.8 mb

26. 26 The Reality of BUFR Structure of BUFR radiosonde data received at FNMOC “Proper” BUFR—one message containing the full sounding (purple dots) “BUFR by parts”—one message for each of the four TEMP parts (red squares) NCO’s “legacy” BUFR uses BUFR by parts, as do New Zealand, Australia, China, Japan NCO’s BMT BUFR provides multiple messages, some of which are proper BUFR ECMWF, UKMO, NCEP do not plan to use BUFR by parts (red squares)

27. 27 Errors arising from decoding TAC prior to encoding in BUFR Wrong leading digits for geopotential height Negative 1000 mb heights incorrectly decoded Various errors for tropopause level (variables mixed up, pressures wrong, etc.) Incorrectly decoded pressure for significant levels with pressure > 1000 mb Other errors/problems in BUFR from TAC Negative dewpoint depressions from India Significant level winds from Vietnam divided by two Inconsistencies in reported time/launch time in BUFR by parts TAC errors propagated into BUFR (e.g., pressure divided by 10 for significant winds above 100 mb from India) Missing information Supplemental information for dropsondes not encoded in BUFR Significant level winds from U.S. data in separate BUFR files—converted from PPBB, PPDD PILOT format messages Incomplete BUFR soundings with data filled in from TAC Truncation of significant level data from Australia Missing data above 100 mb (e.g., South Africa) Examples of Data Errors

28. 28 Examples of Metadata Errors Failure to convert degrees-minutes-seconds to decimal degrees – e.g., 44373 (Dalanzadgad, Mongolia) WMO location (in degrees and minutes): 43 35N, 104 25E BUFR location (in decimal degrees): 43.35°, 104.25° FNMOC location (in decimal degrees): 43.58°, 104.42° – All stations in Mongolia and the Philippines have this error Sign error in longitude – e.g., 70414 (Shemya AFB, AK): FNMOC = 174.10° ; BUFR = -174.11° – Also 60018 (Tenerife, Canary Islands), 71603 (Yarmouth, Nova Scotia) Confusion between Hp and Hha elevations – Hp = barometer height, which should be the elevation corresponding to the surface pressure and so should be the one used for radiosonde data – Hha = either Ha (official airport altitude) or H (ground elevation) – e.g., 70350 (Kodiak, AK) WMO VolA gives Hp = 33.8 m and Hha = 5.5 m BUFR uses 34 m = Hp, FNMOC uses 6 m = Hha Hydrostatic estimates of surface elevation from the lowest mandatory level height favor FNMOC Possible adjustment made to station pressure to correct the value to Hha?

29. 29 Yuma Proving Ground (74004) is an example of a station with bad metadata in NCO’s station list and in the WMO VolA station list. YPG: 32.87, -114.33, 131 m FNMOC: 32.867. -114.333, 136 m BUFR: 32.5, -114.0, 231 m WMO: 32.5, -114.0, 231 m The BUFR/WMO location is actually for 74005, with the latitude of 32° 50’ misinterpreted as 32.50 °. Correct location WMO, BUFR location Metadata Error Example

30. 30. Continuing BUFR Migration Monitoring and Collaboration FNMOC, NAVO, NCEP, 557 th WW -- Subgroup -- weekly conference calls – very active since the January FNMOC, NAVO, NCEP, 557 th WW -- Subgroup -- weekly conference calls – very active since the January Sharing and comparing data and metadata error discoveries and tracking to final fix Sharing and comparing data and metadata error discoveries and tracking to final fix Consulting data program managers (e.g. upper air) to pinpoint problems and find solutions Consulting data program managers (e.g. upper air) to pinpoint problems and find solutions Federal Coordinator actively communicating interagency user needs and requirements to appropriate provider agencies Federal Coordinator actively communicating interagency user needs and requirements to appropriate provider agencies Finding appropriate U.S. and WMO data POCs to fix data issues – both U.S. and Global Finding appropriate U.S. and WMO data POCs to fix data issues – both U.S. and Global

31. 31. Federated Gateway WG/OD will assist by gathering representative proxy data to test operational effectiveness of the federated gateway WG/OD will assist by gathering representative proxy data to test operational effectiveness of the federated gateway Product Distribution and Access (PDA) WG/OD will assist in the transition and testing of products distributed by the DAPE Gateway to the PDA WG/OD will assist in the transition and testing of products distributed by the DAPE Gateway to the PDA WG/OD NESDIS member represents WG/OD at ongoing PDA meetings WG/OD NESDIS member represents WG/OD at ongoing PDA meetings Upcoming Activities

32. 32 Thank you -- questions?