GOES-R User Data Types and Structures Andrew Royle GOES-R Ground Segment Project NASA Goddard Space Flight Center Andrew.W.Royle@nasa.gov Presentation #IN41B-1498 GOES Rebroadcast (GRB) provides near real-time data to users via satellite downlink from GOES-R Series spacecraft National Weather Service receives L2+ sectorized cloud and moisture imagery via the Advanced Weather Interactive Processing System (AWIPS) GOES Rebroadcast Characteristics 31Mbps of Level 1b (L1b) data from all GOES-R instruments New types of data (lightning, space weather, additional imager bands) Increased spatial and temporal resolution Data formatted into NetCDF4 GRB Information Packets sent every 5 minutes and include schedule, operational status, INR, and other information relevant to the user community AWIPS Data and Interface Characteristics Low latency data delivered directly to NWS via dedicated interface Configurable based on selected parameters (bit depth, projection, geographic coordinates, periodicity, and spatial resolution) Data transferred in tiles which are individual netCDF4 files that together comprise the entire end product Credit: NASA Credit: NASA Current GOES satellites switch to geographically constrained modes of operation to cover storms such as 2012’s Hurricane Sandy. GOES-R Series satellites will be able to provide 1000 x 1000 km images every 30 seconds while maintaining routine scanning schedules GOES-R Series satellites will carry a series of advanced space weather instruments capable of generating solar imagery such as the above, from NASA’s Solar Dynamic Observatory (SDO) AWIPS Product Metadata Data source Channel ID Satellite ID Latitude/Longitude Map Projection Type Bit Depth Spatial Resolution Observation Time Projection-specific Metadata Position within End Product File (specific to tile-files) Current GOES GVAR GOES Rebroadcast (GRB) Full Disk Image 30 minutes 5 minutes (Mode 4) 15 min (Mode 3) Other modes Rapid Scan, Super Rapid Scan 3000 km x 5000 km CONUS: 5 min 1000 km x 1000 km Mesoscale: 30 sec Polarization None Dual circular polarized Receive Frequency 1685.7 MHz (L-band) 1686.6 MHz (L-band) Packetization GVAR Blocks CCSDS 133.0-B-1 Data Compression Lossless compression Data Rate 2.11 Mbps ~31 Mbps Antenna Coverage Earth coverage to 5° Data Sources Imager and Sounder ABI (16 bands), GLM, SEISS, EXIS, SUVI, MAG Space Weather ~2 Mbps Lightning Data ~0.5 Mbps GOES-R GS Product Distribution CDR GRB Product Metadata Contained in NetCDF Markup Language (NcML) files Instrument specific ABI metadata includes Projection type, time, coordinates, scaling, and per-pixel data quality flag images GLM metadata includes the numbers of events, groups, and flashes; their locations, time, intensity, and size; and data quality information Wallops Command and Data Acquisition Station, Wallops VA OC-3 (155 Mbps) per line DVB-S2 Transmission AWIPS Network Control Facility, Silver Spring MD CADU Stream Generation AOS Space Packets CCSDS Space Packets Remote Backup, Fairmont WV AWIPS Backup Network Control Facility, Fairmont, WV Distributed to NWS field offices via AWIPS Authorized users subscribe to or conduct ad hoc searches for GOES-R end products through the Environmental Satellite Processing and Distribution System (ESPDS) Product Distribution and Access (PDA) capability Finally, GOES-R data is made available to NOAA’s Comprehensive Large-Array Storage System (CLASS) for long-term archive. CLASS data includes the L1b and L2+ products sent to PDA, along with the Level 0 data used to create these products, and other data used for product generation and processing. This paper will provide a summary description of the data types and formats associated with each of the four primary distribution pathways for user data from GOES-R. It will discuss the resources that are being developed by GOES-R to document the data structures and formats. It will also provide a brief introduction to the types of metadata associated with each of the primary data flows. A secondary pathway for the user community to receive data terrestrially is via the GOES-R Access Subsystem (GAS), which is being developed as a part of NOAA’s Environmental Satellite Processing and Distribution System (ESPDS) Product Distribution and Access (PDA) capability. The ESPDS PDA will service the NWS and other meteorological users through a data portal, which provides both a subscription service and an ad hoc query capability. The PDA includes both the key cloud and moisture imagery data provided via the AWIPS interface and other higher order Level 2 and beyond (L2+) products. Data may be subscribed to either manually or by taking advantage of the PDA’s application programming interface (API). Data is made available for ad hoc queries for up to seven days after it is created. Terrestrially, meteorological data is provided to forecasters at the National Weather Service (NWS) through a direct interface to the Advanced Weather Interactive Processing System (AWIPS). The direct AWIPS interface ensures that time-critical cloud and moisture imagery data from the satellite’s Advanced Baseline Imager (ABI) is provided through a low latency, high availability pathway. The GOES-R GS also provides a configurable product sectorization and data formatting capability for AWIPS data, which allows for data to be reformatted prior to distribution to meet operational forecast requirements. The National Oceanic and Atmospheric Administration (NOAA) and National Aeronautics and Space Administration (NASA) are jointly developing the next-generation series of Geostationary Operational Environmental Satellites (GOES), known as the GOES-R series. GOES-R will provide meteorological data to the science and operational weather forecast community through the GOES-R Ground Segment (GS). The GOES-R GS is an integrated, end-to-end system that supports the collection, processing, and dissemination of atmospheric data, as well as providing the command and control of the GOES-R Series satellite constellation.   GOES-R meteorological data is provided to the operational and science user community through four main distribution mechanisms. The GOES-R GS generates a set of Level 1b (L1b) data from each of the six primary satellite instruments and formats the data into a direct broadcast stream known as GOES Rebroadcast (GRB). GRB is transponded via the GOES-R series satellites and made freely available to the public within each satellite’s broadcast coverage area. GRB Data GRB Hdr CCSDS Hdr GRB Data GRB Hdr CCSDS Hdr GRB Data GRB Hdr CCSDS Hdr GRB Block (compressed) GRB Block (compressed) PDA Data and Interface Characteristics Subscription and ad hoc query service available to registered users Provides most recent seven days of data for ad hoc queries and retrieval Primary distribution pathway for L2+ end-products other than Cloud and Moisture Imagery Products sent from GOES-R in NetCDF4; PDA has McIDAS translation capability GRB Block (compressed) GRB Block (compressed) GRB Block (compressed) GRB Block (compressed) Decompression L1b ABI Block GLM, SEISS, SUVI, MAG, EXIS data (each separately delivered) L1b ABI Block L1b ABI Block NetCDF4 Characteristics Machine-independent format for representing science data in an array structure Self-describing format enabling user-defined data types Appendable to include metadata alongside science data Built-in zlib algorithm enables compression transparent to user NetCDF4 enables use of HDF5 data format NetCDF Files NetCDF Files L1b and L2+ GOES-R Products available through PDA (and CLASS) PDA Interface Characteristics Low latency distribution for all GOES-R End Products Subscription and ad hoc query capability Application Programming Interface for users Limited to authorized NOAA users Data in NetCDF format; translation capability to McIDAS # # Also distributed from PDA in FITS format NOAA’S Comprehensive Large Array-Data Stewardship System (CLASS) enables archive of GOES-R data for retrieval and retrospective analysis CLASS Data and Interface Characteristics Long-term archive for GOES-R data Unique source of certain GOES-R data types (L0 data, algorithm packages) Data is sent to CLASS Receipt Nodes at each GOES-R facility and is then ingested and maintained by NOAA’s National Climatic Data Center (NCDC) and National Geophysical Data Center (NGDC) More information on NetCDF4 available at: http://www.unidata.ucar.edu/software/netcdf/ Unidata NetCDF4 Data Model  Network Common Data Format (NetCDF4) forms a standard underlying structure for all GOES-R data products National Climatic Data Center, Asheville NC National Geophysical Data Center, Boulder CO Data Type Data Format Product Data Elements Level 0 Data CCSDS in NetCDF4 Level 1b Data NetCDF4 Level 2+ Data Non-Product Data Elements Ancillary Data Native Format L0 Instrument Calibration Data L1b Instrument Calibration Data Binary L1b Processing Parameters L2+ Processing Parameters .zip files Algorithm Packages GRB Information Packets XML ISO Series Level Metadata Delivery Notifications Submission Manifest Files File Activity Reports Manifest Activity Reports Error Notifications SOAP message NOAA Satellite Operations Facility, Suitland MD GOES-R data and metadata characteristics are documented in: GOES-R Product Definition and Users’ Guide Separate volumes for L0, L1b, GRB, and L2+ Ground Segment Interface Control Documents for AWIPS, CLASS, and PDA GOES-R Metadata Model GRB Downlink Specification for Users See www.goes-r.gov for further details and to obtain copies of selected GOES-R documentation CLASS Nodes Wallops Command and Data Acquisition Station, Wallops VA L2+ Products (other than KPP and GLM) CLASS Receipt Nodes (at each facility) L0, L1b, L2+ KPP & GLM products GRB Info Pkts Inst. Cal. Data NOAA Satellite Operations Facility, Suitland MD Remote Backup, Fairmont WV Colors correspond to GOES-R instruments Data is sent from GOES-R to CLASS Receipt Nodes at each GS facility