1. GOES-R User Data Types and Structures
Andrew Royle GOES-R Ground Segment Project NASA Goddard Space Flight Center
Presentation #IN41B-1498
GOES Rebroadcast (GRB) provides near real-time data to users via satellite downlink from GOES-R Series spacecraft
National Weather Service receives L2+ sectorized cloud and moisture imagery via the Advanced Weather Interactive Processing System (AWIPS)
GOES Rebroadcast Characteristics
31Mbps of Level 1b (L1b) data from all GOES-R instruments
New types of data (lightning, space weather, additional imager bands)
Increased spatial and temporal resolution
Data formatted into NetCDF4
GRB Information Packets sent every 5 minutes and include schedule, operational status, INR, and other information relevant to the user community
AWIPS Data and Interface Characteristics
Low latency data delivered directly to NWS via dedicated interface
Configurable based on selected parameters (bit depth, projection, geographic coordinates, periodicity, and spatial resolution)
Data transferred in tiles which are individual netCDF4 files that together comprise the entire end product
Credit: NASA
Credit: NASA
Current GOES satellites switch to geographically constrained modes of operation to cover storms such as 2012’s Hurricane Sandy. GOES-R Series satellites will be able to provide 1000 x 1000 km images every 30 seconds while maintaining routine scanning schedules
GOES-R Series satellites will carry a series of advanced space weather instruments capable of generating solar imagery such as the above, from NASA’s Solar Dynamic Observatory (SDO)
AWIPS Product Metadata
Data source
Channel ID
Satellite ID
Latitude/Longitude
Map Projection Type
Bit Depth
Spatial Resolution
Observation Time
Projection-specific Metadata
Position within End Product File (specific to tile-files)
Current GOES GVAR
GOES Rebroadcast (GRB)
Full Disk Image
30 minutes
5 minutes (Mode 4)
15 min (Mode 3)
Other modes
Rapid Scan, Super Rapid Scan
3000 km x 5000 km CONUS: 5 min
1000 km x 1000 km Mesoscale: 30 sec
Polarization
None
Dual circular polarized
Receive Frequency
MHz (L-band)
MHz (L-band)
Packetization
GVAR Blocks
CCSDS B-1
Data Compression
Lossless compression
Data Rate
2.11 Mbps
~31 Mbps
Antenna Coverage
Earth coverage to 5°
Data Sources
Imager and Sounder
ABI (16 bands), GLM, SEISS, EXIS, SUVI, MAG
Space Weather
~2 Mbps
Lightning Data
~0.5 Mbps
GOES-R GS Product Distribution CDR
GRB Product Metadata
Contained in NetCDF Markup Language (NcML) files
Instrument specific
ABI metadata includes Projection type, time, coordinates, scaling, and per-pixel data quality flag images
GLM metadata includes the numbers of events, groups, and flashes; their locations, time, intensity, and size; and data quality information
Wallops Command and Data Acquisition Station, Wallops VA
OC-3 (155 Mbps) per line
DVB-S2 Transmission
AWIPS Network Control Facility, Silver Spring MD
CADU Stream Generation
AOS Space Packets
CCSDS Space Packets
Remote Backup, Fairmont WV
AWIPS Backup Network Control Facility, Fairmont, WV
Distributed to NWS field offices via AWIPS
Authorized users subscribe to or conduct ad hoc searches for GOES-R end products through the Environmental Satellite Processing and Distribution System (ESPDS) Product Distribution and Access (PDA) capability
Finally, GOES-R data is made available to NOAA’s Comprehensive Large-Array Storage System (CLASS) for long-term archive. CLASS data includes the L1b and L2+ products sent to PDA, along with the Level 0 data used to create these products, and other data used for product generation and processing.
This paper will provide a summary description of the data types and formats associated with each of the four primary distribution pathways for user data from GOES-R. It will discuss the resources that are being developed by GOES-R to document the data structures and formats. It will also provide a brief introduction to the types of metadata associated with each of the primary data flows.
A secondary pathway for the user community to receive data terrestrially is via the GOES-R Access Subsystem (GAS), which is being developed as a part of NOAA’s Environmental Satellite Processing and Distribution System (ESPDS) Product Distribution and Access (PDA) capability. The ESPDS PDA will service the NWS and other meteorological users through a data portal, which provides both a subscription service and an ad hoc query capability. The PDA includes both the key cloud and moisture imagery data provided via the AWIPS interface and other higher order Level 2 and beyond (L2+) products. Data may be subscribed to either manually or by taking advantage of the PDA’s application programming interface (API). Data is made available for ad hoc queries for up to seven days after it is created.
Terrestrially, meteorological data is provided to forecasters at the National Weather Service (NWS) through a direct interface to the Advanced Weather Interactive Processing System (AWIPS). The direct AWIPS interface ensures that time-critical cloud and moisture imagery data from the satellite’s Advanced Baseline Imager (ABI) is provided through a low latency, high availability pathway. The GOES-R GS also provides a configurable product sectorization and data formatting capability for AWIPS data, which allows for data to be reformatted prior to distribution to meet operational forecast requirements.
The National Oceanic and Atmospheric Administration (NOAA) and National Aeronautics and Space Administration (NASA) are jointly developing the next-generation series of Geostationary Operational Environmental Satellites (GOES), known as the GOES-R series. GOES-R will provide meteorological data to the science and operational weather forecast community through the GOES-R Ground Segment (GS). The GOES-R GS is an integrated, end-to-end system that supports the collection, processing, and dissemination of atmospheric data, as well as providing the command and control of the GOES-R Series satellite constellation.
GOES-R meteorological data is provided to the operational and science user community through four main distribution mechanisms. The GOES-R GS generates a set of Level 1b (L1b) data from each of the six primary satellite instruments and formats the data into a direct broadcast stream known as GOES Rebroadcast (GRB). GRB is transponded via the GOES-R series satellites and made freely available to the public within each satellite’s broadcast coverage area.
GRB Data GRB Hdr CCSDS Hdr
GRB Data GRB Hdr CCSDS Hdr
GRB Data GRB Hdr CCSDS Hdr
GRB Block (compressed)
GRB Block (compressed)
PDA Data and Interface Characteristics
Subscription and ad hoc query service available to registered users
Provides most recent seven days of data for ad hoc queries and retrieval
Primary distribution pathway for L2+ end-products other than Cloud and Moisture Imagery
Products sent from GOES-R in NetCDF4; PDA has McIDAS translation capability
GRB Block (compressed)
GRB Block (compressed)
GRB Block (compressed)
GRB Block (compressed)
Decompression
L1b ABI Block
GLM, SEISS, SUVI, MAG, EXIS data (each separately delivered)
L1b ABI Block
L1b ABI Block
NetCDF4 Characteristics
Machine-independent format for representing science data in an array structure
Self-describing format enabling user-defined data types
Appendable to include metadata alongside science data
Built-in zlib algorithm enables compression transparent to user
NetCDF4 enables use of HDF5 data format
NetCDF Files
NetCDF Files
L1b and L2+ GOES-R Products available through PDA (and CLASS)
PDA Interface Characteristics
Low latency distribution for all GOES-R End Products
Subscription and ad hoc query capability
Application Programming Interface for users
Limited to authorized NOAA users
Data in NetCDF format; translation capability to McIDAS #
# Also distributed from PDA in FITS format
NOAA’S Comprehensive Large Array-Data Stewardship System (CLASS) enables archive of GOES-R data for retrieval and retrospective analysis
CLASS Data and Interface Characteristics
Long-term archive for GOES-R data
Unique source of certain GOES-R data types (L0 data, algorithm packages)
Data is sent to CLASS Receipt Nodes at each GOES-R facility and is then ingested and maintained by NOAA’s National Climatic Data Center (NCDC) and National Geophysical Data Center (NGDC)
More information on NetCDF4 available at:
Unidata NetCDF4 Data Model
 Network Common Data Format (NetCDF4) forms a standard underlying structure for all GOES-R data products
National Climatic Data Center, Asheville NC
National Geophysical Data Center, Boulder CO
Data Type
Data Format
Product Data Elements
Level 0 Data
CCSDS in NetCDF4
Level 1b Data
NetCDF4
Level 2+ Data
Non-Product Data Elements
Ancillary Data
Native Format
L0 Instrument Calibration Data
L1b Instrument Calibration Data
Binary
L1b Processing Parameters
L2+ Processing Parameters
.zip files
Algorithm Packages
GRB Information Packets
XML
ISO Series Level Metadata
Delivery Notifications
Submission Manifest Files
File Activity Reports
Manifest Activity Reports
Error Notifications
SOAP message
NOAA Satellite Operations Facility, Suitland MD
GOES-R data and metadata characteristics are documented in:
GOES-R Product Definition and Users’ Guide
Separate volumes for L0, L1b, GRB, and L2+
Ground Segment Interface Control Documents for AWIPS, CLASS, and PDA
GOES-R Metadata Model
GRB Downlink Specification for Users
See for further details and to obtain copies of selected GOES-R documentation
CLASS Nodes
Wallops Command and Data Acquisition Station, Wallops VA
L2+ Products (other than KPP and GLM)
CLASS Receipt Nodes (at each facility)
L0, L1b, L2+ KPP & GLM products
GRB Info Pkts
Inst. Cal. Data
NOAA Satellite Operations Facility, Suitland MD
Remote Backup, Fairmont WV
Colors correspond to GOES-R instruments
Data is sent from GOES-R to CLASS Receipt Nodes at each GS facility