Satellite Data Provided by the NASA Ocean Biology Processing Group (OBPG)

Deep-Water Chlorophyll Images common-bin 12-day composite, Winter mg/m^3 SeaWiFS R5 MODIS/Aqua R1

OBPG Satellite Data - MODIS Aqua (2002-present) - MODIS Terra (2000-present) - SeaWiFS (1997-present) - OCTS ( ) - CZCS ( )

HDF File Format Hierarchical Data Format (HDF) is a data file format OBPG data files stored as HDF (L1A, L1B, L2, L3) HDF can be thought of as a “container” to store a variety of data types and meta data in a single file

Example HDF file

HDF-EOS OBPG files stored in HDF-EOS format: –specialized form of HDF –created by NASA in 1993 –standard format for all EOS data SeaDAS is compatible with OBPG HDF-EOS

MODIS Data Levels Level 0 Level 1A Level 1B Level 2 Level 3 binned Level 3 mapped

Data Levels & Flow Level 0 –raw digital counts –native binary format Level 1A –raw digital counts –HDF formatted Level 1B –calibrated reflectances –converted telemetry Level 2 –geolocated geophysical products for each pixel Ancillary data –wind speed –surface pressure –total ozone –Reynolds SST GEO –geolocation –radiant path geometry ATT & EPH –spacecraft attitude –spacecraft position

Data Levels & Flow Level 0 –raw digital counts –native binary format Level 1A –raw digital counts –HDF formatted Level 1B –calibrated reflectances –converted telemetry Level 2 –geolocated geophysical products for each pixel Ancillary data –wind speed –surface pressure –total ozone –Reynolds SST GEO –geolocation –radiant path geometry ATT & EPH –spacecraft attitude –spacecraft position Level 1A Subset –reduced to standard ocean bands only

Level 1A (HDF) –raw radiance counts –Onboard calibration data –navigation data –spacecraft telemetry –instrument telemetry A L1A_LAC

Level 1B (HDF) –calibrated, geolocated at-aperture radiances i.e. Top of Atmosphere (TOA) radiances –quality flags –error estimates –calibration data A L1B_LAC A L1B_HKM A L1B_QKM

Level 2 (HDF) –geophysical values for each pixel –products include chlorophyll-a, SST, nLw’s, K490, flags –geolocation data A L2_LAC A L2_LAC_SST A L2_LAC_SST4

Chlorophyll-a SST

Level 2 Flags

Level 2 Standard MODIS Ocean Products Ocean Color (day only) –Normalized water-leaving radiances - nLw( ) –Chlorophyll - C a –Diffuse attenuation - K d (490) –Aerosol type and concentration Optical thickness -  a Ångström exponent Atmospheric epsilon –Processing flags Cloud, land, glint, atmfail, atmwarn, chlfail, chlwarn, etc. Ocean Temperature –Long-wave SST (11-12  m) (day and night) –Short-wave SST (  m) (night only) –SST quality level (0-4)

Level 2 Non-standard Ocean Products Ocean Color –Alternate C a and K d algorithms –Chlorophyll fluorescence, FLH –Particulate inorganic carbon, Calcite –Inherent optical properties (various bio-optical models) absorption (total, phaeophytin, dissolved matter) backscatter (total, particulate) –Photosynthetically active radiation, iPAR, PAR (coming) –Euphotic depth (Z eu, Z sd ) –Subsurface PAR at 1st optical depth, K d (PAR) –Intermediate atmospheric correction products Ocean Temperature –Brightness temperatures

After Level-2… Level-2 L3 Binning

Level 3 Binned data products (HDF) –data composites –binned Level 2 data (spatially and temporally) –arithmetic means with statistics for each bin –daily, 8-day, monthly, yearly periods –0.5, 1, 2, 4, 9, 36 km 2 resolutions –stored as one file or a main file with subordinates –useful for scientific research A L3b_MO.main

Sinusoidal Equal-Area Projection

Level 3 SMI (mapped) (HDF) –Mapped L3 binned product –Data is scaled to 16-bit values –Equidistant cylindrical projection A L3m_MO_CHLO_4

Ocean coverage over time for binned files

Ancillary data products (HDF and other formats) –NRT data and refined ancillary data –used in various stages of processing: Geolocation (attitude and ephemeris) Calibration (coefficient look-up tables) atmospheric correction (MET/OZONE/SST)

Operational MODIS Ocean Band Suite VIS/NIR Ocean Color Thermal SST

Notes

Ingest queue table L0 Ingest process L1A-L1B ( MOD_PR02 ) L1B-L2 (MSl12) MODISA L2 table Archive - Distrib Server L3BIN (l2bin) MODISA L3-bin table Archive - Distrib Server MODISA L3-map table Archive - Distrib Server Operational MODIS-Aqua Data Flow Software process Hardware system Database table Ancillary input data Sensor CAL Sensor attribs Atm corr Browser CGI / httpd Ocean Color Web Server User Community MySQL DB Ozone L3MAP (smigen) MET MET, Ozone, and OISST data are dynamically selected for each L1A granule Product meta data are populated from production DB August 21, 2015 OISST Level-0 Ingest MODISA L0 table Archive - Distrib Server ATT/EPH Ingest process MODISA atteph table Archive - Distrib Server L0-L1A ( MOD_PR01 ) Geo-Location ( MOD_PR03 ) MODISA L1 table Archive - Distrib Server Full-resolution day- and nighttime via SEN (~60 GB per day) NOAA Realtime System NASA EDOS System

ODPS Data Processing System Current Components Processing Cluster 34 processing nodes 1.5 TB Ingest Servers 2 SeaSpace groundstations 5 storage nodes 6 TB Distribution Servers (ftp) 6 storage nodes 7.2 TB Distribution Servers (web) 1 large server – 3 TB 58 storage nodes 69.6 TB Testing Cluster 13 test nodes 2 TB Network Support Systems Database Server 1 large server 876 GB Backup Servers 1 large server 876 GB+2 TB tape 5 storage nodes 6 TB Extreme Networks Black Diamond 6816 Gigabit Ethernet switch Development Servers 1 processing node 2 storage nodes 2.4 TB User Desktops Cal/Val & QC Systems Mission Operations Systems

Data Processing and Storage database controlled, distributed processing and storage system many Linux-based, processing nodes (34 dual 3GHz Xeon) 200X processing for MODIS/Aqua ocean color 4000X processing through-put for SeaWiFS ocean color 9-month OCTS mission can be reprocessed over lunch all data online for direct access (100TB RAID-5 array) capacity for global mission testing prior to reprocessing, allowing opportunity for community participation in reprocessing decisions direct and near real-time user access to data archives processing node storage node database server gigabit ethernet

Supermicro 6024 rack mount server 2Intel Xeon CPUs 3.06 Ghz 81 GB DDR2 RAM modules 2Intel Gigabit Ethernet network interfaces 1Seagate 80 GB EIDE disk (system) 5Maxtor 73.4 GB Atlas 10K IV Ultra320 SCSI disks (processing) 1CD-ROM (low profile) 1Supermicro X5DP8-G2 motherboard with built-in Ultra320 SCSI controller 1Supermicro 6024H 2U rack mount server case with dual power supplies Operating system: Linux Ocean Biology Data Processing System - Processing Node

NAS 1.2 TB rack mount server 1Intel Pentium-4 CPU 3.06 Ghz 2512 MB DDR2 RAM modules 1Gigabit Ethernet network interface 1Seagate 80 GB EIDE disk (system) 8Maxtor 250 GB MaxLine Plus II EIDE disks (RAID 5, 6 data + 2 hot spares) 1CD-ROM (low profile) 1Asus P4C800-E Motherboard 13ware Escalade RAID controller for 8 drives 1Comix Computer RM-3U9SCA (IDE) ATX rackmount chassis 3 U, 9 hot swappable drive bays Operating system: Linux Ocean Biology Data Processing System - Storage Node

SunFire V880 rack mounted 4-8UltraSPARC-III+ processors 8-16GB RAM GB 10K RPM FibreChannel disks 2Gigabit Ethernet nework interfaces 1-5Fast Ethernet network interfaces 1External FibreChannel adapter Operating system:Solaris Ocean Biology Data Processing System - Large Server

Extreme Networks Black Diamond Gbps total switching capacity Route/filter/forward 192 million pps 160 GigE ports in current configuration All modules are hot swappable 4MSM management modules 2G8Ti 8 port Gigabit Ethernet modules 6G24T3 24 port Gigabit Ethernet modules 4Hot-swappable power supplies Ocean Biology Data Processing System - Network Switch