Data Management System to Collect, Quality Control, Distribute, and Archive Near Real-time Marine Data Jeremy J. Rolph, Jacob T. Rettig, Mark A. Bourassa, Ph.D., Jiangyi Hu, Ph.D., E. Michael McDonald, and Shawn R. Smith Center for Ocean-Atmospheric Prediction Studies Florida State University, Tallahassee, FL , USA Introduction A Data Management System (DMS) for marine meteorology is applied to platforms in the Gulf of Mexico A Data Management System (DMS) for marine meteorology is applied to platforms in the Gulf of Mexico Shipboard Automated Meteorological and Oceanographic System (SAMOS) initiative Shipboard Automated Meteorological and Oceanographic System (SAMOS) initiative –DMS developed in 2004 to support underway data collection of participating research vessels –Manages navigational, surface oceanographic, and meteorological observations –19 participating oceanographic research vessels (several in Gulf of Mexico) –Data support: –Validation studies for satellite sensors and numerical model products –Air-sea interaction research –Developing satellite data retrieval algorithms Northern Gulf of Mexico Institute (NGI) Northern Gulf of Mexico Institute (NGI) –DMS updated in 2008 to provide data stewardship for instrumented Air Force towers in the northern Gulf of Mexico –Observations are managed for one Air Force Tower currently (real-time meteorology, delayed oceanographic) –Data support: – Physical oceanographic, air-sea interaction, and ecosystem research –Real-time validation for offshore forecast from the National Weather Service The Data Management System Combines automated processing programs, a database, and web tools Combines automated processing programs, a database, and web tools Prior to processing, the database must be populated with platform- and Prior to processing, the database must be populated with platform- and instrument-specific metadata The Database: File Tracking, Version Control, & QC Data Flow SAMOS initiative and NGI data flows SAMOS initiative and NGI data flows –differ in data acquisition and final product –identical in automated Data QC processing Preliminary Data Processing Check & Verify - Incoming ASCII files are verified for proper format and if they originated from a recruited platform. Check & Verify - Incoming ASCII files are verified for proper format and if they originated from a recruited platform. Quality Control Quality Control –Verify that observations are over water and within realistic physical limits. –Poor quality data values are not removed from data set, instead they are flagged. Data Merge and secondary QC Raw data at 10 minute intervals via web Raw data at 10 minute intervals via web NetCDF downloads in development NetCDF downloads in development NGI Distribution NGI merges all 10 minute files for a day at 00UTC SASSI - Statistical Spike and Stair-Step Indicator Tower blockage/flow distortion from known angles Redundant sensor comparison? COAPS – COAPS – SAMOS – SAMOS – NGI – NGI – Plans Provide tower data to NDBC in real-time Auto-qc NetCDF files (SASSI) Potential archival to NODC Contacts Acknowledgements The NGI is supported by NOAA Office of Oceanic and Atmospheric Research. SAMOS is supported by NOAA Office of Climate Observation and NSF. Preliminary and merged file/process tracking and DB QC stats (graphical/tabular) available for latest version Spatial track maps available with selectable values vs. time SAMOS vessel tracks in Gulf of Mexico region from 1 January April 2009.