The NODC Glider Technical Specification Tom Ryan, Dan Seidov, John Relph (NODC) and James Bennett (University of Washington) U.S. IOOS National Glider Strategy Development Workshop August 1, 2012

NODC Helping Data Providers to Comply with Requirements to Archive Data NOAA Data Management Procedural Directive National Science Foundation (NSF) Data Management Plan Requirements Grant Requirements 2

Open Archival Information System (OAIS) Reference Model Submission Information Package (SIP) –SIP is what a data collector/producer submits to NODC Archival Information Package (AIP) –AIP is what a collector/producer submits to NODC along with NODC generated files (browse graphic, transforms, metadata, etc.) Dissemination Information Package (DIP) –DIP is what NODC provides to users through discovery and access URLs 3

Glider Technical Specification Goals Simplify the Procedure for Archiving Glider Data at NODC Provide Framework For Glider Data Management Plans Support Interoperability of Glider Data 4

NetCDF File Format Data in netCDF is mandatory for NODC automation –Data are self documenting (metadata and data are maintained in same file) –Improved data access and discovery (OPeDAP, THREDDS, WMS, WCS) Climate and Forecast (CF) Metadata Conventions –Use CF Standard Name Attribute Convention for Dataset Discovery (ACDD) NODC netCDF Templates – Templates do not represent an attempt to create a new standard and are not required for archiving data at NODC. 5

File Naming Convention PPPPP_GGG_DDDD_YYYYMMDD.nc PPPPP – Prefix, Five Submitter defined alpha characters GGGGG – Glider Serial Number, Five alpha numeric characters DDDD – Deployment Dive Number, Four numeric characters YYYYMMDD – Start date of deployment.nc –netCDF file extension 6

Metadata Submitter generated collection level and/or deployment level FGDC/ISO metadata are encouraged but not required NODC will create FGDC/ISO metadata record for SIP with discovery and access URLs to data –Ocean Archive System –Geoportal Server –HTTP and FTP –OPeNDAP’s Data Access Protocol via Hyrax –OGC’s WMS and WCS via THREDDS Data Server 7

Submission Manifest Contact information of submitter List of all SIP files with cryptographic file integrity validation (e.g., MD5 algorithm) -Ensures NODC received what was submitted without any errors -Supports revisions of AIP for updated/corrected data.

Submission sent to contains the following –Data Provider –Contact Person –Collecting Institution –Submitting Institution –Transfer Protocol –Glider Type –Glider name –Manifest of files is part of the SIP and provides provenance to the data. 9

File Transfer Methods Pull – Data is mirrored on NODC servers (NODC preferred protocols) –RSYNC Service Protocol –FTP protocol Push – Data are pushed to NODC Servers –Data must be uploaded to NODC FTP servers in compressed form to allow for re-uploading if transmission errors occurred. 10

Platform Codes Gliders are platforms and NODC will request platform codes from the International Council for Exploration of the Sea (ICES). To facilitate this request, the data submission must include: –Platform Name: –Country: –Platform Class : Autonomous underwater vehicle –Current length (meters): –Year Built: –Notes: 11

Cross-references NODC encourages data submitters to provide all relevant documentation which will be published in Documentation-only Accessions. These accessions will provide persistent URLs to specific documents to be included in netCDF files. Documents to include: –Quality Control Manual –Non-copyrighted journal articles –Technical Reports –Other documentation 12

Conclusion The NODC guidelines for submitting glider data for automated accessioning are straightforward and easy to follow. Following these guidelines by a collector/submitter will reduce the work needed to set up the automated process of archiving their data at NODC by up to 90% and will guarantee ease and smoothness of the archiving process. 13

Acknowledgement The Group for High Resolution Sea Surface Temperature (GHRSST) Data Specification Version 2.0 (GHRSST Science Team, 2010) served as a model for this Technical Specification. 14

Resources NODC Contacts (add as a CC to all NODC netCDF templates – CF Standard Names – ACDD – java/formats/DataDiscoveryAttConvention.htmlhttp:// java/formats/DataDiscoveryAttConvention.html NetCDF Compliance Checker – 15