1. U.S. Department of the Interior U.S. Geological Survey Management of Oceanographic time-series data at the Woods Hole Coastal and Marine Science Center Ellyn Montgomery USGS NERACOOS/NECOSP Data Management Workshop, Sept. 26, 2012

2. NECODP 2012  This presentation applies only to Oceanographic measurements acquired by Woods Hole researchers and colleagues to address scientific questions  The data is mostly used for validation of sediment transport and coastal process models  Management of the data is part of the larger Data Life-cycle that includes everything from planning to archival and publication USGS Coastal and Marine Geology Program

3. NECODP 2012  Our holdings describe conditions at specific locations for limited durations. Except for a 25+ year program in Massachusetts Bay, we don’t do long term monitoring programs.  Once the data files collected during an experiment are processed and QC’d that data is published (Federal requirement)  Updates are made only if a problem is found in the data to correct that problem About our data

4. NECODP 2012  The data are stored in netCDF format files using EPIC conventions  NcML is used to convert to CF conventions  Files are delivered on-line via  file download  OPeNDAP using THREDDS  FGDC metadata for each experiment is being created and harvested to ocean.data.gov  Each file has ISO 19115 metadata (ncISO) About our data II

5. NECODP 2012  Our raw data is included in the WHSC field activity tracking system  We store 3 levels of data on local disks that are backed up nightly, with periodic backups made and stored off-site. These levels are :  Raw data as off-loaded from the instruments  Processed data (and stages leading to it)  Published data  As EPIC  As CF How we manage our data - 1

6. NECODP 2012  Our data may be published directly to the website after approval or with an Open File Report that documents the context of the measurements.  We periodically check the website for broken links or non-responsiveness  We add new interfaces as available (for example making KML files to use with GoogleEarth) Publication and access

7. NECODP 2012 Web access

8. NECODP 2012 Web access

9. NECODP 2012 THREDDS

10. NECODP 2012  NetCDF allows metadata to be stored as attributes with the data in the file  We use standards and conventions to enable interoperability  We have GCMD DIF, FGDC and ISO19115 metadata in as many catalogs and registries as possible  We enhance the metadata as new standards emerge (added Unidata Dataset Discovery terms in 2011) Metadata

11. NECODP 2012  Community Standards development has evolved to enable LOT of great functionality  We are happy with how THREDDS allows remote clients to access data for analysis in real-time, without having to down-load files  Clients like Matlab, IDV, ncBrowse and ncView allow access and visualization  Registering our THREDDS catalogs in larger repositories of ocean measurements enhances discoverability Data Management Successes

12. NECODP 2012  Getting links to our holdings into the catalogs of the regional observatories  Since our data is historical, it hasn’t really fit the observatory mission, but including links to historical measurements in the area would be useful for enriching potential analyses  Identifying which of the emerging technologies will persist into the future  Finding ways to let potential users know the data is available Data Management Challenges

13. NECODP 2012  Use standards to enhance discoverability and enable use of community developed tools  Enable web-based access so the users can access what they need without assistance  Tailor the process to what your clients need to accomplish their missions  Adapt your data and services to take advantage of new techniques as they evolve Take home ideas

14. NECODP 2012  The human-friendly interface to our experimental data, (EPIC conventions) is at: http://stellwagen.er.usgs.gov http://stellwagen.er.usgs.gov  The computer-friendly (CF conventions) version of our data holdings, served by THREDDS at: http://geoport.whoi.edu/thredds/allts_reg_cat alog.html http://geoport.whoi.edu/thredds/allts_reg_cat alog.html  Contact me at: emontgomery@usgs.govemontgomery@usgs.gov  (508)457-2356 Web access