Documenting Data to Support Discovery and Use Dave Neufeld Ted Habermann John Kozimor.

Slides:



Advertisements
Similar presentations
Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09.
Advertisements

Evolution of Metadata Standards: New Features in ISO Ted Habermann NOAA National Data Centers September, 2008
1 NASA CEOP Status & Demo CEOS WGISS-25 Sanya, China February 27, 2008 Yonsook Enloe.
Applications and Tools Jaci Mize NOAA/NESDIS/NODC/NCDDC Rob Dollison USGS.
DMAC ST and the Activities of the IOOS PO Derrick Snowden DMAC Steering Team
NOAA Documentation Improvement Ted Habermann How do we measure and visualize improvements in NOAA Documentation? Record Count Completeness (Rubric Scores)
Briefing for the Upper Colorado River Basin Pilot Update Meeting, February 24, 2011.
1 ISO – Metadata Next Generation International consensus being built on structured metadata within a broader Geomatics Standard under ISO Technical.
Data Discovery 1.Data discovery systems 2.TDS and metadata 3.NcISO services.
NOAA Metadata Update Ted Habermann. NOAA EDMC Documentation Directive This Procedural Directive establishes 1) a metadata content standard (International.
Introduction to Geospatial Metadata – ISO 191** Metadata National Coastal Data Development Center A division of the National Oceanographic Data Center.
ISO/TC211 Geographic Information/Geomatics Implementing ISO Metadata David Danko Work Item 15—Project Leader
Introduction to Geospatial Metadata – FGDC CSDGM National Coastal Data Development Center A division of the National Oceanographic Data Center Please .
Steve Rutz NOAA/NESDIS National Oceanographic Data Center NODC Observing Systems Team Leader June 21, 2011.
Documentation Developments Ted Habermann, NOAA/NESDIS/NGDC Adoption of ISO Metadata Standards - Crossing the Chasm Facilitating the Transition Documentation.
Overview of the ODP Data Provider Sergey Sukhonosov National Oceanographic Data Centre, Russia Expert training on the Ocean Data Portal technology, Buenos.
Metadata (for the data users downstream) RFC GIS Workshop July 2007 NOAA/NESDIS/NGDC Documentation.
ISO-2 and more NGDC Geoportal January 4 th, 2012.
DM_PPT_NP_v01 SESIP_0715_AJ HDF Product Designer Aleksandar Jelenak, H. Joe Lee, Ted Habermann Gerd Heber, John Readey, Joel Plutchak The HDF Group HDF.
Interoperable Documentation Ted Habermann, NOAA/NESDIS/NGDC NCAR Earth Observing Laboratory, June 2010 Links: GEO-IDE Wiki:
IOOS Data Management Integration Standards Plans in the Northeast Eric Bridger (GMRI) Sep
Sept 19,  Provides a common set of terminology and definitions  A framework for describing resources and processes  Enables computer based interoperability.
ATMOSPHERIC SCIENCE DATA CENTER ‘Best’ Practices for Aggregating Subset Results from Archived Datasets Walter E. Baskin 1, Jennifer Perez 2 (1) Science.
U.S. Department of the Interior U.S. Geological Survey Management of Oceanographic time-series data at the Woods Hole Coastal and Marine Science Center.
Glider data (*.*bd files) via Iridium or rsync from WHOI, Umaine or UMaryland Disk NetCDF/Acquire WMO ID IOOS Glider DAC ERDDAP OPeNDAP sequence or download.geoJson,.json.nc,.ncCF,.ncCFMA.csv,.tsv,
Page 1 LAITS Laboratory for Advanced Information Technology and Standards ISO & Status Liping Di Laboratory for Advanced Information Technology.
Accomplishments and Remaining Challenges: THREDDS Data Server and Common Data Model Ethan Davis Unidata Policy Committee Meeting May 2011.
® GRDC Hydrologic Metadata - core concepts - 5 th, WMO/OGC Hydrology DWG New York, CCNY, August 11 – 15, 2014 Irina Dornblut, GRDC of WMO at BfG Copyright.
Metadata Implementation Ted Habermann NOAA National Geophysical Data Center Documentation: It’s not just discovery... 50% change in global average Why?
M u l t I b e a m III W o r k s h o p M u l t I b e a m III W o r k s h o p National Geophysical Data Center / World Data Centers NOAA Slide 1 End-to-End.
Documentation from NcML to ISO Ted Habermann, NOAA NESDIS NGDC.
VO Sandpit, November 2009 CEDA Metadata Steve Donegan/Sam Pepler.
Workshop Resources.
CSDGM Overview CSDGM Tools and Resources. Resources Series Materials: ftp://ftp.ncddc.noaa.gov/pub/Metadata/Online_ISO_Tr aining/Intro_to_Geospatial_Metadata/
Metadata for Data Understandability Ted Habermann NOAA National Geophysical Data Center AGU, Spring 2008.
NODC Metadata Management for Geoportal Server and Beyond John Relph NOAA National Oceanographic Data Center.
University of San Diego Scripps Institution of Oceanography UDDC Adoption and Examples OceanSITES DMT Meeting November 2011.
Transitioning from FGDC CSDGM Metadata to ISO 191** Metadata
Metadata – use data discovery e.g. a library catalog data assessment determine the fitness-for-purpose of a data set data retrieval e.g., format.
Unidata’s TDS Workshop TDS Overview – Part I July 2011.
Kevin O’Brien University of Washington/JISAO NOAA/PMEL The Observing System Monitoring Center Steve Hankin, PMEL Ted Habermann, NGDC David Neufeld, NGDC.
UAF/OSMC Presenters: Kevin O’Brien and Eugene Burger Abstract: Kevin O’Brien and Eugene Burger are from NOAA’s Pacific Marine Environmental Laboratory.
Vers national spatial data infrastructure training program What is Metadata? Introduction to Metadata An overview of geospatial metadata, presentation.
Meteorological Assimilation Data Ingest System (MADIS) and ISO Data Quality Ted Habermann NOAA National Data Centers MADIS observations on April 29, 2004.
Documenting UAF Data Ted Habermann NOAA/NESDIS/National Geophysical Data Center.
1-2-3 February 2006 –Page 1 Mersea Integrated System How to improve Access/Downloading services ? How far do we go in terms of standardization ?
Creating Good Documentation NOAA National Geophysical Data Center
ARMAP, AOV, ISO Metadata, and RESTful Architectures for Data Sharing and Interoperability IASC/SAON Arctic Data Committee meeting 26 October, 2015.
Metadata Training for SEFSC Science Staff Part Two.
ISO 191** Overview A “Family” of Standards. Resources ISO Standards Web Page – Technical.
A look to the past for the future- The North American Profile Sharon Shin Metadata Coordinator Federal Geographic Data Committee.
ILYA ZASLAVSKY RAQUEL CALDERON CHRIS CONDIT JEFFREY GRETHE AMARNATH GUPTA BURAK OZYURT THOMAS WHITENACK DAVID VALENTINE ALICE GILIARINI AARON GONG University.
Next Generation Data Access at NCDC Achieving world class data access.
End-to-End Data Services A Few Personal Thoughts Unidata Staff Meeting 2 September 2009.
New Dog You Met Conventions Ted Habermann NOAA EDMC May 2012 Data needs to be in tools Document.
CLASS Metadata and Remote Sensing Extensions CLASS Data Provider’s Conference September 2005 Anna Milan, Ted.Habermann,
NOAA EDMC Ocean Observatories Initiative Cyberinfrastructure Karen Stocks OOI CI Data Curator University of California, San Diego Ocean Observatories.
ODP V2 Data Provider overview. 22 Scope Data Provider provides access to data and metadata of the local data systems. Data Provider is a wrapper, installed.
10 th GHRSST Science Team Meeting Santa Rosa, CA 3 June 2009.
Global Earth Observation System of Systems (GEOSS) Architecture Implementation Pilot (AIP) Air Quality and Health Working Group Stefan Falke, Rudy Husar,
Making FAAM Flights Discoverable
Big Earth Data Initiative (BEDI) Metadata Improvement: Case Studies
Integrating Data and Information Across Observing System
Presented by Sharon Shin, FGDC Developed by Lynda Wayne, GeoMaxim-FGDC
Identifiers Answer Questions
NORTH CAROLINA state and local government METADATA PROFILE
Incorporating ISO Metadata Using HDF Product Designer
Prepared by: Jennifer Saleem Arrigo, Program Manager
Proposal of a Geographic Metadata Profile for WISE
Chapter 5.5 Metadata John Cima.
Presentation transcript:

Documenting Data to Support Discovery and Use Dave Neufeld Ted Habermann John Kozimor

Where does it go? Station Histories Quality Measurements Data Processing Sensors Extents Parameters People

Levels of Documentation for OceanSITES

Content Distribution Usage Discovery Program (OceanSITES) Network (ESTOC, LINE, MBARI, TAU) Platform (M0, M1, M2,) Deployment (MO-04 M-05, MO-06, MO-07) Sensor (Current Profiler, CTD, CO2 Analyzer, Flurometer, Spectroradiometer )

ISO Content

ncISO: Generating ISO metadata from NetCDF and THREDDS

Discovery: ncISO and the UDDC Service Progress

ISO Integration with Geoportal

Improving Discovery Documentation Work with Data Managers Improve Content Expand Content Review guidance on the wiki and the role of Unidatas Data Discovery Conventionwiki Harmonize UDDC and OceanSITES User manual

Improving Content Depth of Content: Is information missing from existing metadata? –Are all PIs included in the program level record? –Are all network contacts included in network level record? Breadth of Content: Is the standard utilized to the fullest? –Is the data processing workflow described? –Are quality reports included? –Is the instrumentation described? –Is platform/instrument maintenance documented? Organization of Content: Is information documented at the correct level?

Remapping OceanSITES to UDDC Examples amplesCatalog.htmlhttp:// amplesCatalog.html

Expanding Content NetCDF Conventions UDDC CF Importance of Titles, Keywords, Abstract Lineage Quality Instrumentation WHOTS as a starting point for discussion on evolving conventions to include sensor level info: WHOTS Example linkWHOTS Example link

Next Steps: Supporting Use Documentation Extending Existing Conventions –Deployment / Sensor / Lineage Incorporate into ISO Incorporate OSMC ingest

Documentation Hierarchies Enable objects to be grouped by scope (level) Enable levels to be related or connected by hierarchy order Produce a comprehensive documentation structure Produce a complete archive information object

ISO Examples Complete Hierarchy Program Level Network Level Platform Level Deployment Level Sensor Level

Questions?