NOAA Metadata Update Ted Habermann

NOAA EDMC Documentation Directive This Procedural Directive establishes 1) a metadata content standard (International Standards Organization [ISO] Parts 1 and 2) and 2) a recommended representation standard (ISO 19139) for documenting NOAA’s environmental data and information. Roles and Responsibilities Background Information

Building the Foundation Develop and implement common metadata management tools Support training specifically targeted at improving NOAA’s data documentation Initiate teams to work on “special documentation problems” that cross Line and Staff Offices Promote and highlight good examples Encourage and support participation in the ISO and Open Geospatial Consortium (OGC) Use rubrics to establish a baseline and monitor progress NOAA Line/Staff Offices and Programs must build on a strong foundation of support across all of NOAA

Line / Staff Office & Program Responsibilities Data Collectors/Providers Data Users Data Stewards Standards Experts 1. Identify Documentation Expertise 3. Create and Improve Metadata 2. Assess current state of documentation 4. Publish/Preserve Metadata Metadata for Discovery Metadata for Use Metadata and Documentation for Understanding Documentation of Collections Documentation of Datasets Documentation of Services NOAA Data Centers / Archives

Building the Foundation The GOES-R Program recently expanded the scope of their metadata model from just ISO to ISO plus netCDF. This creates an integrated metadata framework that includes metadata in granules (files) as well as the ISO standards. This framework is important across NOAA and for many of our partners (NASA). Initiate teams to work on “special documentation problems” that cross Line and Staff Offices

Building the Foundation The ISO metadata standard adopted by NOAA is currently undergoing a significant revision that addresses many requirements from NOAA and the netCDF-CF community that includes many NOAA data providers and partners. These changes will be adopted as part of the Draft International Standard which is about to be approved by ISO (Nov. 2011) Encourage and support participation in the ISO and Open Geospatial Consortium (OGC)

Building the Foundation ISO metadata standard trainings were held during June in Silver Spring as part of the annual EDMC Data Management Workshop and during October as part of the FGDC ISO Metadata Summit The GEO-IDE Wiki includes over 200 pages of information about the ISO Standards. Support training specifically targeted at improving NOAA’s data documentation

Building the Foundation A set of tools for translating metadata collections from FGDC to ISO have been developed and tested by NODC and NGDC. Tools for evaluating the completeness of ISO metadata that include links to the GEO-IDE wiki have been developed and tested at NGDC. Develop and implement common metadata management tools Use rubrics to establish a baseline and monitor progress

What Goes Where?

Spiral 1: Initial Content Spiral 1: Initial Content Metadata Content Independent of standard Metadata Content Independent of standard Standard Guidance / Implementation Standard Guidance / Implementation Check Back With Users Check Back With Users Spiral 2-N: Scientific Questions New Requirements New Use Cases Check Back With Data Collectors/ Providers Check Back With Data Collectors/ Providers Spiral Development / Training

Identification Id Title Abstract Resource Date Topic Category Theme Keyword Metadata Contact Science Contact Identification Id Title Abstract Resource Date Topic Category Theme Keyword Metadata Contact Science Contact Extent Geospatial Bounding Box Temporal Start/End Vertical Min/Max Place Keywords Extent Geospatial Bounding Box Temporal Start/End Vertical Min/Max Place Keywords Distribution Distributor Contact Online Resource Distribution Format Data Center Keywords Browse Graphic Distribution Distributor Contact Online Resource Distribution Format Data Center Keywords Browse Graphic Text Searches Purpose Extent Description Lineage Statement Project Keywords Text Searches Purpose Extent Description Lineage Statement Project Keywords Quality/Lineage Sources Process Steps Quality Reports / Coverages Quality/Lineage Sources Process Steps Quality Reports / Coverages Acquisition Information Instrument Platform Instrument Keywords Platform Keywords Acquisition Information Instrument Platform Instrument Keywords Platform Keywords Potential Spirals Content Information Attribute Type Attribute Names Attribute Definitions Attribute Units Content Information Attribute Type Attribute Names Attribute Definitions Attribute Units Discovery Understanding Connection OnlineResource: Name Description Function Application Schema Connection OnlineResource: Name Description Function Application Schema

SpiralNone0-33%34-66%67-99%All Identification Connection Extent Distribution Description Content Lineage Acquisition Information Metadata Evaluation for Dataset X Date: 2010/10/24 Spirals and Rubrics Progress GOALSGOALS Each record has a score that can be calculated using a consistent, standard tool

Spiral Development / Training: Rubrics

Questions?

Identify documentation expertise 1.Establish Data Stewardship Teams to facilitate documentation creation and improvement for appropriate organizational units or around programmatic needs. 2.Data Stewardship Teams should include the following expertise/skills: Data Collectors/Providers Data Users Data Stewards Standards Experts Line/Staff Office Responsibilities

Assess current state of documentation 1.Identify existing sources of documentation (Data Collectors/Providers) 2.Classify existing documentation into following categories (Data Stewards and Standards Experts):  Metadata for Discovery  Metadata for Use  Metadata and Documentation for Understanding  Documentation of Collections  Documentation of Datasets  Documentation of Services 3.Identify high-priority targets for improvement (All members of Data Stewardship Team) 4.Highlight best practices, successful teams and individuals (Line and Staff Office Management) Line/Staff Office Responsibilities

Create and Improve Metadata 1.Translate/transform existing metadata into the recommended representation (ISO 19139) 2.Create metadata for undocumented data and information 3.Use spiral approach for improving metadata Line/Staff Office Responsibilities

Publish/Preserve Metadata (Data Stewards) 1.Publish metadata record in new or existing Web-Accessible Folders or using a standard catalog service. This will make it possible to connect metadata records to various discovery portals using standard services 2.Work with NOAA Archives to ensure that documentation and metadata will be preserved for the long-term Line/Staff Office Responsibilities

Cross-NOAA Responsibilities 1.Encourage and support participation in the ISO and Open Geospatial Consortium (OGC) standards development and evolution processes 2.Develop and implement common metadata management tools including mechanisms for evaluating the completeness and quality of data documentation Utilize rubrics to establish the baseline and monitor progress. Engage users in providing feedback on data documentation efforts and opportunities. 3.Promote and highlight good examples of documentation and the individuals involved in their creation. 4.Support training specifically targeted at improving NOAA’s data documentation. 5.Initiate teams to work on “special documentation problems” that cross Line and Staff Offices

EDMC Responsibilities 1.Review this Procedural Directive twice a year to evaluate effectiveness and monitor progress. 2.Work with the CIO Council and the NOAA Observing Systems Council to implement and monitor progress on the Cross-NOAA responsibilities listed above. 3.Encourage and support partnerships with external organizations in the process of migration of metadata from FGDC to ISO Standards.

Measure Progress 1.An increase in the amount of NOAA environmental data and information that is well documented and discoverable via national and international discovery portals. 2.Improvement in the quality of Line and Staff Office and Program data documentation processes.