Documentation Developments Ted Habermann, NOAA/NESDIS/NGDC Adoption of ISO Metadata Standards - Crossing the Chasm Facilitating the Transition Documentation Metric THREDDS/netCDF Documentation Consortium

Technology Adoption Moore described the “chasm” in the adoption life cycle. He proposes that many new technologies do not make it across the chasm between visionaries and early adapters. They fall into the chasm. The technology S-curve with the chasm might look like: TIME

Technological Disruption Era of Ferment Selection Tipping Point Technology Cycle

Who Are Our Partners? GOES-R: NOAA and NASA AMS Metadata Policy Working Group Recommendations: The AMS should adopt ISO and SensorML as the metadata standards for the Nationwide Network of Networks. In coordination with other organizations within the broader atmospheric and oceanic sciences community, the AMS should provide opportunities for its members to become better educated on the nomenclature, structure, and implementation of metadata and the standards recommended below. The AMS should establish a permanent committee to serve as a body of experts … to provide advice to organizations working to implement a Nationwide Network of Networks and to coordinate with the Federal Committee for Integrated Observing Systems (CIOS). AMS Metadata Policy Working Group Recommendations: The AMS should adopt ISO and SensorML as the metadata standards for the Nationwide Network of Networks. In coordination with other organizations within the broader atmospheric and oceanic sciences community, the AMS should provide opportunities for its members to become better educated on the nomenclature, structure, and implementation of metadata and the standards recommended below. The AMS should establish a permanent committee to serve as a body of experts … to provide advice to organizations working to implement a Nationwide Network of Networks and to coordinate with the Federal Committee for Integrated Observing Systems (CIOS). Federal Coordinator for Meteorology Metadata Joint Action Committee The Joint Action Group recommends that the Federal departments and agencies adopt ISO and SensorML as weather observation metadata standards and participate in the process of implementing those standards across the national weather enterprise. Federal Coordinator for Meteorology Metadata Joint Action Committee The Joint Action Group recommends that the Federal departments and agencies adopt ISO and SensorML as weather observation metadata standards and participate in the process of implementing those standards across the national weather enterprise. Chapter VI - Other Post Award Requirements and Considerations 4. Dissemination and Sharing of Research Results b. Investigators are expected to share with other researchers, at no more than incremental cost and within a reasonable time, the primary data, samples, physical collections and other supporting materials created or gathered in the course of work under NSF grants. Grantees are expected to encourage and facilitate such sharing.

The ISO Standards are broad What Goes Where?

Spiral 1: Initial Content Spiral 1: Initial Content Metadata Content Independent of standard Metadata Content Independent of standard Standard Guidance / Implementation Standard Guidance / Implementation Check Back With Users Check Back With Users Spiral 2-N: Scientific Questions New Requirements New Use Cases Check Back With Data Collectors/ Providers Check Back With Data Collectors/ Providers Spiral Development / Training

Identification Id Title Abstract Resource Date Topic Category Theme Keyword Metadata Contact Science Contact Identification Id Title Abstract Resource Date Topic Category Theme Keyword Metadata Contact Science Contact Extent Geospatial Bounding Box Temporal Start/End Vertical Min/Max Place Keywords Extent Geospatial Bounding Box Temporal Start/End Vertical Min/Max Place Keywords Distribution Distributor Contact Online Resource Distribution Format Data Center Keywords Browse Graphic Distribution Distributor Contact Online Resource Distribution Format Data Center Keywords Browse Graphic Text Searches Purpose Extent Description Lineage Statement Project Keywords Text Searches Purpose Extent Description Lineage Statement Project Keywords Quality/Lineage Sources Process Steps Quality Reports / Coverages Quality/Lineage Sources Process Steps Quality Reports / Coverages Acquisition Information Instrument Platform Instrument Keywords Platform Keywords Acquisition Information Instrument Platform Instrument Keywords Platform Keywords Content Information Attribute Type Attribute Names Attribute Definitions Attribute Units Content Information Attribute Type Attribute Names Attribute Definitions Attribute Units Discovery Understanding Connection OnlineResource: Linkage (URL) Name Description Function Connection OnlineResource: Linkage (URL) Name Description Function Potential Spirals

SpiralNone0-33%34-66%67-99%All Identification Connection Extent Distribution Description Content Lineage Acquisition Information Metadata Evaluation for Dataset X Date: 2010/10/24 Spirals and Rubrics

Initial Rubric for FGDC > ISO Translation

Identifying Bounding Extents

Web Accessible Folders - Record Populations Web Accessible Folder Web Accessible Folder Web Accessible Folder

ISO Rubric Score Record Population Scores FGDC - CoRISFGDC - NOSUAF

Documentation Repository ISO 19115, , and extensions Documentation Repository ISO 19115, , and extensions THREDDS netCDF DIF, FGDC, Data.Gov DIF, FGDC, Data.Gov SensorML WCS, WMS, WFS, SOS ISO KML Multiple Dialects

Metadata Data Multiple Dialects: THREDDS Data Server THREDDS Catalog THREDDS Catalog file1.nc file2.nc file3.nc file4.nc file1.nc file2.nc THREDDS Catalog THREDDS Catalog OPeNDAP WMS WCS ISO NcML Rubric We have developed an extension to THREDDS that supports extraction of metadata as NcML, evaluation of that metadata, and translation to ISO. This will be integrated into the next version of THREDDS

Improvement Plan 1.Improve standard (add new classes and keyword types) 2.Create and encourage adoption of shared vocabularies for discovery 3.Improve translation from NcML to ISO 4.Integrate ncISO into TDS 5.Create good xsl stylesheets for documentation display 6.Encourage existing and new data providers to support data discovery conventions 7.Find partners for further improvement in NcML and ISO

TIME Technological Disruption Era of Ferment Selection Tipping Point Technology Cycle

Shared Needs NWS FAA NESDIS NASA USGS NSF WMO The groups that created these documents are leading the transition to the ISO Standards and recognizing the need for guidance and tools. Do they each head off in their own direction… Or come together to communicate and collaborate? Do they each head off in their own direction… Or come together to communicate and collaborate?

The Documentation Consortium NWS NESDIS NASA USGS NSF WMO Consortium: association of organizations with the objective of pooling resources for achieving a common goal: To foster stewardship & meaningful publication of Earth & Space data for world-wide cross- disciplinary & citizen use, today and tomorrow. Initially to coordinate and facilitate the transition to effectively use of ISO Documentation Standards. Standards Training FAA Tool Adoption & Development

Tools that facilitate use of models/protocols /standards Data-stewarding exemplars (tool usage, conventions…) Tools that facilitate use of models/protocols /standards Data-stewarding exemplars (tool usage, conventions…) Models/protocols /standards that support meaningful exchange, preservation & use of Earth & Space data Models/protocols /standards that support meaningful exchange, preservation & use of Earth & Space data Expertise on data models, documentation, protocols, semantics, curation, (on- line) teaching & learning Expertise on data models, documentation, protocols, semantics, curation, (on- line) teaching & learning A “community of practice” that publishes, cites & stewards interoperable Earth & Space data A “community of practice” that publishes, cites & stewards interoperable Earth & Space data Influential advocates for better stewarding/ citing/publishing of Earth & Space data Influential advocates for better stewarding/ citing/publishing of Earth & Space data Standards Training Tool Adoption & Devel Outcomes: 1.Transparent and trustworthy scientific results (on climate change, e.g.) 2.Better reproducibility of results from data-intensive, scientific studies of Earth & Space 3.Accelerated progress in sciences where data-intensity is high and interoperability is needed Mission: To foster stewardship & meaningful publication of Earth & Space data for world-wide cross-disciplinary & citizen use, today and tomorrow. Logic Model

Positive deviance says that if you want to create change, you must scale it down to the lowest level of granularity and look for people within the social system who are already manifesting the desired future state. Take only the arrows that are already pointing toward the way you want to go, and ignore the others. Identify and differentiate those people who are headed in the right direction. Give them visibility and resources. Bring them together. Aggregate them. Barbara Waugh Leadership Model: Positive Deviance