Are Standards Really Standards Any More? Mélanie F. Meaux NASA / GCMD In response to Wyn Cudlip with regards to an IDN profile of ISO …

Questions … What is a standard? What is a standard? Why should standards be used? Why should standards be used? What really makes a standard a standard? What really makes a standard a standard? What are the standards? What are the standards? What are the common metadata standards? What are the common metadata standards? Are profiles impacting interoperability? Are profiles impacting interoperability? How can communities achieve maximum interoperability? How can communities achieve maximum interoperability?

Just what is a standard? “A standard is simply a common set of terms and definitions that are presented in a structured format.” “A standard is simply a common set of terms and definitions that are presented in a structured format.” Federal Geographic Data Committee (FGDC)

Why should standards be used? Data Discovery Data Discovery Data Interoperability / Exchange Data Interoperability / Exchange Data Access / Re-Use Data Access / Re-Use Data Intercomparison Data Intercomparison …and saves time and money !

Why bother with a standard if, “Field choices are not always consistent and enforced (i.e date formats)” “Field choices are not always consistent and enforced (i.e date formats)” “[Standard is already complex enough, but] it does not appropriately handle information specific to my dataset” “[Standard is already complex enough, but] it does not appropriately handle information specific to my dataset” “The metadata structure does not easily handle non-static data sets” “The metadata structure does not easily handle non-static data sets” “The technical terms and document language are difficult to read and understand” “The technical terms and document language are difficult to read and understand” “[The standard is too complex:] you have to wade through too much to get to what actually applies to a particular dataset” “[The standard is too complex:] you have to wade through too much to get to what actually applies to a particular dataset” “[Because of standard complexity,] attention to format can overtake attention to actual content” “[Because of standard complexity,] attention to format can overtake attention to actual content” “The final product is not easily understandable to the general public” “The final product is not easily understandable to the general public” NOAA Coastal Services Center

What really makes a standard a standard? Endorsed by a recognized organization Endorsed by a recognized organization Easily Available Easily Available Well documented (to prevent inconsistent use of the standard) Well documented (to prevent inconsistent use of the standard) Widely used Widely used Flexible / Evolving Flexible / Evolving Easily adoptable (to allow maximum compatibility with existing standards) Easily adoptable (to allow maximum compatibility with existing standards) Guidelines for content change Guidelines for content change

What are the standards? Data (HDF, NetCDF, ASCII) Data (HDF, NetCDF, ASCII) Metadata Metadata Format (ISO 8601 Date/Time) Format (ISO 8601 Date/Time) Controlled Vocabularies / Ontology (RDF) Controlled Vocabularies / Ontology (RDF) W3C Mark-up Languages (XML, HTML, XSLT) W3C Mark-up Languages (XML, HTML, XSLT) Data Exchange Protocols (HTTP, FTP) Data Exchange Protocols (HTTP, FTP) Metadata Exchange Protocols (OAI) Metadata Exchange Protocols (OAI) Web Services Protocols (OGC, SOAP) Web Services Protocols (OGC, SOAP)

What are the standards? Data (HDF, NetCDF, ASCII) Data (HDF, NetCDF, ASCII) Metadata Metadata Format (ISO 8601 Date/Time) Format (ISO 8601 Date/Time) Controlled Vocabularies / Ontology (RDF) Controlled Vocabularies / Ontology (RDF) W3C Mark-up Languages (XML, HTML, XSLT) W3C Mark-up Languages (XML, HTML, XSLT) Data Exchange Protocols (HTTP, FTP) Data Exchange Protocols (HTTP, FTP) Metadata Exchange Protocols (OAI) Metadata Exchange Protocols (OAI) Web Services Protocols (OGC, SOAP) Web Services Protocols (OGC, SOAP)

1960: ROSCOP/CSR (Cruise Summary Report) 1970: MARC (Machine-Readable Cataloging) 1986: Australia New Zealand Land Information Council (ANZLIC) 1988: Directory Interchange Format (DIF) 1991: European Directory of Marine Environmental Datasets (EDMED) 1994: Dublin Core Metadata Initiative (DCMI) 1994: US Federal Geographic Data Committee (FGDC) Content Standard for Digital Geospatial Metadata (CSDGM) 1998: Common Data Index (CDI) 2001: NOAA/NODC Electronic Data Description Format (EDDF) 2004: ISO Geographic Information Metadata International Standard 2004: ISO Geographic Information Metadata International Standard What are some of the common metadata standards?

1960: ROSCOP/CSR (Cruise Summary Report) 1970: MARC (Machine-Readable Cataloging) 1986: Australia New Zealand Land Information Council (ANZLIC) 1988: Directory Interchange Format (DIF) 1991: European Directory of Marine Environmental Datasets (EDMED) 1994: Dublin Core Metadata Initiative (DCMI) 1994: US Federal Geographic Data Committee (FGDC) Content Standard for Digital Geospatial Metadata (CSDGM) 1998: Common Data Index (CDI) 2001: NOAA/NODC Electronic Data Description Format (EDDF) 2004: ISO Geographic Information Metadata International Standard formalized 2004: ISO Geographic Information Metadata International Standard formalized What are some of the common metadata standards?

Profiles of FGDC CSDGM 1999: Biological Data Profile of the CSDGM 2001: ESRI Data Profile of the CSDGM 2001: Shoreline Data Profile of the CSDGM 2002: Extensions for Remote Sensing of the CSDGCM

Draft Profiles of ISO Imagery and Gridded Data Extension (ISO ) WMO Core Metadata Standard Marine Community Profile (AODC) Australian Government’s Office of Spatial Data Management (OSDM) Profile North American Profile for Geospatial Metadata (NAP) –US National Profile –Canada Profile –ESRI Profile Biological Data Profile (NBII) Extension CEOS IDN Community Profile of ISO 19115

Are profiles impacting interoperability? “ “Profiles consist of a selected set of metadata elements. Additional metadata elements and conditionality changes can be established.” Source: ISO 19115:2003, Geographic Information - Metadata

Are profiles impacting interoperability? StandardProfile Broader/complex Narrower/simpler GenericSpecific Core ElementsCore Elements Optional FieldsMandatory Fields Undefined DomainsExplicit domains – extended codelists

Are profiles impacting interoperability? Many profiles – how many is too many? Many profiles – how many is too many? Draft ISO XML schema available at Draft ISO XML schema available at (no official schema at this time) Many interpretations of standard Many interpretations of standard By definition an extended document cannot conform to the ISO19139 schema! By definition an extended document cannot conform to the ISO19139 schema! Sharing outside your community becomes a challenge… Sharing outside your community becomes a challenge…

How can communities achieve maximum interoperability? Specific core elements and formats (i.e. ISO 8601: YYYY-MM-DD) need to be followed Specific core elements and formats (i.e. ISO 8601: YYYY-MM-DD) need to be followed Quality control of metadata is crucial - without validated metadata, meaningful comparisons cannot be made and may result in misrepresentation of the data Quality control of metadata is crucial - without validated metadata, meaningful comparisons cannot be made and may result in misrepresentation of the data Tools are needed to assist with the creation of metadata record – to assure compliance Tools are needed to assist with the creation of metadata record – to assure compliance Not just implementation, but conformance! Not just implementation, but conformance!

What tools are available? SMMS (Spatial Metadata Management System) SMMS (Spatial Metadata Management System) TKME / MP (Metadata Parser) TKME / MP (Metadata Parser) ESRI FGDC / ISO metadata editor ESRI FGDC / ISO metadata editor INTA ISO Metadata editor INTA ISO Metadata editor IODE/MEDI IODE/MEDI DocBUILDER (HTML / Standalone) DocBUILDER (HTML / Standalone) MATT (Metadata AuThoring Tool) MATT (Metadata AuThoring Tool)

What are some common controlled vocabularies for datasets? BODC Parameter Discovery & Usage Vocabulary BODC Parameter Discovery & Usage Vocabulary Global Change Master Directory Keywords Global Change Master Directory Keywords CF Standard Names CF Standard Names AGU Index Terms AGU Index Terms IOOS Core Variables IOOS Core Variables JGOFS Flux Study Parameters JGOFS Flux Study Parameters U.S. GLOBEC Thesaurus U.S. GLOBEC Thesaurus OBIS Taxonomic Categories OBIS Taxonomic Categories

Controlled vocabularies - Achieving maximum semantic interoperability Guidelines & Rules for Additions, Deletions, Guidelines & Rules for Additions, Deletions, and Modifications Notification services for updates Notification services for updates Clear definitions of terms Clear definitions of terms Tools and domain experts are needed to do the vocabulary mappings - work effort is huge to fully deliver interoperability! Tools and domain experts are needed to do the vocabulary mappings - work effort is huge to fully deliver interoperability! Long-term commitment to vocabularies and ontology maintenance Long-term commitment to vocabularies and ontology maintenance

Questions ? Mélanie F. Meaux NASA / GCMD