Rule-Based Approach for Earth Science Metadata Quality Assurance (QA) Tyler Stevens and Ellen Neff NASA’s Global Change Master Directory (GCMD) WYLE Information Systems ESIP Winter Meeting 2015 Session - Metadata Evaluation: Consistency, Compliance, and Improvement January 7, 2015

Outline 1.Importance of Metadata QA 2.Metadata QA Rules 3.Implementation Concept: QA Viewer 4.Conclusions 2

1. Importance of Metadata QA The Metadata QA Process enhances the efficiency and reliability in which users can discover, find, and make use of data that rely on accurate and complete metadata. Process: 3 Automated Validation Manual Review Make Changes to Metadata Notification of Metadata Changes Publish Metadata

1.2 Principles of High-Quality Metadata 4 AccuracyCompletenessConsistency Conciseness Readable/ Understandable

2. Metadata QA Rules A QA Rule is a check that is applied to content within a field of a metadata record to assess the quality. Checks include Controlled Vocabulary Use URL Validity Field Lengths Uniqueness Required Fields Populated 5

Rules are driven by: Metadata Formats (DIF, ECHO, ISO) o Example: ‘Controlled Vocabulary Check’ can check that valid GCMD keywords are being used in the DIF format. Metadata Models (UMM-C) o Example: ‘Required Field Check’ can check that there is content for a required field, i.e., required by the UMM-C model or by the metadata format. System Requirements (GCMD, CMR) o Example: ‘Max Field Length Check’ can check that a string is within a designated character length. Experience o Example: ‘Non-identical Field Check’ can encourage a metadata author to provide a descriptive title, rather than just repeating the EntryID Rule Development

2.2 Rule Categories Link Rules: Applied to fields that are or may contain links to an external source in the metadata. Character Rules: Applied to the number, type, or pattern of characters that are allowed within a field. Date Rules: Applied to date fields to check that dates are in the proper format. Numeric Rules: Applied to fields where the content should be or include a numeric value. Controlled Vocabulary Rules: Applied to content of the field to check that it matches a valid keyword, either by comparing to an author-provided list or to an external source (i.e. KMS). Miscellaneous Rules: Applies other various rules such as exists checks and suffix checks. 7

2.3 Rule Example 8 Field Name (Xpath)RuleNameConfigurationseverityifFailMessageiffail /DIF/DIF_Creation_DateDateFormat Check Date is yyyy- MM-dd ERROR${name} should be in ${format} date format only. Your ${name} is formatted ${actual} /DIF/Summary/AbstractAllURLsExist Check WARNPotential broken link found in ${xpath} Date Rule and Link Rule

3. Implementation Concept: QA Viewer A Web-Based Viewer/Tool That: Accesses metadata records via local files and GCMD and CMR APIs. Applies QA rules for DIF and ECHO metadata. Generates a table showing each field name and content along with the results per rule applied. 9

3.1 QA Viewer Display Panel View QA Results Displays Field Name, Rules Applied, Rule Results, and Message Sorts Results by Name and Type Filters Result Type 10

4. Conclusions Metadata QA rules can assist in assessing and improving metadata. Tools can support initial metadata screening as part of GCMD science metadata QA efforts. Engaging the community in rule-set authoring will improve the quality of the rules and the metadata. Tools can help automate some the metadata QA process to reduce the workload for manual review. 11

If you have questions or need additional information, please contact: Tyler Stevens Ellen Neff Or our user support office 12 Thank You!