1. Big Earth Data Initiative (BEDI) Metadata Improvement: Case Studies
John Kozimor, Ted Habermann, Sean Gordon, Lindsay Powers, and John Farley
This work was supported by NASA/GSFC under Raytheon Co. contract number NNG10HP02C

2. Big Earth Data Initiative (BEDI)
The Big Earth Data Initiative (BEDI) invests in standardizing and optimizing the collection, management and delivery of U.S. Government’s civil Earth observation data to improve discovery, access use, and understanding of Earth observations by the broader user community.
Complete and consistent standard metadata helps address all three goals.
Many agencies included.

3. Projects/Communities
Big Earth Data Initiative
Data Infrastructure Building Block
Metadata Evolution for NASA Data Systems
Arctic Research Mapping Application
NASA, NOAA,
USGS, USDA,
EPA
IEDA, DataONE
ORNL, CUAHSI
EDAC, NKN
NASA
DAACS,
CMR
ADIWG,
NSIDC
NASA
Communities
DIF, ECHO,
ISO, FGDC,
EML
FGDC, ISO
Custom, EML,
ACDD
DIF, ECHO,
ISO, ISO-1
This slide requires animation – please view in slide show mode.
The BEDI project includes:
-numerous federal agencies such as NASA, NOAA, USGS, USDA, EPA, etc
-numerous dialects
Similar metadata evaluation is being done for other projects and communities
FGDC,
ISO, ISO-1
Dialects

4. Overview Tools and techniques has been developed to:
Compare recommendations and dialects
Identify the structure of metadata collections
Characterize and compare the structure of metadata collections
Evaluate and measure metadata completeness with respect to recommendations
Evaluate and measure metadata completeness with respect to specific organization goals
These tools and techniques were applied to NASA, and USGS metadata collections.
The results of this analysis revealed some interesting findings which are presented in the following Case Studies.

5. Case Study 1: What metadata are important across communities?
Overview: A recommendation is a set of concepts required for achieving a documentation goal. Recommendations often include multiple levels; such as mandatory, recommended and suggested. Purpose: Identify concepts that are both common and not common across mandatory level recommendations. Scope: Recommendations considered include: ACDD Highly Recommended, CSW Core Queryables, DIF Required, ECHO Mandatory, ISO Mandatory, UMM-Common Required Expectation: Significant concept overlap would exist across recommendations.

6. Case Study 1: Results Distribution Mandatory Recommendations Concepts
Bar chart shows the distribution of concepts across recommendations
Y axis = number of communities (recommendations)
X axis = number of concepts
2 concepts (Resource Title and Abstract) were common to all 7 recommendations
1 concept (Theme Keyword) was common in 5 recommendations
1 concept (Modified Date) was common in 4 recommendations
2 concepts (Distribution Contact and BB) in 3 recommendations
9 concepts were common in 2 recommendations
30 concepts were included in just 1 recommendations
Recommendation Comparison Tool:

7. Case Study 2: Do my dialects support my requirements?
Overview: Metadata recommendations change as new communities and needs emerge. Metadata management tools are driven by dialects. Changing those tools and training people are difficult, so adoption of new dialects is relatively slow.
Purpose: Identify gaps between existing organizational capabilities (dialects) and new recommendations (requirements).
Scope:
CSDGM dialect support with respect to DataCite Recommendations.
DIF, ECHO and ISO dialect support with respect for the Catalog Services for the Web Recommendations.
2A Does the CSDGM dialect support the DataCite Recommendations?
2B Do the DIF, ECHO, and ISO dialects support the CSW recommendations?

8. Case Study 2: Results DataCite Recommendation / CSDGM Dialect11 8 7 6 4 3
2A Does the CSDGM dialect support the DataCite Recommendations?
-the blue line shows the number of DataCite concepts
-the green line show the number of DataCite concepts supported by CSGDM
-the space between the lines shows the the number of DataCite concepts not supported by CSDGM
-Blue line the shows the number recommendation concepts for each recommendation level-
-Green line shows the number of recommendation concepts supported by the dialect
-The space between the blue and green line shows the number of recommendation concepts not supported by the dialect (GAP)
Recommendation-Dialect Comparison Tool:

9. Case Study 2: Results CSW Recommendation / NASA Dialects15
GAP
CSW & ISO 10 9 9 8 8 7
DIF 6
2B Do the DIF, ECHO, and ISO dialects support the CSW recommendation?
-GAP exists between CSW recommendation and the DIF ECHO dialects
-note the ISO line shown in green aligns with the dashed CSW line
-this shows that no GAP exists between CSW and ISO
-all CSW concepts are supported by ISO dialect 4
ECHO
Dashed = CSW
Green = ISO, Blue = DIF, Purple = ECHO

10. Case Study 3: How complete is my metadata with respect to recommendations?
Overview: Tools and techniques have been developed to enable the completeness of metadata records and collections to be evaluated with respect to community recommendations (requirements). These tools are very flexible. They enable metadata collections in any supported dialect to be evaluated with respect to any supported recommendation.
Purpose: Evaluate and measure the completeness of NASA and USGS metadata collections with respect to community recommendations.
Scope:
617 CSDGM records from 12 USGS metadata collections evaluated with respect to DataCite.
2600 DIF, ECHO and ISO records from 6 metadata collections evaluated with respect to CSW.

11. Case Study 3: Results USGS CSGM / DataCite
-USGS csgm metadata was evaluted for completeness with respect to the DataCite Recommendation
-3 recommendation levels were considered Mandatory (red), Recommended (blue), Optional (green)
-This graph shows the number of concepts missing for each recommendation level
-Publisher concept was missing from 252 out of 617 records
-Resource Type concept was missing from 293 out of 617 records
-Resource Format concept was missing from 558 out of 617 records
The Figure shows the number of records missing fields in the DataCite recommendation.
Legend: Mandatory (red), Recommended (blue), Optional (green)

12. Case Study 3: Results NASA Climate Data Initiative CSW
-The NASA Cimate Initiative was evaluated for completeness with respect to CSW Core and Optional Recommendations
-2 recommendation levels were considered for 3 dialects (DIF, ECHO, ISO)
-This graph shows the number of CSW Core Concepts missing for DIF, ECHO and ISO dialects
-Core concepts are displayed in red and optional concepts in green.
Review ECHO / ISO for Optional Concepts
The Figure shows the number of records missing fields in the CSW recommendation.
Legend: Mandatory (red), Optional (green)

13. Summary Case Study 1.0 – What Metadata are Important?
Compared Recommendations serving 7 different communities to identify concepts most valued by the communities..
Result – Very little overlap.
Highlights the Recommendation comparison tool.
Case Study 2.0 – Does my dialect support my requirements?
Evaluated 4 dialects to determine how well they support 2 Recommendations.
Considered CSGDM with respect to DataCite.
Considered DIF, ECHO and ISO with respect to CSW.
Identified recommendation concepts not supported by dialects.
Highlights the Dialect/Recommendation comparison tool.
Case Study 3.0 – How complete is my metadata?
Evaluated USGS and NASA Climate Initiative metadata Collections to determine completeness with respect to CSW and DataCite Recommendations.
Identified recommendation concepts missing in the metadata collections.

14. Acknowledgements
This work was supported by NASA/GSFC under Raytheon Co. contract number NNG15HZ39C