ESDIS METADATA ARCHITECTURE STUDY DISCUSSION Presented by Kevin Murphy and the Metadata Architecture Study Team Members 2013 ESIP Summer Meeting | Chapel.

Slides:



Advertisements
Similar presentations
What is ECHO? HTTP-based Search and Ordering Using ECHOs REST and OpenSearch APIs How Can.
Advertisements

NASA Earth Science Data Preservation Content Specification H. K. (Rama) Ramapriyan John Moses 10 th ESDSWG Meeting – November 2, 2011 Newport News, VA.
EOSDIS Common Metadata Repository (CMR) Unified Metadata Model for Collections (UMM-C) Meeting June 12, 2014.
NASA COMMON METADATA REPOSITORY (CMR) Update and Near Term Plans – CEOS WGISS 37 1 Andrew Mitchell Earth Science Data and Information Systems (ESDIS) National.
Transformations at GPO: An Update on the Government Printing Office's Future Digital System George Barnum Coalition for Networked Information December.
NOAA Metadata Update Ted Habermann. NOAA EDMC Documentation Directive This Procedural Directive establishes 1) a metadata content standard (International.
System Design/Implementation and Support for Build 2 PDS Management Council Face-to-Face Mountain View, CA Nov 30 - Dec 1, 2011 Sean Hardman.
Data Ingest Automation GHRC Status and Plans Helen Conover GHRC DAAC Operations Manager Presented at ESIP Summer Meeting 2015.
What is Business Analysis Planning & Monitoring?
Metadata Tools and Methods Chris Nelson Metanet Conference 2 April 2001.
Updates from EOSDIS -- as they relate to LANCE Kevin Murphy LANCE UWG, 23rd September
Status of the Antarctic Master Directory SCADM Meeting, August 22, 2014.
North American Profile: Partnership across borders. Sharon Shin, Metadata Coordinator, Federal Geographic Data Committee Raphael Sussman; Manager, Lands.
GCMD/IDN STATUS AND PLANS Stephen Wharton CWIC Meeting February19, 2015.
Documentation from NcML to ISO Ted Habermann, NOAA NESDIS NGDC.
Sharing Metadata Recommendations Ted Habermann, John Kozimor Earth Science The HDF Group 1 John Farley Raytheon.
Consolidated Metadata Repository (CMR) Status and Look Ahead CWIC 2015 Annual Meeting, February 18-19, 2015 This work was supported by NASA/GSFC under.
Adoption of RDA-DFT Terminology and Data Model to the Description and Structuring of Atmospheric Data Aaron Addison, Rudolf Husar, Cynthia Hudson-Vitale.
What is CWIC? Authors: Doug Newman Andrew Mitchell
Using the Global Change Master Directory (GCMD) to Promote and Discover ESIP Data, Services, and Climate Visualizations Presented by GCMD Staff January.
WGISS-40: IDN Report Michael Morahan WGISS-40 Fall meeting / Harwell, United Kingdom
Adoption of RDA-DFT Terminology and Data Model to the Description and Structuring of Atmospheric Data Aaron Addison, Rudolf Husar, Cynthia Hudson-Vitale.
National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology Pasadena, California EDGE: The Multi-Metadata.
Retrieving and Transforming ISO Metadata ESIP Summer 2014.
COMMON METADATA REPOSITORY (CMR) Update and Near Term Plans – ESIP Winter ESIP Winter Meeting | Washington DCJanuary Dan Pilone
NASA Perspectives on Data Quality July Overall Goal To answer the common user question, “Which product is better for me?”
EOSDIS User Registration System (URS) 1 GES DISC User Working Group May 10, 2011 GSFC, NASA.
ESO Developing and Emerging Standards, Practices, and Technologies Yonsook Enloe, Helen Conover, Allan Doyle ESDIS Standards Office 7/14/ Summer.
PoDAG XXI: SEEDS SEED: NSIDC Potential Interactions NSIDC DAAC should prepare an evaluation of their desired future roles in "core activities" and in mission.
Documenting UAF Data Ted Habermann NOAA/NESDIS/National Geophysical Data Center.
Registering Earth Science Data and Data Related Services Using NASA’s Global Change Master Directory (GCMD) Tyler Stevens (GIS/Services Coordinator) ESIP.
ASDC Ingest Automation Efforts through Collaboratory for quAlity Metadata Preservation (CAMP) Presented by: Aubrey Beach (Booz Allen Hamilton, NASA ASDC)
1 1 ECHO Extended Services February 15, Agenda Review of Extended Services Policy and Governance ECHO’s Service Domain Model How to…
Public Access and Spatial Metadata Values: Semantic Network Services Response to EU Directives Maria Rüther Federal Environment Agency,
CWIC Developers Meeting January 28 th 2014 Calin Duma CSW and OpenSearch from the CWIC Start client perspective.
ECHO AND ISO ESO SIG June 18, 2013 Raytheon EED Program | ESO SIG: ECHO and ISO.
Foundational Program Overview September  2004 Copyright RosettaNet. RosettaNet Foundational Programs Program Overview ProgramPhase InvestigateDesignImplement.
Report of the Architecture and Data Committee (ADC) R.Shibasaki (ADC, Japan)
Evaluate Phase Pertemuan Matakuliah: A0774/Information Technology Capital Budgeting Tahun: 2009.
WISE Working Group D September 2009, Brussels Jon Maidens.
LPDAAC/USGS_EROS CMR Metadata Reconciliation Meeting March 4-5, 2015.
NSIDC CMR Metadata Reconciliation Meeting December 10-11, 2014.
The Proliferation of Metadata Standards and the Evolution of NASA’s Global Change Master Directory (GCMD) Standard for Uses in Earth Science Data Discovery.
Science Review Panel Meeting Biosphere 2, Tucson, AZ - January 4-5, 2011 Vegetation Phenology and Vegetation Index Products from Multiple Long Term Satellite.
Global Change Master Directory (GCMD) Mission “To assist the scientific community in the discovery of Earth science data, related services, and ancillary.
What is ECHO? ECHO Open Search ECHO Facts NASA’s Earth Observing System ClearingHOuse (ECHO) acts as the core metadata.
ESO and the CMR Life Cycle Process Winter ESIP, Jan 2015 ESDIS Standards Office (ESO) Yonsook Enloe Allan Doyle Helen Conover.
Adoption of RDA-DFT Terminology and Data Model to the Description and Structuring of Atmospheric Data Aaron Addison, Rudolf Husar, Cynthia Hudson-Vitale.
1 Raytheon EED Program | ECHO Technical Interchange 2013.
DM_PPT_NP_v01 Configuration Management of UMM Models January 2016.
ISWG / SIF / GEOSS OOSSIW - November, 2008 GEOSS “Interoperability” Steven F. Browdy (ISWG, SIF, SCC)
PDS4 Project Report PDS MC F2F UCLA Dan Crichton November 28,
PDS4 Project Report PDS MC F2F University of Maryland Dan Crichton March 27,
ESA UNCLASSIFIED – For Official Use INSPIRE Orthoimagery TWG Status Report Antonio Romeo ESRIN 15/02/2012.
1 Digital Object Identifiers Update ESIP Data Stewardship Committee Meeting May 16, 2016 Presenters: Nate James, ESDIS Lalit Wanchoo, ADNET Systems Inc.
ISWG / SIF / GEOSS OOS - August, 2008 GEOSS Interoperability Steven F. Browdy (ISWG, SIF, SCC)
International Planetary Data Alliance Registry Project Update September 16, 2011.
DAAC Roles with Common Metadata Repository (CMR) ESDIS System Engineering Technical Interchange November 2014 Copyright © 2014 Raytheon Company. All Rights.
ESDIS Standards Office Yonsook Enloe, CTSI 3/2/ FGDC Meeting1.
Geospatial metadata Prof. Wenwen Li School of Geographical Sciences and Urban Planning 5644 Coor Hall
IPDA Registry Definitions Project Dan Crichton Pedro Osuna Alain Sarkissian.
CrossCutting topic: Data Quality and European Network of EO Networks
NASA Earth Science Data Stewardship
Michael Morahan CEOS WGISS-43 Meeting
CAP-378 and “Conhecer para não ignorar”
Persistent Identifiers Implementation in EOSDIS
WGISS-41: IDN Report Michael Morahan CEOS WGISS-41 Meeting
A Case Study for Synergistically Implementing the Management of Open Data Robert R. Downs NASA Socioeconomic Data and Applications.
Proposal of a Geographic Metadata Profile for WISE
IT Next – Transformation Program
Presentation transcript:

ESDIS METADATA ARCHITECTURE STUDY DISCUSSION Presented by Kevin Murphy and the Metadata Architecture Study Team Members 2013 ESIP Summer Meeting | Chapel Hill, NC

Outline Background MAS Phase I MAS Phase II Community Feedback Open Discussion 2 July 11, ESIP Summer Meeting | Chapel Hill, NC

BACKGROUND 3 July 11, ESIP Summer Meeting | Chapel Hill, NC

EOSDIS Mission The Earth Science Data and Information System (ESDIS) project provides access to data through the development and operation of the science systems of the Earth Observing System (EOS) Data and Information System (EOSDIS). July 11, ESIP Summer Meeting | Chapel Hill, NC 4

Metadata Architecture Study (MAS) ESDIS initiated a study of EOSDIS metadata capabilities/processes in the spring of 2012 to evaluate opportunities for improved data access and use, reduce efforts by data providers, and improve metadata integrity. Goals: Unify several disparate metadata formats (DIF, ECHO, EMS). Provide consistent results, performance, and access to users regardless of the mechanism through which they access EOSDIS metadata. Improve the consistency and quality of EOSDIS metadata holdings. Provide a platform capable of expanding EOSDIS metadata systems to incorporate services, tools, documents, etc. 5 July 11, ESIP Summer Meeting | Chapel Hill, NC

Ground Rules Metadata Harmony Study EOSDIS metadata are currently a widely used resource which must remain available with at least the same level of service during system definition, development and implementation. Enhancements to metadata systems must focus on improving user interactions with the system (End Users, International Users, Data Providers), sustaining engineering of the system, and the integrity of metadata. 6 July 11, ESIP Summer Meeting | Chapel Hill, NC

METADATA ARCHITECTURE STUDY (PHASE I) 7 July 11, ESIP Summer Meeting | Chapel Hill, NC

MAS I Research Study Objective: Identify opportunities for improved data access and use, reduce efforts by data providers, and improve metadata integrity. Of interest were the overlapping roles and interfaces between the GCMD and ECHO. The five-member study team included representatives from ESDIS, GCMD, ECHO, EMS, and the DAACs. Results - Eight recommendations in four categories: 8 Common Approaches to Metadata (2) Improved Communications (4) Keywords (1) Looking Forward (1) July 11, ESIP Summer Meeting | Chapel Hill, NC

MAS I Recommendations Move to a single, shared NASA Earth Science Metadata repository for submission of collection level metadata. ESDIS should define a consistent dataset identifier policy to be used across DAACs and central data systems (ECHO, GCMD, EMS). Consider a regular communication forum to address and clarify ESDIS requirements. Schedule periodic technical telecons between GCMD and the DAACs to provide insight into new or planned capabilities. GCMD should implement a mechanism to notify DAACs of DIF and SERF structure modifications. Both ECHO and GCMD should develop a DAAC-accessible mechanism for tracking implementation plans and status for requested changes. GCMD should roll out the new Keyword Management System and its RESTful APIs to the DAACs as soon as practical. ESDIS should continue to monitor emerging metadata needs and related technologies, and work with the community to define conventions for these new metadata. 9 July 11, ESIP Summer Meeting | Chapel Hill, NC

METADATA ARCHITECTURE STUDY (PHASE II) 10 July 11, ESIP Summer Meeting | Chapel Hill, NC

MAS II Architecture Development Objectives: Develop a Unified Metadata Model (UMM) that unifies the EOSDIS metadata assets into one cohesive model. Good position for future ISO compatibility. Create a Concept of Operations document that provides an overview of how the Unified Metadata Model (UMM) and Common Metadata Repository (CMR) will be used. 11 Unified Metadata Model (UMM) Common Metadata Repository (CMR) Metadata Harmony July 11, ESIP Summer Meeting | Chapel Hill, NC

UNIFIED METADATA MODEL (UMM) July 11, ESIP Summer Meeting | Chapel Hill, NC 12

Overview Unification Process What was in scope, what was not in scope Vocabulary UMM Core Vocabularies Extendable Vocabularies Knowledge Base System Documentation July 11, ESIP Summer Meeting | Chapel Hill, NC 13

Unification Process Initial focus is to unify the following models: Current conversions are “lossy”. By unifying these models ESDIS can latter focus on creating one mapping to ISO utilizing the upcoming NASA “flavor”. The ISO model was mapped where possible, and will be in scope for a future phase. 14 Metadata July 11, ESIP Summer Meeting | Chapel Hill, NC

Unifying the Models DIF ECHO Data Set Citation Personnel Instrument Platform Temporal Coverage Paleo-Temporal Coverage Spatial Coverage Location Data Resolution Project Quality Access Constraints Use Constraints Distribution Data Set Language Data Set Progress Related URL DIF Revision History CitationForExternalPublication ProcessingLevelId Price SpatialKeywords TemporalKeywords Temporal Contact ScienceKeywords Platform Instrument Sensor Campaigns TwoDCoordinateSystem OnlineAccessURL OnlineResource AssociatedDIFs Spatial ArchiveCenter AdditionalAttributes AssociatedBrowseImages SpatialInfo CollectionProgress RestrictionComments 15 July 11, ESIP Summer Meeting | Chapel Hill, NC EMS Not Shown

Unified Fields 16 July 11, ESIP Summer Meeting | Chapel Hill, NC

Vocabularies UMM Core Vocabularies Extendable Vocabularies July 11, ESIP Summer Meeting | Chapel Hill, NC 17

Benefits of Core Vocabularies Provides a classification system for metadata. Prevents zero-hit queries. Improves metadata accuracy and completeness. Improves the usability of earth science metadata. 18 July 11, ESIP Summer Meeting | Chapel Hill, NC

Core Vocabularies - Unique Keyword Identifiers Controlled vocabularies are referenced in the UMM using keyword identifiers (UUID’s) to SKOS concepts in the GCMD Keyword Management System Service. Updates to keywords referenced in metadata happen automatically since keywords are only “referenced” using these identifiers. Additions and/or changes to the vocabularies can be made through the existing governance process. 19 July 11, ESIP Summer Meeting | Chapel Hill, NC

UMM Core Vocabulary Types Science Keywords Platforms Instruments Sensors Projects Organizations Organization Types Locations Temporal Resolution Range Keywords Horizontal Resolution Range Keywords Vertical Resolution Range Keywords Chronostratigraphic Units Dataset Languages Related URL Types and Protocols ISO Topic Categories DataSet Progress DataSetLanguage Temporal Coverage Types Processing Level Identifiers Units for Resolutions Units for Geographic Coordinates 20 July 11, ESIP Summer Meeting | Chapel Hill, NC

Extendable Vocabularies AncillaryKeyword Class Provides a way to reference controlled vocabularies external to the CMR. This UMM class has a resource attribute associated with it, allowing the metadata author to reference an external RDF concept with its own namespace identified through a URI. Example: Wind Speed The example above links to a CF standard_name keyword provided by the Marine Metadata Interoperability Project (MMI). 21 July 11, ESIP Summer Meeting | Chapel Hill, NC

July 11, ESIP Summer Meeting | Chapel Hill, NC 22 Knowledge Based System (Many Interconnections)

UMM Documents (Finalized by Early August) UMM Overview Document High level UML diagrams of key parts of the model and descriptions of how metadata can be represented in the model. UMM Crosswalk Study Document Analysis of each model element, similarities, differences, and provides recommendations. UMM Crosswalk Spreadsheet Detailed mapping between DIF, ECHO, EMS. Maps UMM to ISO (where possible). UMM Schema created from the model UMM is not intended to be a format, but can be useful to describe in this way. Online Javadoc style navigation of the UMM Schema 23 July 11, ESIP Summer Meeting | Chapel Hill, NC

COMMON METADATA REPOSITORY (CMR) July 11, ESIP Summer Meeting | Chapel Hill, NC 24

2013 ESIP Summer Meeting | Chapel Hill, NC URS GIBS EMS EOSDIS Discovery Tools DAACs Near Real Time (LANCE) ??? CMR July 11,

2013 ESIP Summer Meeting | Chapel Hill, NCJuly 11,

2013 ESIP Summer Meeting | Chapel Hill, NCJuly 11,

Metadata Concept Support 2013 ESIP Summer Meeting | Chapel Hill, NCJuly 11,

2013 ESIP Summer Meeting | Chapel Hill, NCJuly 11, Extensibility

2013 ESIP Summer Meeting | Chapel Hill, NC ECHO 10GCMD DIFEMS ISO19115 UMM Ingest Adapter VisualizationDocumentationServicesParameters July 11,

Improving Metadata Quality with Assessment and Scoring 2013 ESIP Summer Meeting | Chapel Hill, NCJuly 11,

2013 ESIP Summer Meeting | Chapel Hill, NCJuly 11,

2013 ESIP Summer Meeting | Chapel Hill, NCJuly 11,

How Will This Affect Me? July 11, ESIP Summer Meeting | Chapel Hill, NC 34

Metadata Providers You won’t have to reconcile multiple systems Most systems will continue to work ECHO FTP ingest going away Current formats will continue to function You will need to be engaged in the adaptation process You will need to support controlled vocabularies You will have a voice through the Technical Committee July 11, ESIP Summer Meeting | Chapel Hill, NC 35 Streamlined Processes, More Engagement

Client Developers 36 More Metadata, Better Quality Richer, higher quality metadata will be available Existing APIs will be supported Standards-compliant APIs will be available Metadata will be available in an ISO19115 compliant format July 11, ESIP Summer Meeting | Chapel Hill, NC

End Users 37 More Metadata, Better Quality Richer, higher quality metadata will be available Unified view of EOSDIS holdings Easier access through a variety of clients, data casting, etc Metadata will be available in an ISO19115 compliant format July 11, ESIP Summer Meeting | Chapel Hill, NC

COMMUNITY FEEDBACK 38 July 11, ESIP Summer Meeting | Chapel Hill, NC

Early Reviewers NASA Standards Interest Group (SIG) Review Stakeholder Review/Interviews 39 Review Groups July 11, ESIP Summer Meeting | Chapel Hill, NC April 2013May – June 2013 June – July 2013

Community Feedback Process Reviewers submitted feedback and received comments using the EOSDIS Issue Tracking Software – JIRA 334 “Issues” created: 206 Resolved/Closed, 68 for Future Study 40 July 11, ESIP Summer Meeting | Chapel Hill, NC

Summary of Feedback Received Provide consistent product identifiers (i.e. UUID’s) Address ISO and NASA flavor of ISO Include more required fields Provide common values for project, platform, instrument Offer more controlled vocabulary…such as processing level Contain more structured fields...such as resolution units Provide types and descriptions for any URL Include provenance tracking 41 July 11, ESIP Summer Meeting | Chapel Hill, NC

Next Steps 42 July 11, ESIP Summer Meeting | Chapel Hill, NC Wrap-Up/Document Delivery (Early August) 1 Requirements, Prototyping, and Design 2 Implementation and Testing 3 Data Cleanup/Transition 4

MAS II Team Lead: Kevin Murphy ECHO Representatives Katie Baynes John Farley Dan Pilone EMS Representative Lalit Wanchoo GCMD Representatives Thomas Cherry Tom Northcutt Scott Ritz 43 July 11, ESIP Summer Meeting | Chapel Hill, NC

OPEN DISCUSSION 44 July 11, ESIP Summer Meeting | Chapel Hill, NC