Consolidated Metadata Repository (CMR) Status and Look Ahead CWIC 2015 Annual Meeting, February 18-19, 2015 This work was supported by NASA/GSFC under Raytheon Co. contract number NNG10HP02C. 1

Interactive Data Discovery High Quality Metadata Fast Search (Spatial!) High Scalable Fault Tolerant

Common Metadata Repository (CMR) A single source of unified, high-quality, and reliable Earth Science metadata A high performance ingest and search architecture for submission and discovery of Earth Observing System Data and Information System (EOSDIS) metadata A metadata lifecycle that helps evolve metadata standards to ensure ‘richness’ and models with new concepts proposed by stakeholders

CMR Policy CMR will be the authoritative management system for all EOSDIS metadata for all EOSDIS data holdings. With ECHO it was our policy to never change metadata that was sent to us by the provider. If changes needed to be made, it was up to the provider to make those updates. With CMR, we will be providing more of a hands-on role with CMR metadata. In concert with the data provider, the CMR Operations team will be able to support metadata creation and enhancements (single or bulk). This will also be exercised through metadata adapters. –This policy will enforce higher quality metadata. The adapter solution allows this to happen quickly (making CMR more nimble) while allowing the data providers (who have their own schedules and priorities) to catch up on a mutually agreed-upon plan. 5

CMR STATUS CWIC 2015 Annual Meeting, February 18-19,

CMR MVP Status 7

CMR MVP (Minimum Valuable Product) provides the minimal set of functionality that address the major goals laid out in a CMR Operations Concept. 8

Current MVP Activities Sub-second search CMR Search Results being returned Unification of EMS, ECHO, and GCMD metadata Enhanced metadata management capabilities with the Metadata Management Tool (MMT) Metadata Validation Service (ECHO10, DIF, and ISO) [will be incorporated in MMT] 9

CMR In the Real World 10

Pre-CMR Overview 11

CMR Rollout and Testing 12

CMR in Operations 13

Common Metadata Repository Performance Snapshot – Average Comparison 14

Common Metadata Repository Performance Snapshot – 99 th Percentile Comparison 15

CMR Impact on Operations ECHO’s REST API is fully supported by the CMR, however… CMR’s native REST API is slightly faster and provides more capabilities –inline facets, more complex search expressions, etc. ECHO OpenSearch is fully supported and will leverage enhanced CMR capabilities 16

Metadata Quality Improvements IDN will leverage CMR for high performance and high quality metadata Metadata is being reconciled between IDN/GCMD and CMR to a single record Metadata Quality is a first class capability in the Metadata Management Tool (MMT) and CMR 17

NEXT STEPS CWIC 2015 Annual Meeting, February 18-19,

Candidate CMR Enhancements Search Relevancy Ranking Semantic and Relationship Search DOI assignments Subscriptions Datacasting support 19

Metadata Management Tool (MMT) The MMT will include functionality for: –Assessing the quality of CMR metadata –Suggesting revisions to metadata based on the quality assessment –Notifying providers of the suggested revisions –Allowing providers to accept or reject the suggested revisions –Science Keyword Support 20

Defining processes: CMR Life Cycle Document – Describes process for assessing, reviewing, and approving requirements for all of CMR including UMM concepts, GCMD Keywords, and other controlled vocabularies GCMD Keywords Community Guide – Targeted for the community, contains information on GCMD keywords, keyword structure and rules, and an overview of how GCMD engages with the community to keep the keywords relevant and useful 21

Implementing processes for: UMM-C (Collections) - Metadata model for collections that harmonizes the GCMD DIF and ECHO10 models, including mappings to ISO UMM-G (Granules) - Metadata model for granules, largely based on ECHO10  UMM-S (Services) – Metadata model for services, initially based on GCMD SERFs UMM-P (Parameters/Variables) – Metadata model for parameters/variables (e.g. AESIR) UMM-V (Visualization) – Metadata model to support visualization tools, initially GIBS UMM-D (Documentation) – Metadata model to support documents (use cases TBD) 22

Metadata Quality Improvements Provide enhanced Metadata Quality checking services Clearly define Operations Concept and Governance Model for CMR as a product of the GCMD and ECHO merger. Solidify and document the User Support and Issue Resolution workflow and processes. 23

TIMELINE CWIC 2015 Annual Meeting, February 18-19,

Schedule (1 of 2) ActivityApproximate TimelineNotes CMR Subsecond SearchReleased Jan. 12 th 2015 Faster Search Response CMR Ingest, REST Ordering and Validation Tools Mid 2015 Direct CMR Ingest Optional REST Ordering Implementation Dry-run Metadata Validation Metadata Management Tool March 2015 (Requirements &Design) PUMP Replacement Metadata Authoring Earthdata Search Client Operational Mid 2015 New Web-based Search Interface Metadata Concept Implementation Scheduling in progress Search interfaces for parameter metadata Service metadata Visualization metadata Results Relevancy ranking and sorting 2 nd half 2015 More informative search results 25

Schedule (2 of 2) ActivityApproximate TimelineDAAC Impacts ECHO SOAP API Retirement Order processing ACL management Group management, etc Throughout 2015 All SOAP interfaces must be REST compatible SOAP Order fulfillment is not slated for retirement Metadata ReconciliationThroughout 2015 FTP Ingest Retirement Provider by Provider basis as part of Metadata Reconciliation SERF Migration into CMR2 nd half 2015 GCMD Services available in CMR Begin Migration of GCMD Interfaces to Utilize CMR Backend May

QUESTIONS? CWIC 2015 Annual Meeting, February 18-19,