Data Ingest Automation GHRC Status and Plans Helen Conover GHRC DAAC Operations Manager Presented at ESIP Summer Meeting 2015

Data Ingest and Publication Issues – Field Campaigns Many small and varied datasets delivered within a short time period Data formats and metadata vocabularies often specific to user community PIs may not have funding to provide full documentation, ATBD, etc. Real time acquisition and distribution of ancillary datasets may be required to support field phase of a campaign 15 June 2015ESIP Summer Meeting 20152

Field Campaign Data Categories and Levels of Service 15 June 2015ESIP Summer Meeting Publication Priority Data Category Skinny Catalog (metrics) Full Catalog (search, landing page) DOI and Citation README Guide 1 NASA research instruments (airborne or ground, with NASA-sponsored PI)  2 Affiliated researcher instruments (e.g., PI from partner university)  3 Other agency research instruments (e.g., PI sponsored by NOAA, DOE)  ?  4 Ancillary research data (e.g., PERSIANN model, TRMM flood maps)  ?  5 Other agency operational data (e.g., GOES imagery, NWS radar, USGS stream gauges)  ?

Current Data Ingest and Publication Information Flow 15 June 2015ESIP Summer Meeting Science Pis Proposed dataset title Brief description Basic metadata Citation author list Data, imagery, documents, software (optional) GHRC Staff Final dataset title GCMD keywords Full metadata DOI Guide document text Master Info Repository Data and Docs Manual text entry FTP Dataset Info Inventor y info CMR (ECHO, GCMD) Dataset Landing Pages Guide Documents Metadata exports Dynamic web docs Downloadable documents exchange

GHRC Approach Plan to adapt ORNL DAAC process, workflow and software Gathering requirements from DBA and Data Management Group o Review current manual processes o Evaluate proposed database schema changes o Define automated metadata publication paths Review design with DMG and User Working Group Agile development approach 15 June 2015ESIP Summer Meeting 20155

Data Ingest and Publication Information Flow 15 June 2015ESIP Summer Meeting Online Metadata Editor Science Pis Proposed dataset title Brief description Basic metadata Citation author list Data, imagery, documents, software (optional) GHRC Staff Final dataset title GCMD keywords Full metadata DOI Guide document text Master Info Repository Data and Docs Manual text entry FTP Dataset Info Inventor y info CMR (ECHO, GCMD) Local Elasticsearch Dataset Landing Pages THREDDS Catalog Guide Documents Metadata exports Dynamic web docs Downloadable documents

Notional Schedule 15 June 2015 ESIP Summer Meeting TaskTimeframe Initial requirements / design phase July - August Early working prototype collection level metadata editor September Preview data provider screens at UWG meeting October 7-8 Implementation and test phase October - November Initial version of complete systemDecember