1 2.5 DISTRIBUTED DATA INTEGRATION WTF-CEOP (WGISS Test Facility for CEOP) May 2007 Yonsook Enloe (NASA/SGT) Chris Lynnes (NASA)

2 CEOP Data Access CEOP science community uses OPeNDAP based clients to access in situ and model data JAXA assisting CEOP central archives in making their data available via OPeNDAP –In situ, model and satellite data NASA providing satellite data subsets to Univ. of Tokyo NASA also developing a tool to improve access to distributed satellite data collections

3 NASA Contribution: NASA Satellite Data Request for selected MODIS and AIRS products for EOPs submitted to NASA. Products will be provided by multiple facilities. –AIRS – Goddard DAAC –180 TB MODIS Atmosphere, Land – MODAPS –MODIS Ocean – Goddard ODPS Technical discussions to refine request underway –Metadata specifications –Data service requests – format, projection, etc. –Delivery schedules

4 NASA Contribution: WTF-CEOP Prototype NASA’s focus is on improving CEOP community access to satellite data. In early discussions, CEOP scientists identified need for easier access to satellite data. The NASA prototype will facilitate the use of satellite data for inter-comparisons with model output and reference site data and lead to better model improvements. CEOP scientists also desire direct access to full range of satellite data products and services from base providers –Latest versions of products –Flexible access to data services Growing use of Open Geospatial Consortium (OGC) specifications by satellite data providers Initial proof of concept demonstrated at CEOP and CEOS meetings

5 Integration of Satellite and Model Data Using Grads Client

6 Prototype vs. Real Need Science need - –Data discovery / data search –CEOP science community makes use of Grads and other OPeNDAP clients extensively –OPeNDAP client access to the full range of WCS services for satellite data NASA developed a CEOP Satellite Data Server Prototype that demonstrated OPeNDAP client (Grads) access to the OGC-WCS server capabilities –Demonstrated ability to access satellite data from OPeNDAP client No catalog services Default data services –Complement to the CSDIC at Univ. of Tokyo Support from NASA’s Advancing Collaborative Connections for Earth- Sun System Science (ACCESS) Program –Combined with second proposal - OGC to OPeNDAP Gateway

7 NASA Prototype: Satellite Data Server Web Coverage Server (NWGISS WCS) WCS Client Satellite Data, Swath and Grid Grads Client Grads-DODS Server (GDS) Other OPeNDAP Clients Field Data Model Output Standard Functions e.g. subsetting Standard Functions e.g. subsetting Custom Functions (Grads) OPeNDAP Request Translator Default georectification CF Attrubute Constrtuction Custom Requests (future)

8 “Virtual” Gateway Functions/Approach Transformation of WCS metadata to be consistent with other CEOP data sources –Climate and Forecast Metadata Conventions –Must handle various coordinate reference systems –Augment with metadata on requested WCS operations Enable request of WCS operations –Construct DAP request to include WCS services (Georectification, reprojection, subsetting, …) –Issue request to WCS –WCS Response in NetCDF CF understood by baseline Hyrax DAP Server 4 release.

9 Status and Plans Project initiated April 2006 Prioritization of WCS services discussed with CEOP science representative First release scheduled for June 2007 OPeNDAP Group’s DAP Server (Hyrax) releases will contain modifications needed for general OPeNDAP client access to WCS servers – to facilitate additional OPeNDAP clients accessing WCS servers Plan to deploy the Satellite Data Server at NASA and other sites with satellite data of interest to the CEOP community – however no operational commitment yet Plan to provide access to CEOP science community to the prototype for feedback and iteration

10 The OGC-Geoscience Catalog Need of the Catalog –Make Geoscience catalog (THREDDS), and the cataloged data in the OPeNDAP community available to the OGC community –Make THREDDS catalog searchable through OGC interface THREDDS catalog doesn’t have a searchable interface itself Design of the Catalog –OGC Catalog Service for Web (CSW) ebRIM profile for the interface –Use ISO metadata standard as the metadata information model in CSW –Map THREDDS XML metadata schema into ISO metadata schema –Ingest THREDDS XML catalog file into database accessible by the CSW server Current Status –Finished the design of THREDDS to ISO19115 metadata mapping –Implemented an Initial version of THREDDS-to-CSW ingestor –Deployed the initial version of the ingestor –Ingested samples of THREDDS catalog, including those in Unidata and NCDC, into the CSW database for testing