1. CWIC/IDN synchronization Lingjun Kang (lkang3@masonlive.gmu.edu)lkang3@masonlive.gmu.edu Michael Morahan (michael.p.morahan@nasa.gov)michael.p.morahan@nasa.gov

2. Outline CWIC/IDN sync overview – Workflow – Module interaction Tests supporting synchronization Integration of sync/dataset testing/audit – Sync-test-sync model – Sync-audit-sync model Syn in IDN Future plan – Contact info synchronization

3. CWIC/IDN sync overview sync  test IDN mapping table IDN CSW/GCMD/OS 2. Harvest existing dataset entries (‘prod’ tag in IDN) 1. Harvest new dataset entries (tagged as ‘test’ in IDN) sync  test  audit  sync Tag datasets with ‘prod’ (passing testing) Tag datasets with ‘test’ (not passing dataset)

4. Tests supporting synchronization CWIC/IDN synchronization is achieved by tagging dataset entries in IDN based on testing results: – Dataset passing tests  Be published to ‘CWIC’ accessible list in IDN – Dataset failing tests  Be removed from ‘CWIC’ accessible list in IDN – Dataset returning 0 granule in tests  Be removed from ‘CWIC’ accessible list in IDN

5. Tests supporting synchronization Accessibility testing – Objective Verify if all granules under a dataset are searchable from data provider metadata inventory – Methods Query dataset against corresponding metadata inventory

6. Tests supporting synchronization Query and check granule number for each dataset – Testing each dataset with requests containing spatial/temporal parameters being harvested form IDN – Check granule number

7. Tests supporting synchronization Query dataset holding from data provider API (if available, e.g. NASA ECHO RESTFUL API) – Testing is performed on single HTTP GET transaction for all datasets – Less system workload is imposed in data provider inventory – Not skew the metrics of both CWIC and data provider inventory

8. sync/testing/audit integration Sync  Test  Sync – For new dataset entries from IDN Harvest dataset entries (dataset ID) and corresponding valids (spatial/temporal) Create testing suite (construct request based on valid/other request options) Run the testing suite Tag IDN based on testing results (automatic or manual)

9. sync/testing/audit integration Sync  Audit  Sync – For existing dataset entries in both IDN and CWIC (daily) Update dataset valids (if valid is changed in IDN, e.g. NRT dataset) Create testing suite (construct request based on valid/other request options) Run the testing suite Tag IDN based on testing results (automatic) – If fails/0 granule  tag with ‘test’ – If succeed  audit granule number

10. CWIC sync/testing/audit integration Support of module integration Sync moduleTest/audit module IDN Tag module Sync valids from IDN Read valids and test Update testing status Read testing status and tag Update tagging status

11. CWIC sync/testing/audit integration Testing summary example Testing status Tagging status: tag ‘test’ in IDN mapping table

12. CWIC sync/testing/audit integration Testing summary example Testing status Tagging status: tag ‘test’ in IDN mapping table

13. IDN Test (DEV) CSW server Process: CWIC Mapping Table Synchronization Benefits: tagging a record without modifying the DIF entry. Add Entry_ID, Data Provider ShortName, and either the Entry_Title or ECHO Dataset ID to the mapping table for the new entry.

14. IDN Production CSW server Process: CWIC Mapping Table Synchronization Update cwic_status tag to "value=prod” by CWIC or IDN.

15. IDN CWIC Portal (OpenSearch) Process: CWIC Mapping Table Synchronization Add Entry_ID, Data Provider ShortName, and either the Entry_Title or ECHO Dataset ID to the mapping table for the new entry. CWIC OpenSearch Portal links: Production: http://gcmd.nasa.gov/KeywordSearch/default/ openSearch.jsp?Portal=cwic Test: http://gcmd.nasa.gov/KeywordSearch/default/ openSearch.jsp?Portal=cwictest

16. Future plan Synchronization of dataset level info (TBD) – Contact info – Data center info (hierarchy structure of ownership) – Other?

17. Questions and Discussion