1. Page 1 CSISS Center for Spatial Information Science and Systems CWIC Development Team Meeting, 2014 Overview CWIC/IDN Synchronization Lingjun Kang lkang3@masonlive.gmu.edu Center for Spatial Information Science and Systems (CSISS) George Mason University CWIC Workshop, Jan 28-30, 2014 Greenbelt, MD

2. Page 2 CSISS Center for Spatial Information Science and Systems Outline Dataset item synchronization (IDN/CWIC TEST) Dataset item synchronization (IDN/CWIC PROD) Dataset valids synchronization Revision of synchronization process

3. Page 3 CSISS Center for Spatial Information Science and Systems Dataset item synchronization (CWIC/IDN TEST) 1. When new ‘CWIC’ datasets are added in IDN TEST: –Detect new datasets tagged with ‘CWIC’ in IDN TEST –Semi-automatically collect dataset info from IDN TEST Native dataset ID –For ECHO datasets, this info was manually collected. Data provider –manually identify which portal (ECHO, USGSLSI, GHRSST, etc) serves new datasets Valids (spatial/temporal range) –Automatically harvest from IDN CSW

4. Page 4 CSISS Center for Spatial Information Science and Systems Dataset item synchronization (CWIC/IDN TEST) –Update CWIC TEST list with collected info Special case for USGSLSI/INPE: also update internal mapping file (source.json) –Testing for new datasets If datasets pass testing, notify IDN to migrate datasets to IDN PROD. If datasets do not pass testing, notify data provider

5. Page 5 CSISS Center for Spatial Information Science and Systems Dataset item synchronization (CWIC/IDN TEST) 2. When ‘CWIC’ datasets are removed from IDN TEST: –Identify datasets untagged with ‘CWIC’ in IDN TEST –Automatically testing for these datasets –Notify data provider and IDN with testing results for final decision –Good practice was realized among CWIC, IDN and GHRSST. Because we know GHRSST point of contact (Mrs. Li) who manages their repository and makes final decision. IDN point of contact (Michael) who helps us to update ‘CWIC’ datasets in IDN.

6. Page 6 CSISS Center for Spatial Information Science and Systems Dataset item synchronization (CWIC/IDN PROD) –Detect new datasets tagged with ‘CWIC’ and datasets untagged with ‘CWIC’ in IDN PROD –Automatically update dataset list in CWIC PROD. But since some datasets without testing were either added or removed from IDN PROD, the procedure was changed to ‘manual’ procedure. More human checking and third party confirmation is required before a dataset is added to or removed from CWIC PROD.

7. Page 7 CSISS Center for Spatial Information Science and Systems Dataset valids synchronization –Harvest spatial/temporal valids from IDN TEST/PROD –Automatically update dataset valids in CWIC TEST/PROD.

8. Page 8 CSISS Center for Spatial Information Science and Systems Improve dataset item synchronization between IDN/CWIC TEST Premise of automatic synchronization: –Data provider is required to provide and input native dataset ID (if different from entry ID) during registration if they would like to make their datasets accessible through CWIC. –Dataset should not be tagged with ‘CWIC’ until corresponding native dataset ID is provided. –Both native dataset ID and data provider info are stored in IDN mapping table and queryable through web API. With the premises, once new datasets are detected by CWIC syn, corresponding native dataset ID and data provider info will be automatically acquired. Revision of synchronization process

9. Page 9 CSISS Center for Spatial Information Science and Systems Revision of synchronization process Improve dataset item synchronization between IDN/CWIC PROD Recommendation for adding new datasets to IDN PROD: –Three premises for datasets qualified for being migrated to IDN PROD (1) Datasets should be in both IDN/CWIC TEST. (2) Datasets should pass testing in CWIC TEST. (3) Adding datasets should be carried out after positive testing result is given from CWIC. Recommendation for removing datasets from IDN PROD –CWIC should be notified with dataset removal before removal is carried out. Then CWIC can assist IDN and data provider in final decision by providing testing results.

10. Page 10 CSISS Center for Spatial Information Science and Systems Question & Discussion