1. CWIC Status Report Yonsook Enloe yonsook. k. enloe@nasa
CWIC Status Report Yonsook Enloe WGISS-44 Sept 26, 2017

2. Architecture WGISS Data Assets

3. CWIC CWIC Mechanism Heterogeneous protocols CWIC data partners Open
Search
CSW
CWIC
Heterogeneous
protocols
CWIC data partners

4. Welcoming CWIC Data Partners
Data Partner Guides (CSW & OpenSearch)
To be better integrated with CWIC, data provider inventory systems are expected to meet the following requirements:
Query by dataset identifier
Spatial/temporal query
Support pagination
Support returning essential metadata info
Exception handling

5. What Metadata is Returned in Search Response?
Responses Dublin Core or ISO (CSW) or Atom (OpenSearch)
Collection contact information
Granule identifier information
Granule temporal extent
Granule spatial extent
Browse image links
Download/order links

6. Data Provider Protocols and Requirements
Requirements of exception handling
Data provider remote inventory system should indicate errors and return exception status through either application level (e.g. CSW exception response) or generic protocol level (e.g. HTTP status code)

7. Adding CWIC Datasets Publishing new CWIC datasets through four stages
Stage description
Stage
Registering CWIC datasets in IDN (CMR)
Registration stage
Synchronizing datasets from IDN (CMR) in CWIC
Testing stage
Testing datasets in CWIC
Tagging ready-to-search CWIC datasets in IDN(CMR)
Tagging stage
CWIC datasets are:
Collection-level info is ready to search in IDN (CMR)
Granule-level info is ready to search in CWIC
Production stage

8. Trending: CEOS OpenSearch
CEOS OpenSearch kickoff meeting in May 2013
V1.0 released in May 2015
V1.2 released in June 2017
Data partners are implementing CEOS OS:
USGS converted to OS from heritage protocol
INPE, ISRO are in process of converting to CEOS OS
NOAA will be converting to CEOS OS – testing will start soon
For Data partners implementing CSW, CWIC will continue to provide a dual translation capability
New NRSCC CWIC partner implementing CSW
Australian data partners planning to use CSW
GEOSS support for CSW
More clients are using OpenSearch – see metrics

9. OS vs CSW Successful search on granules
Compare the total number, daily average, and unique IPs over period 2016 and 2017
2016
(1/1-12/31)
2017
(1/1-9/18)
CSW (total)
188836
148446
OS (total)
406355
316988
CSW
(daily average)
517
568 OS
1113
1214
CSW (ips)
121 72
OS (ips)
176
230
Select count(*) as CNT from METRICS where uop = 'GetRecords' AND date >= ' ' AND date <= ' ' AND sprotcol = 'CSW‘
Select count(*) as CNT from METRICS where uop = 'GetRecords' AND date >= ' ' AND date <= ' ' AND sprotcol = 'OS‘
Select count(*) as CNT from METRICS where uop = 'GetRecords' AND date >= ' ' AND date <= ' ' AND sprotcol = 'CSW‘
Select count(*) as CNT from METRICS where uop = 'GetRecords' AND date >= ' ' AND date <= ' ' AND sprotcol = 'OS'

10. OS vs. CSW (last 3 months, top 30 IPs)
opsapp.echo.nasa.gov
( ) IP
COUNTRY
CNT
United States
19083
16106
8716
Belgium
4922
4642
3611
3387
3031
2582
2118
1852
1634
1393
1295
1200
1144
1104
1068
1037
1030
910
903
Germany
849
816
740
739
637
619
550 IP
COUNTRY
CNT
United States
14729
Canada
423 35 23 4 3 2
Italy 1
select IP, country, count(*) as CNT from METRICS where uop = 'GetRecords' AND date >= ' ' AND date <= ' ' AND status='OK' and sprotcol = 'CSW' GROUP BY IP, COUNTRY ORDER BY CNT DESC LIMIT 30
select IP, country, count(*) as CNT from METRICS where uop = 'GetRecords' AND date >= ' ' AND date <= ' ' AND status='OK' and sprotcol = 'OS' GROUP BY IP, COUNTRY ORDER BY CNT DESC LIMIT 30

11. Errors Main error sources Elimination of erroneous requests
No datasetId blind harvesting
Expired datasetId
Elimination of erroneous requests
Contact major error sources
Last month metrics showed that all major error sources are eliminated
2016
2017 (before 9/4)
2017/9
CSW (daily)
420
530
OS (daily) 34
504
CSW (Invalid datasetID)
36%
95%
OS (Invalid datasetID %)
53%
45%
select count(*) as CNT from METRICS where uop = 'GetRecords' AND date >= ' ' AND date <= ' ' AND status !='OK' and sprotcol = 'OS‘
select count(*) as CNT from METRICS where uop = 'GetRecords' AND date >= ' ' AND date <= ' ' AND status !='OK' and sprotcol = 'CSW‘
select count(*) as CNT from METRICS where uop = 'GetRecords' AND date >= ' ' AND date <= ' ' AND status !='OK' and sprotcol = 'OS‘
select count(*) as CNT from METRICS where uop = 'GetRecords' AND date >= ' ' AND date <= ' ' AND status !='OK' and sprotcol = 'CSW'
select status, count(*) as CNT from METRICS where uop = 'GetRecords' AND status != 'OK' AND date >= ' ' AND date <= ' ' AND sprotcol = 'CSW' and datasetid = '-1' group by status

12. CWIC Futures Continue to support the WGISS System Level Team
Testing end to end data search and access for CWIC tagged datasets
System evolution support
Provide on-boarding support for new data providers
Outreach to client partners
What is or is not working? What else is needed?
Search specificity and returned results info sufficient?
System status/error handling sufficient?
Provide tech support for new client partners
Continue to provide support for CWIC Connectors
Continue support for GEOSS