1. GSICS Data Management Working Group
15th Session of Executive Panel
Guangzhou, China
Manik Bali(Chair), Aleksandar Jelenak,
Rob Robelling, Peter Miu, Masaya Takahashi, Byung-II
NOAA Satellite and Information Service
16 May 2014

2. Outline Progress Past 1 Yr Future Introduction GDWG Membership News.
GDWG –Progress in Action Items .
GSICS Communications
GSICS user messaging service.
GSICS Dev server.
Review of activities since the last meeting.
NOAA. JMA, EUMETSAT
CMA, KMA
Roadmap-2014
Summary/Conclusions.
Progress
Past 1 Yr
Future
15th Session of Executive Panel
Guangzhou, China
16 May 2014

3. Introduction Masaya Takahashi Younghooi Hwang Sergey Uspensky
GDWG Membership News
Min Min
Zhigou Rong
Yuan Li
Peter Miu
Simon Elliott
R. K Giri
Masaya Takahashi
Younghooi Hwang
Sergey Uspensky
Kazuo Umezawa
Jerome Lafeuille
Masaya Takahashi has indicated that he would become the chair when he gets back to JMA.
Peter Miu has indicated that he would be the vice chair.
15th Session of Executive Panel
Guangzhou, China
16 May 2014

4. Review of activities since the last meeting
Action Items Progress
26 action items from the last Annual Meeting.
24 completed.
2 overdue (some reflect change of circumstances; some will likely be reformulated at this meeting)
15th Session of Executive Panel
Guangzhou, China
16 May 2014

5. Communication 259 Subscribers.
GSICS User Messaging Service
259 Subscribers.
Since last annual meeting 41 new subscribers added.
Registration in GUMS is steadily increasing
GDWG maintains the GUMS
15th Session of Executive Panel
Guangzhou, China
16 May 2014

6. Communication
GSICS Dev Server
GDWG maintains the Dev Server

7. Review of activities since the last EP meeting
IMD GDWG progress
GEO and LEO Data receiving stations have been setup.
Real time visualization and quality monitoring for NPP, METOP-A/B , MODI NOAA available
Infrastructure for cross calibration being setup.
15th Session of Executive Panel
Guangzhou, China
16 May 2014

8. Review of activities since the last meeting
NOAA GDWG progress
Support to GCC for the transition of MW FCDR product demo and then to pre-operational phase.
SRF of GEO instruments in NetCDF format [support to DCC]
Developed Product Taxonomy
This is supposed to be the basis for redoing the THREDDS configuration to group GSICS correction products differently. Also the taxonomy would come handy in planning how to utilize the file naming convention and any other data management task.
Developed a data model for satellite instrument event logs and translated it into an extension to the ISO metadata standard's XML Schema.
15th Session of Executive Panel
Guangzhou, China
16 May 2014

9. Review of activities since the last meeting
NOAA GDWG progress
Initiated process to get DOI number for all GSICS products and documents.
DOI for MW already assigned by NOAA ( outside of GSICS )
GDWG to have a web meeting in June to decide on doi Vs oid.
Other GSICS products need review to progress to op.
(need clarity from EP on archival & GPPA modification, Chair’s decision)
New website for GDWG ( content requested from members).
New GPPA ( proposed by Peter Miu, Manik Bali).
15th Session of Executive Panel
Guangzhou, China
16 May 2014

10. Review of activities since the last meeting
JMA GDWG progress
Satellite event log
Provided the information on the JMA website in Mar/June 2013
Change of the GEO-LEO IR RAC NetCDF contents
Met the requirements of EUMETSAT bias plotting tool in July 2013
Provision of official SRF data to GRWG VIS/NIR sub-group
Provided data to GDWG in Apr 2013
Collaboration with WIS
Reported feasibility to the GDWG chair in May 2013
Calibration change alerts
Reported feasibility to gsics-dev in June 2013
15th Session of Executive Panel
Guangzhou, China
16 May 2014

11. EUMETSAT GDWG progress
Review of activities since the last meeting
EUMETSAT GDWG progress
Proposed New GPPA.
Support to all GDWG activities.
Visualization tool.
15th Session of Executive Panel
Guangzhou, China
16 May 2014

12. EUMETSAT GDWG progress
On Event Logging Data Model…..
Before leaving GSICS, the NOAA GDWG member took a lead in the development of a model for a common satellite event logging system.
The model has been briefly evaluated by EUMETSAT and their feedback it that it is too complex for the needs of GSICS.
Further evaluation needs to be done.
EUMETSAT is submitting a CGMS paper proposing a way forward to simplify the approach to the resolution of this action.
Based on the response to this paper, the appropriate actions will be taken.
CGMS is tasking GSICS to support activities relevant to their actions (e.g. Event logging). If progress is expected in this or any GSICS action, the GSICS EP needs to urgently and decisively take a lead in managing this long standing issue
EUMETSAT (Miu)
15th Session of Executive Panel
Guangzhou, China
16 May 2014

13. EUMETSAT on Bias Monitoring tool…. The tool is under development.
EUMETSAT GDWG progress
EUMETSAT on Bias Monitoring tool….
The tool is under development.
15th Session of Executive Panel
Guangzhou, China
16 May 2014

14. Review of activities since the last meeting
CMA progress
IASI acquirement transferred from CLASS to EUMETSAT pushing (March, 2013)
Update GSICS data collection system for FY new sensors (FY-3C) and new reference instrument (NPP-CrIS, NPP-VIIRS)
Automatically PICS data extraction for FY-3/MERSI/VIRR
Provide the instrument telemetry/Engineering data (OBC files) from FY operational processing system
CMA GPRC new website is developing and will finish shortly
15th Session of Executive Panel
Guangzhou, China
16 May 2014

15. Review of activities since the last meeting
KMA progress
Developing KMA’s GSICS website
General Introduction of COMS inter-calibration
Methods and Bias Monitoring for IR and VIS
Links to Other GSICS Website
Corrected Data filename and format
15th Session of Executive Panel
Guangzhou, China
16 May 2014

16. Internal Data Collection External Data Collection
KMA progress
KMA GSICS
NMSC Homepage
Data
Display
Four steps
1) Data Collection
COMS, IASI, AIRS, MODIS, OMI
2) Data Processing
Daily, Monthly Process and archiving
3) Data Conversion
NetCDF format conversion
4) Display
GSICS in NMSC Website
Visible and Infrared
Statistics
Module
Image Module
Data
Conversion
Storage
NetCDF
Module
Storage
Data
Process
Cloud
VISCC
Desert
VISCD
Ocean
VISCO IR
AIRS/IASI
Internal Data Collection
External Data Collection
Data
Collection
COMS Data
MODIS, OMI, AIRS, IASI
KMA, Byung
NOAA
NASA
16 May 2014

17. New Filename : GSICS standard
KMA progress
KMA GSICS
Old Filename
MT1R_ _0600_AIRS_060.nc
MT1R_ _1100_IASI_ IASI_xxx_1C_M02_
958Z_ Z_N_O_ Z__ nc
New Filename : GSICS standard
pflag_productidentifier_oflag_originator_yyyyMMddhhmmss[_freeformat].type[.compression]
AIRS : W_KR-KMA-NMSC,SATCAL+NRTC+GEOLEOIR,COMS+MI-
Aqua+AIRS_C_RKSL_ nc
IASI : W_KR-KMA-NMSC,SATCAL+NRTC+GEOLEOIR,COMS+MI_
MetopA+IASI_C_RKSL_ nc
 From April 1, 2011 to present
KMA, Byung
15th Session of Executive Panel
Guangzhou, China
16 May 2014

18. FROM RESEARCH TO PRODUCTION BASIC PATHWAYS/TOOLS HAVE BEEN MET
15th Session of Executive Panel
Guangzhou, China
16 May 2014

19. Road Map for 2014 Maintenance of current services in 24x7 mode
THREDDS Server
Product Catalog and Products archival.
Wiki Page/ Web Page
GDWG Support to GPPA to enable acceptance of Newer Data into GSICS fold.
Centralized Bias monitoring system.
15th Session of Executive Panel
Guangzhou, China
16 May 2014

20. Road Map for 2014
Satellite Instrument Event Logging specification, design and implementation.
Identify a process for archiving GSICS products- need a policy document
Implementing new capabilities in the GSICS product plotting tool.
GSICS document management plan definition and associated document templates.
Doi /Oid numbers for Documents and Products.
Address resource crunch
15th Session of Executive Panel
Guangzhou, China
16 May 2014

21. Road Map for 2014 Asses New tools like Big Data that can
Perform analytics when instruments are compared with stable references
Detect instrument degradation trends
Detect impacts of diurnal/seasonal instrument temp variations.
15th Session of Executive Panel
Guangzhou, China
16 May 2014

22. Summary
NOAA GDWG has been able to ensure running of key services like Product Catalog, GSICS Dev and GUMS
File naming and DOI number for products.
NOAA has the resources to assign DOI number .
Outside of GSICS MW product already got the doi number.
15th Session of Executive Panel
Guangzhou, China
16 May 2014

23. Summary Progress Reports NOAA, CMA, JMA , KMA.
Excellent progress demonstrated by members towards achieving goals of GSICS.
Calibration Change Alerts ( Peter Miu, Rob Robelling)
Automation, Connection , Harmonization and online depiction discussed.
Big Data (Lauri Rokke)
Performance of algorithms, analytics
GSICS Archiving Process ( Peter Miu)
Procedure for archiving, where to archive and until when.
15th Session of Executive Panel
Guangzhou, China
16 May 2014

24. Summary Event Log ( Rob).
WMO to include GPRC event logging links to the OSCAR website.
GPPA improvements ( Peter Miu, Manik Bali )
To make GPPA submission online and reduce time of review.
GEO-LEO Product promotion ( Manik Bali ).
GDWG to extend file naming and meta data standards to other channels.
GDWG to support to User workshop
15th Session of Executive Panel
Guangzhou, China
16 May 2014

25. Summary Harmonize working hours of GDWG members.
Successfully collaboration requires all GSICS GPRCs to:
Nominate a representative to the GDWG.
Each GDWG representative needs to commit time to work on GSICS activities (see details in the 2014 GDWG minutes).
CGMS is tasking GSICS to support activities relevant to their actions (e.g. Event logging). If progress is expected in this or any GSICS action, the GSICS EP needs to urgently and decisively take a lead in managing this long standing issue.

26. THANKYOU
15th Session of Executive Panel
Guangzhou, China
16 May 2014

27. GDWG status of members.
The following GPRCs are represented in the GDWG:
EUMETSAT – 2 members
JMA – 1 member
KMA – 1 member
NOAA – 1 member
Successfully collaboration requires all GSICS GPRCs to:
Nominate a representative to the GDWG.
Each GDWG representative needs to commit time to work on GSICS activities (see details in the 2014 GDWG minutes). .

28. Introduction
GSICS EP
GRWG
GDWG
GPRC
GCC
GCC Dir.
Product Quality Manager
GSICS Quarterly Editor
US/European/
Asian Correspondent
Product Baseline Manager
Web Master
GCC Deputy IR UV
VIS MW
Advise the Executive Panel in the planning and implementation of data management activities.

29. Introduction Concept Implementation Milestones since 2007 2010 2007
Archiving requirements. 
Access methods. 
Data sets. 
Meta Data  v  Documentation  v  Security Requirements.
2010
The essential requirements of the server as a data exchange platform were satisfied. The server was ready for operations.
2011
Bias Monitoring.
NetCDF file naming conventions
2008
Concept of Catalogue. 
Data format. 
Event log.
Data Management Services 
2012
GSICS User Messaging Service
NetCDF file naming conventions
THREDDS Config. finalization
2009
GPPA  (Product Acceptance )
Docs on WIKI 
NetCDF file definitions 
2013
Product Catalogue.
Web-based bias monitoring tool
2014

30. Key Discussions and Action items
Resources needed to maintain the running of servers, websites ,catalogs
AK Sharma from IMD indicated that IMD would identify a person to work on GDWG.
Accounts on Wiki would be provided to Masaya and Peter.
Masaya (VC), Peter ( Nominated himself VC).
Harmonize working hours of GDWG members.
Aleksandar’s ideas on Taxonomy for products was presented.
It was felt that currently we don’t have resources to implement it at server level. However GCC website would give an interface that would more or less follow the Taxonomy.

31. Adoption of the proposed product taxonomy.
GCC applying Taxonomy directly on the Catalog.
New organizing of the products on the THREDDS catalog based on proposed taxonomy
Not necessary and need resources.
NetCDF file content organization of new products
GDWG would look into it and asses its feasibility.
Formats and procedure for storing and exchanging satellite instrument event logs.
Revisit GSICS Product Acceptance.
Is Big Data an answer to our questions.

32. 5-8 March 2012 GSICS Joint Working Groups Meeting, Beijing, China