Summary of GDWG breakout session Peter Miu / Masaya Takahashi CMA, CNES, EUMETSAT, ISRO, IMD, JMA, KMA, NASA, NIST, NOAA, ROSHYDROMET, USGS, WMO

# of GDWG members attended: 7-12 (new world record) # of Agenda Items Discussed: 18 # of Actions Generated: 36 Wednesday: new record of GDWG breakout session time (19:00)! Masaya Takahashi JMA GDWG Baseline Reviews - Website Peter Miu EUMETSAT GDWG Baseline Reviews - Standards Existing NetCDF format Updates New GSICS products Data versioning Updates Manik Bali NOAA Proposed MW Standards Peter Miu EUMETSAT Repository for GSICS Work - Code Existing Tools Updates Zhe Xu CMA Products' Content Access Study N.Puviarasan IMD Proposal of IMD satellite data center Rob Roebeling Event log - Landing page Event log - Future implementation Thursday Masaya Takahashi JMA Repository for GSICS Work - Clarification on Archiving Tools for GRWG activities Directory Structure Update Peter Miu EUMETSAT Mirroring of Products Manik Bali NOAA GPPA Timelines Document Management System

Status of actions in 2014 Action Ref Description Who Due Date State GDWG_14.1 GDWG to investigate how many people currently use GSICS products. GCC Closed GDWG_14.2 GDWG to investigate methods to structure the order the global attributes within the netCDF file - e.g. between attributes related to the method and the application. JMA 01 Mar 2015 New action is assigned GDWG_14.3 GDWG to investigate how to include algorithm type in filename All 01 May 2015 GDWG_14.5 GDWG shall have a THREDDS configuration web meeting in July. NOAA Closed (no need any more) GDWG_14.6 GDWG shall have a DOI/OID web meeting in May/June EUM

Status of actions in 2014 Action Ref Description Assigned to Due Date State GDWG_14.9 KMA to send products to the EUMETSAT GSICS Data and Products server until the CMA collaboration server is available. KMA 31 Aug 2015 Open GDWG_14.10 All GPRCs to provide satellite instrument specific links to calibration events to WMO-OSCAR. All Closed by limitation GDWG_14.11 All GPRC to seek consensus on common naming conventions for calibration events at the level of Main-Categories/Sub-Categories/Event Types. 01 May 2016 GDWG_14.12 EUMETSAT/NOAA/NASA/JMA to define a calibration events database design, and if needed, discuss this in a future webex meeting. 01 May 2018 GDWG_14.14 All GPRCs to review their GSICS websites to take into account of the new developments in GSICS as these websites will be reviewed in the next joint meeting. 01 May 2015 Closed New action is assigned GDWG_14.15 IMD to update their website to provide GSICS information. This will be provided on the WMO for inclusion into their website. IMD 01 Mar 2016? GDWG_14.16 EUMETSAT to support the creation of a GitHub? account for uploading the GSICS related codes for collaboration development. EUMETSAT 1 May 2015 New action

Status of actions in 2014 Action Ref Description Assigned to Due Date State GLCWS_14.7 GSICS Data Working Group to identify a suitable mechanism for sharing the lunar calibration dataset. GDWG (M.Takahshi) 01 Dec 2015 Closed New action is assigned GLCWS_14.9 The GSICS Data Working Group is asked to develop a procedure for verifying users’ implementation of GIRO. (M.Takahashi) 24 Mar 2014

GDWG discussion on the calibration coefficients – comparison of the current/proposed variables Current GRWG proposal float offset(date, chan) ; offset:valid_max = 200.f ; offset:valid_min = -200.f ; offset:_FillValue = "-999999" ; offset:long_name = "Regression Offset" ; offset:units = "mW m-2 sr-1(cm-1)-1" ; float offset_se(date, chan) ; offset_se:valid_max = 200.f ; offset_se:valid_min = -200.f ; offset_se:_FillValue = "-999999" ; offset_se:long_name = "Standard Error of Regression Offset" ; offset_se:units = "mW m-2 sr-1(cm-1)-1" ; float slope(date, chan) ; slope:valid_max = 2.f ; slope:valid_min = -2.f ; slope:_FillValue = "-999999" ; slope:long_name = "Regression Slope" ; slope:units = "1" ; float slope_se(date, chan) ; slope_se:valid_max = 2.f ; slope_se:valid_min = -2.f ; slope_se:_FillValue = "-999999" ; slope_se:long_name = "Standard Error of Regression Slope" ; slope_se:units = "1" ; float covariance(date, chan) ; covariance:valid_max = 200.f ; covariance:valid_min = -200.f ; covariance:_FillValue = "-999999" ; covariance:long_name = "Regression Coefficients Covariance" ; covariance:units = "mW m-2 sr-1(cm-1)-1" ; float coeff(ncoeff, date, chan) ; offset:valid_max = 200.f ; offset:valid_min = -200.f ; offset:_FillValue = "-999999" ; offset:long_name = "Regression coefficients" ; offset:units = "mW m-2 sr-1(cm-1)-1" ; float covar(ncoeff, ncoeff, date, chan) ; covariance:valid_max = 200.f ; covariance:valid_min = -200.f ; covariance:_FillValue = "-999999" ; covariance:long_name = "Regression coefficients covariance matrix" ; covariance:units = "mW m-2 sr-1(cm-1)-1" ; GRWG proposal is a general form of the regression

How to specify “valid hour range” ? “2”, this means start/end time for each result double validity_period(date, validity) ; validity_period:long_name = "correction validity period" ; validity_period:standard_name = "time" ; validity_period:calendar = "gregorian" ; validity_period:units = "seconds since 1970-01-01T00:00:00Z" ; comment = “valid hour range: 00:00 to 12:00 UTC”; Add new attributes to specify “valid hour”, if needed (e.g., INSAT-3D/Imager) Example values validity_period[783][2] [0], 1.2926304E9, 1.295136E9 [1], 1.2927168E9, 1.2952224E9 [2], 1.2928032E9, 1.2953088E9 ...

How to implement multiple methods/references results in one file S1: Use netCDF-4 “grouping” functionality (enhanced format) S2: Use different global attributes / dimensions / variables for each method Combined result (DCC+Moon+…) Global attributes, Dimensions, Variables Method1 (Deep Convective Cloud) Method2 (Moon) SEVIRI vs. Aqua/MODIS VISNIR product Method3 (Desert) schematic image of usage of “grouping” Common variables can be merged float coeff(nmethod, ncoeff, date, chan) ; offset:_FillValue = "-999999" ; offset:long_name = “Inter-calibration coefficients" ; offset:units = “W m-2 sr-1 um-1" ; double date(nmethod, date) ; standard_name: time; calendar: gregorian; units: seconds since 1970-01-01T00:00:00Z; To put method-specific variables, additional information (e.g., prefix) is necessary for variable name float dcc_mon_mode_dc(date, chan) ; float dcc_ref_mode_dc(date, chan) ; float moon_mon_irr_obs(date, chan) ; float moon_mon_irr_rolo(date, chan) ; … Prefix to distinguish the inter-calibration method GDWG’s recommendation is S2, but if option 1 is preferred then the following action is needed: Action: identify if using the enhanced data model has impact to the usage of the new products.

Filenaming for GSICS Prime Correction 1) Existing GEO-LEO-IR products One reference in one file (e.g., MSG3/SEVIRI-MetOpA/IASI, MSG3/SEVIRI-MetOpB/IASI) Need to specify the “Reference Instrument Name” in a filename e.g., W_XX-EUMETSAT-Darmstadt,SATCAL+RAC+GEOLEOIR,MSG2+SEVIRI-MetOpA+IASI_C_EUMG_20110101000000_preop_01.nc 2) GSICS Prime Correction (at that time, this is for GEO-LEO-IR) Multiple reference (blended results) in one file No need to specify the “Reference Instrument Name” in a filename e.g., W_XX-EUMETSAT-Darmstadt,SATCAL+RAC+GEOLEOIR,MSG2+SEVIRI_C_EUMG_20110101000000_preop_01.nc Regarding the proposed Prime Correction WMO filename, the following format is recommended:   W_XX-EUMETSAT-Darmstadt,SATCAL+RAC+GEOLEOIR.MSG3+SEVIRI-PRIME_C_EUMG_...

Requirements for updating the GSCIS data versioning For the the version controlling of GSICS deliverables, we already have data versioning rules on: https://gsics.nesdis.noaa.gov/wiki/Development/NetcdfConvention#Data_Versioning Based on the discussion in 2010: https://groups.google.com/forum/#!topic/gsics-dev/vU3468oHH4w Data version identifier: "vX.Y.Z” (X, Y, and Z are integer numbers) X: major version number, Y: minor version number, Z: revision number GRWG Requirements Bug fixes (w/ and w/o big impact on the outcome) should be assigned to either one of the three versions. Procedure to update versions should be clarified. Our definition is the same as the right figure (from Wikipedia)? Usage (definition) of major version 0 should be clarified. Is the operational version be assigned as v1.0.0 – YES How should bug fixes be identified – example => v0.0.1, v0.0.2, v0.0.3, ... How should minor updates be identified – example => v0.1.0, v0.2.0, ... How should major updates be identified – example => v1.2.3 => v2.0.0   Action: GCC to support the revising of the GSICS versioning and update the Wiki. http://en.wikipedia.org/wiki/Software_versioning

SRF netCDF towards “GSCIS standard netCDF” Each agency’s official instruments’ SRF Available on the GCC website, but does not cover all the instruments IASI, AIRS, MODIS, AVHRR, HIRS, AMS-A/B, MHS, VIIRS, FY-2/SVISSR, JMA GEO imagers, GOES/Imager,Sounder, MFG/MVIRI, MSG/SEVIRI A variety of SRF formats (text, HDF, csv, Excel, …) netCDF SRF files converted from the official SRFs Initially prepared for the development of DCC inter-calibration Lunar calibration (GIRO): first user Lots of GEO/LEO instruments (incl. non-GSICS members’ ones) such as MFG/MVIRI, MSG/SEVIRI, GOES/Imager, JMA GEO imagers, FY-2/SVISSR, FY-3/VISSR, COMS/MI, [Aqua,Terra]/MODIS, S-NPP/VIIRS, Terra/ASTER, Landsat/OLI, … We would like to propose considering these data as “GSICS Standard SRF” GRWG/GDWG/GCC cross-cutting issue Easy-to-establish the common calibration tool to be shared within developers Easy-to-use for users (e.g., users who generate FCDR/TCDR)

Repository for source codes: GDWG discussion on the repository for codes/tools/datasets for the ATBDs Repository for source codes: S1: GPRC to provide a code to members for development. S2: GCC to be the central repository for GSICS developed Code. S3: Investigate a “free” tool for code configuration Repository for tools/datasets: Current GSICS THREDDS servers are NOT considered as the data archiving system S1: GDWG to define GSICS THREDDS servers as the archiving system S2: GDWG to recommend to use the GSICS wiki S3: GDWG to find online storage services such as Dropbox