Proposal for input/output data of GIRO application Masaya Takahashi (JMA), Bartolomeo Viticchie (EUMETSAT) GRWG/GDWG web meeting, 24 June, 2014

Input/Output files of GIRO application We propose both input and output files to be in NetCDF Input: lunar observation and instrument SRF files, Output: lunar calibration file Common format data: easy-to-handle in the executable Detailed document and NetCDF template will be found on the GSICS Wiki https://gsics.nesdis.noaa.gov/wiki/Development/20140624 The contents are based on: GSICS NetCDF convention https://gsics.nesdis.noaa.gov/wiki/Development/NetcdfConvention  Relies on the following netCDF conventions: GSICS file naming convention https://gsics.nesdis.noaa.gov/wiki/Development/FilenameConvention  Follows the rules given in the General File Naming Conventions section of the W.M.O. Manual on The Global Telecommunication System GRWG/GDWG web meeting, 24 June, 2014

General information about NetCDF http://www.unidata.ucar.edu/software/netcdf/docs/ easy-to-understand User’s guide and tutorials Utilities: many CUI/GUI apps, ncdump/ncgen are quite useful http://www.unidata.ucar.edu/software/netcdf/docs/html_guide/netcdf_utilities_guide.html ncdump: convert NetCDF file to text form (CDL) ncgen: generates a NetCDF file or a C/FORTRAN code that creates a netCDF dataset from CDL “NetCDF is a set of software libraries and self-describing, machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. NetCDF was developed and is maintained at Unidata. ...” (from Unidata website above) GRWG/GDWG web meeting, 24 June, 2014

Input-1: Lunar observation NetCDF File name (e.g., MTSAT-2/Imager) W_JP-JMA-MSC,VISNIR+SUBSET+MOON,MTSAT2+Imager_C_RJTD_20130701062451_01.nc satellite and instrument name lunar observation time as yyyyMMddhhmmss To know the naming details, please refer to: https://gsics.nesdis.noaa.gov/wiki/Development/FilenameConvention Each participant has to generate this file Single file for each lunar occurrence, including all-channel information Variable units: must be the same as those in a template Recommend to use classic netCDF4 Could be used as “GSICS source dataset” in the future TO STRESS: the file name follows an official (i.e. GSICS) convention. Participants should follow the filename template as indicated. GRWG/GDWG web meeting, 24 June, 2014

global attributes: Lunar observation NetCDF Please follow our proposal if you have a plan to share the lunar observation NetCDF within GSICS community in the future NAME CONTENT Conventions CF-1.6 Metadata_Conv entions Unidata Dataset Discovery v1.0 standard_name_ vocabulary CF Standard Name Table (Version 19, 22 March 2012) project e.g. Global Space-based Inter-Calibration System http://gsics.wmo.int title e.g. MSG-2+SEVIRI lunar imagtette file summary e.g. Lunar observation file keywords “free content” references TBD institution e.g. EUMETSAT licence “terms of use” creator_name e.g., EUMETSAT - European Organisation for the Exploitation of Meteorological Satellites NAME CONTENT creator_email e.g. ops@eumetsat.int creator_url e.g. http://www.eumetsat.int instrument e.g. MSG2 SEVIRI instrument_wmo_code e.g. (56, 207) data_source “free content” e.g., Level 1.0 date_created YYYY-MM-DDTHH:MM:SSZ date_modified history “free content” id File name wmo_data_category 101 for Imager data(satellite) wmo_international_dat a_subcategory 0 for Multi-purpose VIS/IR imagery processing_level e.g. “v1.0.0” doc_url URL of document doc_doi DOI of document (if exists) GRWG/GDWG web meeting, 24 June, 2014

dimensions/variables: Lunar observation NetCDF NAME VALUE DESCRIPTION chan e.g. 4 for SEVIRI Number of channels chan_strlen e.g. 6 for SEVIRI Maximum length of variable: chan date 1 Number of date sat_ref_strlen 6 Maximum length of variable: sat_pos_ref (i.e. “ITRF93”) sat_xyz 3 X, Y, and Z axis of satellite position col, row e.g. 499 for SEVIRI Number of columns and rows of the imagette Compulsory Imagette (optional) Additional (optional) variables NAME [dimension] LONG NAME UNITS TYPE date[date] time of lunar observation seconds since 1970- 01-01T00:00:00Z  double precision floating point channel_name[chan,chan_strlen] channel identifier character ,e.g. “VIS006” sat_pos[sat_xyz] satellite position km sat_pos_ref[sat_ref_strlen] satellite position x y z character, either “J2000” or “ITRF93” irr_obs[chan] observed moon irradiance W m-2 µm-1 rad_obs_imgt[row,col,chan] observed moon radiance imagette W sr-1 m-2 µm-1 dc_obs_imgt[row,col,chan] observed moon digital counts imagette 1 32-bit integer array moon_pix_thld[chan] digital counts threshold for moon mask 32-bit integer pix_solid_ang[chan] pixel solid angle sr ovrsamp_fa[chan] oversampling factor dc_obs[chan] integrated digital counts of moon observation dc_obs_offset[chan] averaged digital counts offset of deep space moon_pix_num[chan] number of moon pixels opr_slope[chan] official calibration slope W sr-1 m-2 µm-1 DC-1 opr_offset [chan] official calibration offset 1 (i.e. DC) TO STRESS: in the case when both the observables and the imagettes are provided, the imagette is used ONLY for check purposes. The observables as provided by the user are the values used for deriving the end results. In case of discrepancies, a output message will notify it but there will be no data substitution. TO STRESS: in the case imagettes are provided, both counts and radiances must be given. It is necessary to avoid complexity in the code to apply the calibration formulaes. GRWG/GDWG web meeting, 24 June, 2014 must be the same as that in the SRF netCDF origin: lower-left

Example : Lunar observation NetCDF MTSAT-2/Imager (1 visible channel) Case 1) : Compulsory Keep all variables in the NetCDF and fill invalid variables with a value of -999 (i.e. _FillValue) Dimensions of col and row (# of lunar imagette pixels) -> 1 Values of rad_obs and dc_obs -> -999 File size: ~5KB Case 2) : Compulsory + Imagette + Additional Dimensions of col and row -> 700 File size: ~6MB GRWG/GDWG web meeting, 24 June, 2014

Input-2: instrument SRF NetCDF File name (e.g., Meteosat-10/SEVIRI): W_XX-EUMETSAT-Darmstadt,VIS+IR+SRF,MSG3+SEVIRI_C_EUMG.nc Each participant DOES NOT have to prepare this file Generated by GDWG in 2013 using official SRFs provided by each GPRC http://gsics.nesdis.noaa.gov/thredds/instrument-srf.html GOES-11/-12/-13/-15, Meteosat-7/-9, MTSAT-1R/-2, COMS “This file was produced in support of GSICS activities and thus is not meant for public use although the data in the file is in the public domain.” (from docs on the NOAA GSICS Data server) After some modification, files will be put on GSICS Wiki (or GSICS Data server) How to generate new SRF NetCDF files The GDWG (i.e. Masaya) will do it when official SRF file is provided If it is impossible to share the file for some reasons, please let us know GRWG/GDWG web meeting, 24 June, 2014

Output: Lunar calibration NetCDF File name (e.g., Meteosat-7/MVIRI): W_XX-EUMETSAT-Darmstadt,SATCAL+ROLOVISNIR,MET7+MVIRI_C_EUMG_201307010634_01.nc Single file contains all-time, all-channel information (following the GSICS approach adopted to generate inter-calibration products, like GEO-LEO-IR Re-Analysis Correction NetCDF) Contents Observation time, positions of satellite, Moon, Sun, observables, reference lunar irradiance, ... Starting point for developing a GSICS product in the future Concept: applicable to the future GSICS GEO-LEO VIS/NIR products GRWG/GDWG web meeting, 24 June, 2014

global attributes: Lunar calibration NetCDF Some differences from the lunar observation NetCDF NAME CONTENT Conventions CF-1.6 Metadata_Conve ntions Unidata Dataset Discovery v1.0 standard_name_v ocabulary CF Standard Name Table (Version 19, 22 March 2012) project e.g., Global Space-based Inter-Calibration System http://gsics.wmo.int title e.g. GSICS MSG-2+SEVIRI vs ROLO1.0 lunar calibration file summary Lunar calibration file keywords “free content” references TBD institution “institution name” (e.g. EUMETSAT) licence “terms of use” creator_name e.g., EUMETSAT - European Organisation for the Exploitation of Meteorological Satellites creator_email ops@eumetsat.int creator_url http://www.eumetsat.int NAME CONTENT instrument e.g., MSG2 SEVIRI instrument_wmo_code (56, 207) date_created YYYY-MM-DDTHH:MM:SSZ date_modified history “free content” id file name wmo_data_category 30 for Calibration dataset (satellite) wmo_international_dat a_subcategory 5 for Re-Analysis Correction processing_level e.g. “v1.0.0” atbd_url ATBD URL atbd_doi ATBD DOI time_coverage_start time_coverage_end phase_angle_min 2 [degree] phase_angle_max 92 [degree] reference_model e.g. ROLO version 1.0 GRWG/GDWG web meeting, 24 June, 2014

dimensions: Lunar calibration NetCDF NAME VALUE DESCRIPTION chan e.g. 4 for SEVIRI Number of channels chan_strlen e.g. 6 for SEVIRI Maximum length of variable: chan date 1 Number of date sat_ref_strlen 6 Maximum length of variable: sat_pos_ref (i.e. “ITRF93”) sat_xyz 3 X, Y, and Z axis of satellite position lambda_rolo 32 Number of ROLO spectrum lambda_apollo 221 Number of APOLLO spectrum obs_strlen Length of lunar observation NetCDF file name Differences from lunar observation NetCDF GRWG/GDWG web meeting, 24 June, 2014

variables: Lunar calibration NetCDF 3 options: Basic, Partial: Basic+, Complete: Partial+ NAME LONG NAME UNITS TYPE date[date] time of lunar observation seconds since 1970-01-01T00:00:00Z double precision floating point channel_name[chan,chan_strlen] channel identifier character, e.g. “VIS006” central_wavelength[chan] nominal channel central wavelength µm irr_obs[date,chan] observed moon irradiance W m-2 µm-1 irr_rolo[date,chan] rolo irradiance dc_obs[date,chan] integrated digital counts of lunar observation 1 32-bit integer dc_obs_offset[date, chan] averaged digital counts offset of deep space moon_pix_num[date,chan] number of moon pixels pix_solid_ang[date,chan] pixel solid angle sr ovrsamp_fa[date,chan] oversampling factor moon_pix_thld[date,chan] digital counts threshold for moon mask sat_selen_lat[date ] satellite selenographic latitude degrees sat_selen_lon[date] satellite selenographic longitude dist_sat_moon[date] satellite moon distance km sun_selen_lat[date] sun selenographic latitude sun_selen_lon[date] sun selenographic longitude dist_sun_moon[date] sun moon distance AU phase_ang[date] moon phase angle sat_pos_ref[sat_ref_strlen] satellite position x y z character, either “J2000” or “ITRF93” sat_pos[date, sat_xyz] satellite position sun_pos[date, sat_xyz] sun position moon_pos[date, sat_xyz] moon position GRWG/GDWG web meeting, 24 June, 2014

variables: Lunar calibration NetCDF 3 options: Basic, Partial: Basic+, Complete: Partial+ NAME LONG NAME UNITS TYPE rolo_spectr[date, chan, lambda_rolo] rolo reflectance spectrum at rolo wavelength 1 double precision floating point rolo_lambdas[lambda_rolo] rolo wavelength µm rolo_smooth_spectr[date, chan, lambda_apollo] rolo reflectance spectrum at apollo wavelength apollo_lambdas[lambda_apollo] apollo wavelength m double precision moon_obs_file[date, obs_strlen] lunar observation file name character GRWG/GDWG web meeting, 24 June, 2014

Points of contact If you have any questions about: Generating lunar observation NetCDF Generating new instrument SRF NetCDF Troubleshooting of GIRO etc. Please email us! Before asking, please check updates of documentations on the GSICS wiki (https://gsics.nesdis.noaa.gov/wiki/Development/20140624) Masaya Takahashi (EUMETSAT visiting scientist from JMA): Masaya.Takahashi -at- eumetsat.int (or masa.takahashi80 -at- gmail.com) Bartolomeo Viticchie: Bartolomeo.Viticchie -at- eumetsat.int GRWG/GDWG web meeting, 24 June, 2014

Thanks for your attention GRWG/GDWG web meeting, 24 June, 2014