GDWG Agenda Item: Tools for GRWG activities

Slides:



Advertisements
Similar presentations
GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group 1 Report from the GSICS Data Management Working Group Peter Miu / Masaya Takahashi.
Advertisements

Report from JMA 17th GSICS Executive Panel, Biot, 2-3 June 2016 Masaya Takahashi on behalf of Kazumi Kamide Meteorological Satellite Center, Japan Meteorological.
GSICS 2015 GDWG Breakout Session 1 Repository For GSICS Work Code Peter Miu EUMETSAT CMA, CNES, EUMETSAT, ISRO, IMD, JMA, KMA, NASA, NIST, NOAA, ROSHYDROMET,
2015 GSCIS annual meeting, March, 2015, New Delhi, India 1 GDWG Agenda Item New GSICS Products Masaya Takahashi, JMA CMA, CNES, EUMETSAT, ISRO, IMD,
2015 GSCIS annual meeting, March, 2015, New Delhi, India 1 GRWG Requirements Summary on Data, Products, and Tools - for GDWG Understanding Masaya.
JMA GDWG Report 1 Masaya Takahashi Meteorological Satellite Center / Japan Meteorological Agency (visiting scientist at EUMETSAT from Mar to Feb.
GSICS 2016 – Plenary Session, GDWG Report 1 GDWG Report 2015 / 2016 Peter Miu / Masaya Takahashi CMA, CNES, EUMETSAT, ISRO, IMD, JAXA, JMA, KMA, NASA,
2013 GSICS Joint Meeting, Williamsburg VA, USA, March GSICS Collaboration Servers Status: 2013 Peter Miu (EUMETSAT) CMA, CNES, EUMETSAT, ISRO,
GSICS 2015 GDWG Breakout Session 1 GSICS Collaboration Servers Peter Miu EUMETSAT CMA, CNES, EUMETSAT, ISRO, IMD, JMA, KMA, NASA, NIST, NOAA, ROSHYDROMET,
2015 GSCIS annual meeting, March, 2015, New Delhi, India 1 GSICS Products' Content Access Study Proposal Zhe(Thomas) Xu, CMA CMA, CNES, EUMETSAT,
GSICS Baseline Website Review
CF Conventions Support
Updating Action Tracking Way
GSICS Baseline Review: Tools – e.g. GSICS Bias Plotting Tool
DCC Product Requirements for GSICS Plotting Tool
GSICS Data Management Working Group Report
GSICS Collaboration Servers Statuses and Updates
Summary of GDWG breakout session
Jerome Lafeuille WMO Space Programme Office
GSICS Collaboration Servers a Vehicle for International Collaboration Status 2011 Peter Miu EUMETSAT.
GDWG Agenda Item Existing netCDF Format Updates
Reviewing the GSICS Data Management Working Group Terms of Reference
GSICS Baseline Reviews - website, products metadata and structures
DOI Overview to Support its Use in GSICS
GDWG Agenda Item THREDDS Directory Structure Updates
Spectral Response Function netCDF towards “GSICS Standard netCDF”
GDWG Agenda Item Repository for GSICS Work - Clarification on Archiving Masaya Takahashi, JMA CMA, CNES, EUMETSAT, ISRO, IMD, JAXA, JMA, KMA, NASA, NIST,
Requirements for GSICS Plotting Tool to support VIS/NIR products
GSICS Data Management and Availability to Users
Improvement Suggestions to the Procedure for Product Acceptance
5g. Repository For GSICS Work Use of github for codes etc
GDWG Agenda Item Baseline Reviews: Website
Manik Bali, Larry Flynn NOAA/STAR
Masaya Takahashi (JMA), Peter Miu (EUMETSAT) and Ashi Mitra (IMD)
GSICS Data Management and Availability to Users
Masaya Takahashi (JMA) and Sebastien Wagner (EUMETSAT)
Report from the GSICS Data Management Working Group
JMA Agency Report 2017 Arata Okuyama, Masaya Takahashi and Hidehiko Murata Meteorological Satellite Center, Japan Meteorological Agency.
YourOrganisation Agency Report Year
Masaya Takahashi (JMA)
Updating GSICS Plotting Tool to support VISNIR products
GIRO APPLICATION Current status and NEXT STEPS
GSICS Product Plotting Web Tool Status: 2013
GDWG Joint Meeting Summary and Actions 2017
ToR & Future Chairing of the GDWG
JMA Agency Report 2018 Masaya Takahashi, Yusuke Yogo and Hidehiko Murata Meteorological Satellite Center, Japan Meteorological Agency.
Summary of GDWG breakout session and Actions 2018
DOI Overview and its Usage for EUMETSAT GSICS Objects
GDWG Meeting Summary 2016 and Work Plan 2016 / 2017
Brief Report of GSICS-EP-18 (June 2017 in Jeju, Korea)
GDWG Fact Sheet - Information Presentation
Traceability of the GIRO to the ROLO: Definition of a benchmark
GDWG – Future Collaboration
Masaya Takahashi (JMA)
Chairing of the GDWG Peter Miu
Summary of 2019 GDWG breakout session
GSICS Baseline Review: Product meta-data and structures
GSICS Collaboration Servers 2019
Masaya Takahashi Japan Meteorological Agency
GSICS Data Management Working Group Summary and Action Items
YourOrganisation Agency Report Year
GDWG Terms of Reference
JMA Agency Report 2019 Masaya Takahashi, Yusuke Yogo and Kazuki Kodera
6n. Use of gitHub for GSICS developments
GRWG Chairing Dohyeong Kim GRWG Chair.
Masaya Takahashi Japan Meteorological Agency
Automated Validation Tool for GSICS Candidate Products (GPPA automation) Peter Miu CMA, CNES, ESA, EUMETSAT, ISRO, IMD, JAXA, JMA, KMA, NASA, NIST, NOAA,
GSICS Existing Tools Updates
Lunar Calibration Workshop Status of the preparation
GSICS Documentation Discussion - classification and storage
Presentation transcript:

GDWG Agenda Item: Tools for GRWG activities Masaya Takahashi, JMA CMA, CNES, EUMETSAT, ISRO, IMD, JAXA, JMA, KMA, NASA, NIST, NOAA, ROSHYDROMET, USGS, WMO

Overview and Purpose Overview of tools developed in the GSICS GRWG GIRO (GSICS Implementation of the ROLO model) by EUMETSAT (C) GIRO input/output handling utilities by EUMETSAT (IDL) GDWG GSICS plotting tool by EUMETSAT (Java and GWT) SRF netCDF converter by NOAA (python) netCDF metadata checker by NOAA (Java and shellscript) Framework to automatically generate GSICS products, proposed by CMA GIRO output netCDF checker (to be developed) GLCWS_14.10 The GSICS Data Working Group is asked to develop a procedure for verifying users’ implementation Purpose of this presentation To introduce the GIRO and GRWG requirements To discuss the feasibility of developing the GIRO output netCDF checker To discuss tools to be developed in future (GSICS product checker) GRWG requirements

Topic 1) GIRO - Introduction (not a bicycle race!) GIRO (GSICS Implementation of the ROLO model) by EUMETSAT (C) A tool to calculate reference values for the lunar calibration (GRWG VIS/NIR sub-group) To be shared within the lunar calibration community (GSICS, CEOS/IVOS,…) Input/Output files: follow the GSICS filenaming/netCDF convention Outcome from the original GIRO is only considered as “legal” However, someone needs to modify the source codes to implement the GIRO to his/her agency’s computer system Needs for the checker which validates the consistency of the GIRO output with the original one GLCWS_14.10 The GSICS Data Working Group is asked to develop a procedure for verifying users’ implementation

Requirements for the GIRO output netCDF checker To compare the GIRO outputs by users with reference data Reference GIRO outputs To be executed under certain conditions (32/64 bit Linux, Mac OS X, …) One netCDF file for each instrument GIRO output netCDF contains time series of the results (similar to RAC) Checking items Values of all the variables (and dimensions, attributes) Global attributes should also be checked (from the GSICS point of view) Output of the checker Printout / Output checking results Text messages are enough How to check the consistency by a third person Ask users to provide dataset to be checked (under gentleman’s agreement) Ask users to submit source codes. A third person build the executable, run that, and compare with reference data (robust but time-consuming check) In this case, automation process (e.g., web apps) would be preferable

GDWG Proposed Solution for issued on the GIRO output checker S1: GDWG to develop the checker Pros: experience could be applicable to the GSICS Correction checker Cons: limited human resources S2: GRWG to develop the checker Pros: GDWG can save human resources Cons: less reliable on the comparison result (from GDWG point of view)? Q: Is this a role of GDWG? (I think so)

Topic 2) GSICS Correction checker Existing checker To check netCDF metadata, developed by NOAA in 2013 Need a template (reference) file for each product (and algorithm type) Now, only GOES/Imager and MTSAT-2/Imager GEO-LEO-IR are supported Proposal If we enhance the functionality to check not only metadata but also variables, this could be a “GSICS Correction Checker” Would be useful for GRWG developers (no official requirements at present) Could be useful for GDWG members (in particular a chair) when we check the GSICS Correction through the GPPA Are you interested in this checker? FYI: CF-convention checker http://puma.nerc.ac.uk/cgi-bin/cf-checker.pl

GDWG discussion on the details on the proposed solution (to be updated in the discussion) The GDWG accepted/rejected the GRWG proposal because… Resources Needed: Man Days Effort: Prepare a documentation and develop the GIRO output netCDF checker - 2 weeks. Enhance the functionality of the GISCS metadata checker – 4 weeks. HW & SW Procurements: not applicable Input Data Acquisition Effort: not applicable Proposed Time Scale: GIRO output netCDF checker is required as soon as possible once GIRO source codes become available (planned in 2015), so this change should be done by the end of 2015 GSICS Correction checker is useful, but no need in a hurry, so this change is expected to be done by the 2016 GSICS annual meeting Deliverables: New / enhanced GSCIS netCDF checker(s) Priority of the Task: GIRO output netCDF chcker: High – by the end of 2015 GSICS Correction chcker: Low – by the next GSCIS annual meeting

Agenda item summary; assign action identifier, make EP recommendations and propose a lead for the action (to be updated in the discussion) Action Identifier: GDWG_15.XX: XXXX to … Substantial effort if required by the GDWG, GDWG chair should inform this activity to GSICS Executive Panel, and ask for feedback regarding: Should GDWG resources be used to update the GSICS bias plotting tool for GSICS activities. The GDWG estimates 6 weeks of resources is needed to support this action. Identify the Working Group Member Taking the Lead on this Action: XXXX The EP is invited to comment on this if the action is to be performed.

End of Presentation: Thank you for your attention WMO GSICS Portal http://gsics.wmo.int GSICS Coordination Centre - http://www.star.nesdis.noaa.gov/smcd/GCC/index.php GSICS Product Catalog - http://www.star.nesdis.noaa.gov/smcd/GCC/ProductCatalog.php EUMETSAT’s Data and Management Server http://gsics.eumetsat.int