GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group 1 Report from the GSICS Data Management Working Group Peter Miu / Masaya Takahashi.

Slides:



Advertisements
Similar presentations
GSICS EP Meeting 2015 – GSICS Archiving Strategy 1 GSICS Archiving Strategy Peter Miu / Boulder, USA CMA, CNES, EUMETSAT, ISRO, IMD, JAXA, JMA, KMA, NASA,
Advertisements

CGMS Secretariat, CGMS-40, 6 November 2012 v1 Coordination Group for Meteorological Satellites - CGMS Type of actionActions at CGMS-39 ClosedComments Actions.
GSICS 2015 GDWG Breakout Session 1 Repository For GSICS Work Code Peter Miu EUMETSAT CMA, CNES, EUMETSAT, ISRO, IMD, JMA, KMA, NASA, NIST, NOAA, ROSHYDROMET,
GSICS Data and Products Servers Volker Gärtner (EUMETSAT) For GSICS Data Working Group.
GSICS GDWG Report, Toulouse, 9 February GSICS Data Working Group Status Report V. Gaertner CMA, EUMETSAT, JMA, KMA, NOAA/NESDIS, WMO.
2015 GSCIS annual meeting, March, 2015, New Delhi, India 1 GDWG Agenda Item New GSICS Products Masaya Takahashi, JMA CMA, CNES, EUMETSAT, ISRO, IMD,
2015 GSCIS annual meeting, March, 2015, New Delhi, India 1 GRWG Requirements Summary on Data, Products, and Tools - for GDWG Understanding Masaya.
GSICS 2016 – Plenary Session, GDWG Report 1 GDWG Report 2015 / 2016 Peter Miu / Masaya Takahashi CMA, CNES, EUMETSAT, ISRO, IMD, JAXA, JMA, KMA, NASA,
2013 GSICS Joint Meeting, Williamsburg VA, USA, March GSICS Collaboration Servers Status: 2013 Peter Miu (EUMETSAT) CMA, CNES, EUMETSAT, ISRO,
GSICS 2015 GDWG Breakout Session 1 GSICS Collaboration Servers Peter Miu EUMETSAT CMA, CNES, EUMETSAT, ISRO, IMD, JMA, KMA, NASA, NIST, NOAA, ROSHYDROMET,
2015 GSCIS annual meeting, March, 2015, New Delhi, India 1 GSICS Products' Content Access Study Proposal Zhe(Thomas) Xu, CMA CMA, CNES, EUMETSAT,
Highlights (Actions) of GSICS Exec Panel 30 October 2009 Volker Gärtner.
GSICS Baseline Website Review
CF Conventions Support
Updating Action Tracking Way
GSICS Baseline Review: Tools – e.g. GSICS Bias Plotting Tool
GSICS Data Management Working Group Report
GSICS Collaboration Servers Statuses and Updates
GDWG Agenda Item: Tools for GRWG activities
Jerome Lafeuille WMO Space Programme Office
GSICS Collaboration Servers a Vehicle for International Collaboration Status 2011 Peter Miu EUMETSAT.
GDWG Agenda Item Existing netCDF Format Updates
Reviewing the GSICS Data Management Working Group Terms of Reference
GSICS Baseline Reviews - website, products metadata and structures
DOI Overview to Support its Use in GSICS
GDWG Agenda Item THREDDS Directory Structure Updates
Spectral Response Function netCDF towards “GSICS Standard netCDF”
GDWG Agenda Item Repository for GSICS Work - Clarification on Archiving Masaya Takahashi, JMA CMA, CNES, EUMETSAT, ISRO, IMD, JAXA, JMA, KMA, NASA, NIST,
GSICS Data Management and Availability to Users
Improvement Suggestions to the Procedure for Product Acceptance
5g. Repository For GSICS Work Use of github for codes etc
GDWG Agenda Item Baseline Reviews: Website
Manik Bali, Larry Flynn NOAA/STAR
Masaya Takahashi (JMA), Peter Miu (EUMETSAT) and Ashi Mitra (IMD)
Peter Miu / Masaya Takahashi
Rob Roebeling, Viju John, Frank Ruethrich, Tim Hewison EUMETSAT
GSICS Data Management and Availability to Users
Report from the GSICS Data Management Working Group
JMA Agency Report 2017 Arata Okuyama, Masaya Takahashi and Hidehiko Murata Meteorological Satellite Center, Japan Meteorological Agency.
YourOrganisation Agency Report Year
Masaya Takahashi (JMA)
GSICS Product Plotting Web Tool Status: 2013
GDWG Joint Meeting Summary and Actions 2017
ToR & Future Chairing of the GDWG
Summary of GDWG breakout session and Actions 2018
DOI Overview and its Usage for EUMETSAT GSICS Objects
GDWG Meeting Summary 2016 and Work Plan 2016 / 2017
Brief Report of GSICS-EP-18 (June 2017 in Jeju, Korea)
GDWG Fact Sheet - Information Presentation
GDWG – Future Collaboration
Masaya Takahashi (JMA)
Masaya Takahashi (JMA), Ashim Mitra (IMD) and Peter Miu (EUMETSAT)
Chairing of the GDWG Peter Miu
Summary of 2019 GDWG breakout session
GSICS Baseline Review: Product meta-data and structures
EUMETSAT PathFinders Peter Miu
GSICS Collaboration Servers 2019
Bias Monitoring Web Page Design
GSICS Data Management Working Group Summary and Action Items
YourOrganisation Agency Report Year
GDWG Terms of Reference
JMA Agency Report 2019 Masaya Takahashi, Yusuke Yogo and Kazuki Kodera
6n. Use of gitHub for GSICS developments
Instrument Specifications and Calibration Information GDWG, March 2019
Updates of GDWG Recommendations for GSICS Action Tracking Tool
Masaya Takahashi Japan Meteorological Agency
Progress in CMA on GSICS Collaboration Servers Synchronisation
Automated Validation Tool for GSICS Candidate Products (GPPA automation) Peter Miu CMA, CNES, ESA, EUMETSAT, ISRO, IMD, JAXA, JMA, KMA, NASA, NIST, NOAA,
GSICS Existing Tools Updates
GSICS Documentation Discussion - classification and storage
Presentation transcript:

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group 1 Report from the GSICS Data Management Working Group Peter Miu / Masaya Takahashi Boulder, USA CMA, CNES, EUMETSAT, ISRO, IMD, JAXA, JMA, KMA, NASA, NIST, NOAA, ROSHYDROMET, USGS, WMO

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Overview  GDWG Joint Meeting Highlights  GDWG Data Management Systems Status  A New Approach for Collaboration  GDWG Challenges  GDWG Work Plan 2015/2016 2

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Highlights 2015  First GSICS Meeting to take place in India.  Great Organisation & Hospitality from IMD. 3

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Data Management System Status  Collaboration Servers major updates:  Product directory restructuring.  Introduction of new GSICS products.  Mirroring of products across servers.  Bias Plotting Tool major updates:  Updates to support final changes to the GSICS products before operations.  Updates to support new collaboration servers and new products. 4

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group A New Approach to Collaboration  Cloning... 5

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Why do we need a better approach?  Improves overview of collaboration activities.  Supports the quantification of resources needed to gauge the commitments required so that clear statements can be made to GDWG members what their obligations are to support the Data Management collaboration effort.  Identify if there is the need to propose initiatives to support GDWG members to take the lead for actions.  Goal is to “Make Progress” and to “Manage Expectations” regarding GDWG activities as well as keep GSICS members and users informed. 6

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Current view of activities  The following table shows the number of attendees for all GSICS joint meetings (taken from the minutes and my memory). The numbers for the 2015 are look “good”.  This table provides interesting data but the information it contains is unclear. For example, the 8 actions from the 2008 meeting at NOAA included the development of the collaboration servers and the definition of the GSICS data management guidelines, conventions and standards.  Whereas, the 54 actions in the 2011 meeting were recording minor cosmetic updates to websites.  In addition to this, attendee number is also misleading as contribution to these meetings can be from the same members.  Thus we need a better approach to looking at this information to support the collaboration process WhereEUMNOAAJMACNESKMACMANOAAEUMIMD GDWG attendance Actions

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Improving the Overview (1)  Actions are now recorded in an online spreadsheet with new categorisations. This is available to all members to view and the chairs to update. An original version is kept by the chair(s). 8

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Improving the Overview (2)  Members are the working groups are encouraged to take a look at the action list especially if they have open actions.  The Working Group Chair(s) is expected to check the action list and kindly request the status of open actions.  If progress is made, then the chair closes the action.  If progress is not made according to the proposed completions date (this was agreed with the actionee), then working group members can be asked to provide support.  Working in this way should spread out the GSICS work over the time between GSICS joint meetings as well as maintain regular contact with other members.  Maybe a future task of the GDWG is to automate this process. 9

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Quantification of Resources  A quantification (of resources) is needed to estimate the effort for completing an Action.  Initially a simple approach is taken to provide a effort measure needed to perform an action.  The effort measure can then be used to provide an expectation and used together with subjective “urgency wish” support planning.  This effort measure maybe refined in the future based on use cases. 10

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Task Categorisation to indentify an Effort Measure TypeDescriptionEffort Measure Analysis Examining a subject and reporting if it is suitable/applicable for GSICS. 3 Configuration Updates to websites, documentation and/or data files for GSICS Data Management systems. 3 Implementation The development of a GSICS Data Management System.5 Information Provision of existing known material for use in GSICS.1 Specification The development of a GSICS Data Management Concept.4 Tech Support Provision of existing known material that requires tailoring for use in GSICS. 2 11

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Actions Analysis CMACNESEUMISROIMDJAXAJMAKMANASANISTNOAAROS H USGSWMO At Meeting Action Lead Action Support Anal(3) Config(3) Imple(5) ¼ Info(1) 000½ Spec(4) Tech(2) Total Effort

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Challenges (1)  Resource commitment to GSICS activities:  Finding sufficient reliable long term human resources to support the current and future GSICS Data Management activities. At present there is no clear reference addressing this need.  Tech Skills are Essential:  Data Management working group members must be able to support computing related activities. This is essential otherwise resources are wasted in providing “education” on computing fundamentals.  Technology Information Exchange is Needed:  The Computing technologies currently used in GSICS Data Management deliverables is complex. Support in information flow should be encouraged through exchange visits to cooperatively work on GSICS tasks. Some exchange visits have already taken place and these have proved to be fruitful towards the sharing of information to support the GSICS collaboration effort. 13

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Challenges (2): An Example  GDWG_2015.4i: EUMETSAT to provide estimate of the resources needed and the expectations for when the updates will be implemented, and present these to the GDWG and EP to request for resources.  Focusing on EUMETSAT, we have taken the lead in 7 actions, and are supporting further 5 actions with the highest effort measure of 33. The effort needed to support these actions will take more than the proposed 30 / 40 days for a GDWG member and the chair.  This does not include the co-chairing activities; coordination, organisation, participation in the GPAT, etc.  This does not include updates to guidelines, standards and convention, and product template updates. 14

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Challenges (3): An Example  In addition to this, the following are estimates in man x days for the development/update of GSICS tools: 15 ToolsMan DaysExp. CompletionPriority Collaboration Servers – Major Updates (EUM/JMA)2 weeksBefore 1 st OP productHigh Plotting Tool – Updates to support changes (EUM)1 weekBefore 1 st OP productHigh CF / netCDF checker (JMA / KMA / CMA / ISRO)2 weeksASAPMedium NetCDF Product Generation framework (EUM)1 monthEnd 2015: nice to have and required by IMD to support their GSICS product generation Medium Sharing GIRO Software and Reference SRFs (EUM)2 weeksASAPHigh GSICS Spectral Response Function Convertor (JMA)1 week Medium

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Challenges (4): An Example  Some ideas to address these Challenges:  Currently we have a ToR for the GDWG chair, we also need a ToR for GDWG members.  Both ToRs need to state the minimum tech qualification required for the roles and the minimum number of days committed in working on GDWG actions.  This minimum number of days commitment is in line with other WMO working groups. The following commitment statements are proposed. Chair / Vice Chairs / Co-chairs: a minimum of 40 days per GSICS year (~20% of workdays). Members: a minimum of 30 days per GSICS year (~15% of workdays).  In addition to being a GDWG members, responsibility for GDWG specific tools need to be administer NOT by a single GPRC. Members are encouraged to take responsibility through technology information exchange.  To facilitate this exchange, GPRCs are encouraged to budget for cooperation visits.  The EP members are invited to discuss and propose a way forward to address these challenges. 16

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Work plan 2015/2016 (1)  Notable New Action Items (see action spreadsheet for all actions):  GSICS product generation framework; validation and usage of a framework to support the development of GSICS products.  GSICS products’ templates; all GSICS products of a particular type shall be formatted according to these templates.  Product Format Validation Tool; automated syntactic validation of GSICS products using the products’ templates.  Code Storage Proposal; what mechanism to use to store code for Data Management Systems to facilitate collaboration. 17

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group Work plan 2015/2016 (2)  Notable New Action Items (see action spreadsheet for all actions):  Event logging support; landing pages definition and input to CGMS team.  Provision of User access to GIRO software and GSICS lunar observation data components.  Specification of a subscription service for GSICS Products; development of a tool to automatically download GSICS products. Tool is also used for mirroring on GSICS servers.  Specification of the Archiving Strategy for GSICS products. 18

GSICS EP Meeting 2015 – Report from the GSICS Data Management Working Group 19 End of Presentation: Thank you for your attention