1. “PRESERVATION DOMAIN” Data Stewardship Interest Group
WGISS-42 Meeting – ESA-ESRIN, Frascati
19–22 September, 2016
Mirko Albani, European Space Agency
ESA UNCLASSIFIED – For Official Use

2. Background and Purpose
WGISS accomplishes its Data Preservation and Curation efforts through the Data Stewardship Interest Group (DSIG)
Purpose of the activities:
Enable the sharing of agency investigations, developments, experiences and lessons learned relating to EO data stewardship
Draft common cross-agency best practices or guidelines of data stewardship for possible adoption by WGISS
Sponsor technical exchanges and the conduction of Joint Activities and/or Pilot Projects on specific data stewardship topics
Establish and maintain a CEOS “Data Purge Alert” service
Contributing to GEO and Standardization activities
ESA UNCLASSIFIED – For Official Use

3. DSIG sessions during this meeting
Overview and Updates
Associated Knowledge Preservation Best Practice
CEOS Maturity Matrix
Session: Report on Agencies Stewardship Activities
Preservation Topics Discussion
Landing Page & DOI for knowledge
Glossary Extension
Transcription Chains
Thesaurus
ESA UNCLASSIFIED – For Official Use

4. Data Stewardship Interest Group
Overview & Updates
ESA UNCLASSIFIED – For Official Use

5. Data Management Principles Task Force
Feedback from GEO
Data Management Principles Task Force
(DMP-TF)
ESA UNCLASSIFIED – For Official Use

6. GD-07: GCI Development Overview
M.Albani & R. Moreno members of GD task team
ESA UNCLASSIFIED – For Official Use

7. Data Management Principles

8. Status of Data Management Principles Implementation Guidelines
Data Management Principles Implementation Guidelines presented as info document at GEO-XII Plenary (Nov15):
Description / explanation of each principle
Guidance on implementation, with examples
Resources implications for implementing each principle
Suggested metrics measuring adherence to each principle
Contribution from M.Albani and R.Moreno
Public review process held in Q1 2016:
Online survey and Review process led by IEEE (S. J. S. Khalsa)
Document updated on the basis of survey results, reviewed at GEO WP Symposium (2-4 May) and GEO Program Board (July).
Draft currently with GEO Secretariat: will be submitted for adoption at GEO-XIII Plenary (Nov)
In rosso punto di domanda
ESA UNCLASSIFIED – For Official Use

9. Data Management Principles Implementation Guidelines – Online Survey
Harmonised feedback was provided by European Space Agencies in the frame of LTDP WG activities:
Improvement of Data Quality Assurance aspects as only monitoring and assessment addressed
Include long term planning and knowledge preservation aspects
Include appraisal phase
ESA UNCLASSIFIED – For Official Use

10. Data Management Principles Implementation Guidelines – Survey Results
ESA UNCLASSIFIED – For Official Use

11. Data Management Principles Implementation Guidelines – Results
DMPs implementation
ESA UNCLASSIFIED – For Official Use

12. Data Management Principles Implementation Guidelines – Results
Difficulty in DMP Implementation
ESA UNCLASSIFIED – For Official Use

13. Data Management Principles Implementation Guidelines – Results
DMP Implementation Challenges
ESA UNCLASSIFIED – For Official Use

14. GEO GD07.03 3 Year Work Program – In Process
Subtask Description: Develop Data Management Principles (DMP) Guidelines and Implementation Process
2017 Proposed Subtask Activities
Propose topics, learning outcomes, and opportunities for conducting training and workshops centered around the DMP Principles.
Identify additional flagships for demonstrating the DMP Implementation Guidelines (IG) and how existing flagships can implement DMPs.
Identify criteria and process for DMP branding of archives implementing DMP IG.
2018 Proposed Subtask Activities
Request suggestions for improving and adding to the DMP Implementation Guidelines.
Request feedback for improving training and workshops.
Revise DMP IG further to reflect feedback, evolving practices, and possibility of reviews.
Revise proposed learning outcomes (developed during 2017?) to reflect feedback from participants.
Convene training and workshops about the DMP IG
ESA UNCLASSIFIED – For Official Use

15. GEO GD07.03 3 Year Work Program – In Process
2019 Proposed Subtask Activities
Revise topics and learning outcomes for training and workshops, based on feedback.
Create process to review candidate archives to receive DMP brand for implementing DMP IG
Identify and invite candidate archives to apply for DMP brand for implementing DMP IG
Initiate process of triennial update of DMP, or process of determining evaluation criteria
ESA UNCLASSIFIED – For Official Use

16. Best Practices/Guidelines
Data Stewardship
Best Practices/Guidelines
ESA UNCLASSIFIED – For Official Use

17. CEOS Data Stewardship Best Practices Document Tree
Policy Documents
Individual organizations' policies (stewardship, access, ...)
EO Data Stewardship Cooperative Framework
Applied to
CEOS EO Space Data Sets
Data Purge Alert Procedure & White Paper
High level framework documents
Applied to
Technical Documents
Preservation Workflow
Glossary of Acronyms and Terms
General guidelines and best practices
Support
Guidelines and best practices on specific topics
Persistent Identifiers Best Practice
EO Data Preservation Guidelines
Preserved Data
Set Content
Associated Knowledge Preservation
CEOS Maturity Matrix …
Technical implementation procedures
EO Data Set Consolidation Process …

18. Documents “Almost Done” and “Working in Progress” – Data Purge Alert White Paper
Final version v1.0 circulated to WGISS-all.
Recommended for adoption today

19. Data Purge Alert Procedure Status
Purge Alert Procedure endorsement at CEOS Plenary in Nov 2015
Finalization of Purge Alert White Paper and material needed to ensure visibility of Purge Alert Procedure on CEOS and WGISS Web Sites
Define approach to inform Commercial Data Owners of the existence of the procedure (data at highest risk, in particular old ones). Possibility for them to use the procedure.
TO BE DONE
ESA UNCLASSIFIED – For Official Use

20. Data Purge Alert Page in WGISS web site: already implemented

21. Data Purge Alert NEWS: to be implemented
In case of Purge Alert notification, the WGISS Chair will inform all CEOS Agencies Principals and publish a ”Purge Alert News” on the CEOS and WGISS web pages.
Main CEOS Web page: a flashing “News”
link to the WGISS relevant web page
WGISS Web Page: specific information on the purge alert notification

22. PURGE ALERT BY USGS
USGS' intent to purge most of their Landsat MSS and TM film and would like to utilize the CEOS Purge Alert to identify organizations that may be interested in obtaining the film
Further information awaited by USGS (appraisal information according to Purge Alert procedure)
Data Records description
List of available associated knowledge
Results and documentation of a Data Record Appraisal if available
Overview of constraints and conditions if any
Recommendations concerning Data Records handling
Point of Contact

23. Documents “Almost Done” and “Working in Progress” – Associated Knowledge Preservation
DEDICATED SESSION
ESA UNCLASSIFIED – For Official Use

24. Documents “Almost Done” and “Working in Progress” – CEOS Maturity Matrix
DEDICATED SESSION
ESA UNCLASSIFIED – For Official Use

25. Documents for adoption today
Assessment of Drafting Status and Discussion
Issue to DSIG for Review, comments and feedback
Final Presentation and Formal Approval v
Purge Data Alert White Paper v
Final Issue to WGISS v
REVIEW
FINALIZATION
WGISS#41
WGISS#42
03/16
04/16
05/16
06/16
07/16
08/16
09/16

26. Future documents timeline proposal
Assessment of New Drafting Status and Discussion
Issue to DSIG for Review
Comments & Feedback
Final Presentation and Formal Approval v v v
Associated Knowledge Preservation Best Practices
Final Issue to WGISS v
REVIEW
FINALIZATION
WGISS#42
WGISS#43
09/16
10/16
11/16
12/16
01/17
02/17
03/17

27. Future documents timeline proposal
Topic Presentation and Discussion
Comments & Inputs from DSIG
Assessment of New Drafting Status and Discussion
Issue to DSIG for Review
Comments & Feedback
Final Presentation and Formal Approval v v v v v
Final Issue to WGISS v
WGISS#42
WGISS#43
REVIEW
FINALIZATION
WGISS#44
09/16
10/16
11/16
12/16
01/17
02/17
03/17
04/17
05/17
06/17
07/17
08/17
09/17
10/17
CEOS Maturity Matrix

28. Document Management Table
Document name
Custodian
Document ID
File name
Current version
Date
Status
Planned release date
EO Data Preservation Cooperative Framework (DSIG Plan)
M. Albani, ESA
None
Presentations
Updated at each WGISS meeting
Q3/2016
Preservation Workflow
I. Maggio, ESA
CEOS/WGISS/DSIG/PW
1.0
03/2015
APPROVED
EO Data Stewardship Glossary
CEOS/WGISS/DSIG/GLOS
EO-DataStewardshipGlossary
1.1
09/2015
EO Data Preservation Guidelines
CEOS/WGISS/DSIG/EODPG
EO Data Set Consolidation Process
R. Cosac, ESA
CEOS/WGISS/DSIG/GEODSCP
Generic Earth Observation Data Set Consolidation Process
Preserved Data Set Content
R. Leone, ESA
CEOS/WGISS/DSIG/EOPDSC
EO Preserved Data Set Content
Persistent Identifiers Best Practice
K.Molch, DLR
CEOS/WGISS/ DSIG/PIDBP
CEOS Persistent Identifier Best Practices
01/2016
Data Purge Alert White Paper
CEOS/WGISS/DSIG/DPA
Presentation; to be published on WGISS web site. White paper
0.2
n/a
Update ongoing
Associated Knowledge Best Practice
CEOS/WGISS/DSIG/AKPBP
Associated Knowledge Preservation Best Practices
04/2016
Being drafted
Q1/2017

29. Cooperation with other WGs, VCs, Ad-hoc groups Initiatives
ESA UNCLASSIFIED – For Official Use

30. WGISS initiatives in support to Carbon Action – SIT Tec Workshop 2016
ECVs/CDRs Discovery and Access through WGISS Systems
Objective: facilitate discoverability and accessibility of ECV
Products and space-born CDRs relevant for the CEOS Carbon
Action via WGISS Interoperability Systems & Standards (FedEO/
CWIC/IDN, OpenSearch).
WGISS Carbon Portal
Objective: development as WGISS project of a CEOS WGISS Carbon Portal prototype similar to the Water portal
( to allow displaying Carbon datasets and providing assistance to scientists and general users in the development of related services & tools.
ESA UNCLASSIFIED – For Official Use

31. Cooperation with other WGs - Action
Action WGISS-41-08: In order to discover how WGISS can assess the needs of other CEOS working groups, virtual constellations, and ad-hoc groups with regards to WGISS capabilities:
Action WGISS-41-08a: WGISS members to identify, from within their own agencies, members of said groups.
Action WGISS-41-08b: WGISS-Exec to coordinate one-on-one interactions with said contacts.
ESA UNCLASSIFIED – For Official Use

32. Cooperation with other WGs - WGCLimate
WGClimate need & announcement:
Population of the ECV inventory with the characteristics of Climate Data Records and ECV products made available by CEOS agencies.
Rationale: ensure that ECVs and CDRs holdings are widely visible to users.
Dedicated website, to support the population of the inventory;
ECV Inventory Questionnaire Guide which provides guidance on how to populate the ECV inventory.
This periodic activity (update every 2 years) will provide the basis for a gap analysis,​and subsequent generation of a coordinated action plan to address any detected gaps and/or opportunities for the delivery of additional and improved Climate Data Records.
ESA UNCLASSIFIED – For Official Use

33. Recovery of historical datasets – initial draft list to be investigated
Mission(s)
Sensor(s)
Mission Owner
Data Set Location
Ozone profiles from 30 to 110 km
EURECA
ORA
ESA
Belgian Institute for Space Aeronomy (TBC)
Grille vertical profiles
Spacelab 1; ATLAS-1
Grille (Infrared Spectrometer)
Spacelab 1: NASA-ESA
ATLAS-1: NASA
Deuterium and Hydrogen atomic concentrations
ATLAS-1
ALAE
NASA
Unknown (Sensor Owner: CNRS)
Solar spectral irradiance
Spacelab 1; EURECA; ATLAS missions
SOLSPEC
EURECA: ESA
ATLAS missions: NASA
Institut Pierre Simon Laplace (TBC)
ESMR data sets from the polar regions
Nimbus-5
ESMR
NASA (NSIDC - National Snow & Ice Data Center)
ESMR Imagery of Brightness Temperature on 70 mm Film
GSFC - Goddard Earth Science Data and Information Services (GES DISC)
SeaSat SMMR data
SeaSat
SMMR
NASA/JPL
JPL Physical Oceanography DAAC
WGClimate requested to help in identifying additional datasets of interest for ECVs generation (and extension back in time) which were acquired in the past but are not accessible today.

34. Data Stewardship Related Conferences
ESA UNCLASSIFIED – For Official Use

35. Conference on BIG DATA: Feedback
ESA UNCLASSIFIED – For Official Use

36. BiDS’ 2016 Presentations and proceedings
62 papers selected for oral and 32 for poster presentations
Demonstration/industry session (with 10 demos) organised with poster session, to give the possibility to present live demos on big data applications.
Among the papers selected for oral presentation, 18 originated from 5 invited sessions devoted to horizontal themes, namely, Organisational Challenges for Big Data Space Missions, Advances in Big Data using OGC Standards, Image Information and Data Mining for Big Data, Big Data Impact on Industry, and Data Science Skills.
Presentations and web site:
Proceedings:
Next conference will be held in November 2017 at CNES (Toulouse)
ESA UNCLASSIFIED – For Official Use

37. Living Planet Symposium (LPS) 2016: Feedback
ESA UNCLASSIFIED – For Official Use

38. LPS16 Prague 9-13 May 2016
3100 participants Largest EO conference in the world
2700 abstracts submitted: 1800 posters and 900 orals presented in 11 parallel sessions
300+ students participated to the school Lab activities, organised by ESA and Charles University (Prague)
Plenary session - welcome by Czech Prime Minister, Minister of Transport, ESA Director General and ESA Director of EO programmes
Industry exhibition with more than 30 companies operating worldwide including Google and Amazon
Public exhibition attended by hundreds of Prague citizens

39. PV2017: 7-9 November 2017
Venue: Rutherford Application Laboratory (RAL), Harwell, UK
Dates: 7 – 9 November 2017 (2.5 days)
Anticipated attendance: people
ESA UNCLASSIFIED – For Official Use

40. Thanks you for your attention !!!
ESA UNCLASSIFIED – For Official Use