ESA UNCLASSIFIED – For Official Use Data Stewardship Interest Group Session LTDP Activities & DSIG Introduction WGISS-37 Meeting Cocoa Beach (Florida-US) - April 14-18, 2014

Long Term Data Preservation (LTDP) Activities at ESA and In Europe ESA UNCLASSIFIED – For Official Use

LTDP Main Drivers at ESA Data and Knowledge Preservation Preserve any mission acquired data & associated knowledge as humankind asset safeguarding MSs investment & effort in space R&D & operational data acquisition. With the assumption/awareness that one can never anticipate the exploitation value of the data asset in the future. Preservation to be addressed in all EO mission phases; incremental approach for past/current missions, systematic planning/approach for future ones. Data Valorization Make sure full content of the ESA archives is fit for contemporary purpose and maintained accessible and usable in the long term. Valorize the data assets for current/future exploitation through building long time series of coherent data (harmonized formats and metadata) and data collections for specific applications (e.g. Fundamental Climate Data Records); pursue multi- disciplinary data use and “cross-fertilization”. Optimize investments in future missions GS Dev&Ops, anticipating LTDP requirements and operational scenarios.

ESA LTDP Programme Activities (EO Part) 1.LTDP Coordination and International Cooperation Continue international cooperation on LTDP in EO and implement basic activities to sustain the European LTDP cooperative framework for national and international missions. Consolidate the ESA EO Data Preservation Scenario and apply to all ESA missions and data holdings. 2.EO Historical Missions Datasets Consolidation, Access, Preservation & Curation Consolidate and archive ESA EO Historical Missions data & associated information; ensure LTDP missions data discovery & access as is today and perform full reprocessing & alignment to current missions when feasible. 3.LTDP Specific Developments and Evolutions Implement LTDP specific evolution and consolidation activities necessary to meet programme objectives.

European LTDP Framework EC Projects ESA membership Standards WGISS DSIG Task IN-02 LTDP Activities International Context (since 2008)

EO International Coordination and Cooperation Activities (1) ESA is coordinating the LTDP cooperation activities in the Earth Observation domain with European partners through the LTDP WG formed within the Ground Segment Coordination Body (GSCB). Implemented the basic rules of the European LTDP Framework in Earth Observation: “LTDP Common Guidelines and Preserved Data Set Content” Reflecting the consensus of the European EO Data providers. Issued in GEO Reviewed at CEOS and with NASA, being reviewed with QA4EO. “EO Preservation Workflow” and harmonized high level procedures for E2E data consolidation and data media transcription under definition.

LTDP CORE DOCUMENTS LTDP Guidelines Issue 2.0 available at: Continuously evolving: next update in Q CGMS WG (Korea, ESA, EUM, WMO) is considering the LTDP Guidelines as starting point for the generation of data preservation guidelines. PDSC Issue 4.0 available at: Continuously evolving: next update in Q Preservation Workflow and E2E Consolidation Procedure being drafted: Based on the experience gained through historical missions data set curation activities; Reviewed and commented by the ESA sensors experts team and by the LTDP WG.

Completed three joint studies: LTDP Initiatives and Standards Survey LTDP User Requirements Study (FIRST) Future Archive Technology Study (LAST) EO International Coordination and Cooperation Activities (2)

Defined the initial data set to be preserved, including the related glossary. Several technical activities ongoing or planned: LTDP/QA4EO Study LTDP Architecture Definition Project Interagency project for AVHRR coherent data series generation (harmonized L0 data consolidation, processing, discovery & access). SAFE Consolidation Persistent Identifiers Guaranteed the information flow through workshops, web sites, participation to conferences and LTDP related events. EO International Coordination and Cooperation Activities (3)

ESA EO Long Time Data Series Holdings (excerpt) Missions in red are covered in the LTDP Programme

ESA EO Data Preservation Scenario implementation Historical Missions Assessment of archived data and information holdings versus “PDSC” on-going for all ESA and ESA managed TPMs starting from historical ones and ERS-1/2. Consolidation of Level-0 datasets (and associated information), and generation of two copies to be stored in two different locations; ingestion in robotic tape libraries in the format of standard Submission Information Packages (SIPs); data migration to latest available archive technology. Ensure basic discoverability via ESA catalogues, products generation through the currently available processors and dissemination through ESA standard systems; reprocessing when feasible for alignment to future missions with particular focus on Climate Change Initiative requirements and needs. Progressive historical archives rationalization. Lessons learned used to update the Preservation Scenario.

Information flow guaranteed through workshops, web sites, participation to conferences Institutional web site: Contains basic LTDP documents (e.g. Guidelines) LTDP events and related material (e.g. Workshops) Technical Web Site: Contains technical documentation and results of LTDP Cooperation activities Forum for technical discussions Web Sites and communication

Data Stewardship Interest Group (DSIG) Introduction and Objectives

ESA UNCLASSIFIED – For Official Use DSIG Introduction (1) Purpose: Provide WGISS EO Data Stewardship Guidance Goals: Sharing of agency investigations, developments, and lessons- learned relating to EO data stewardship Share experiences and lessons learned Draft common cross-agency best practices or guidelines of data stewardship for possible adoption by WGISS Sponsor technical exchanges at WGISS meetings or elsewhere Scope: Focus on Data, Metadata and Product Topics Including: Long-Term Archive Strategies Data Formats Data Preservation Data Lifecycle Concepts Archive Media Background: Former WGISS Archive Task Team Renewed Interest by WGISS Dating to WGISS-27 (Toulouse)

ESA UNCLASSIFIED – For Official Use DSIG Introduction (2) ACHIEVEMENTS White Papers/Reports: Long-Term Archive Strategies Data Preservation Techniques Data Lifecycle Models and Concepts Browse Survey Guidelines for GIS-Ready Products Browse Guidelines Document (version 2) Offline Media Trade Study Best Practices/Recommendations: European Long Term Preservation of Earth Observation Space Data Preview Image Principle Data Management Statement

ESA UNCLASSIFIED – For Official Use DSIG Introduction (3) Current Point of Contact: John Faundeen (TBC) Members:

ESA UNCLASSIFIED – For Official Use Outlook on Possible Future Activities 1. Sharing Information and experiences about agency investigations, developments, studies, and lessons-learned relating to EO data stewardship Periodic report from CEOS agencies during WGISS meetings 2. Draft common cross-agency best practices or guidelines on data stewardship for possible adoption by WGISS: Preservation Workflow Preservation Guidelines (update) Preserved Data Set Content (update) E2E Level-0 data consolidation Archived data media transcription Persistent Identifiers Purge alert procedure Appraisal procedure Propose CEOS DSIG best-practices/guidelines for adoption in GEO and as input to standardization activities (e.g. OGC/CCSDS)

ESA UNCLASSIFIED – For Official Use Short Term Proposed Activities 1.Extend DSIG membership through invitation to WGISS-all with DSIG plan of activities and request of expression of interest 2.Set-up DSIG reporting by CEOS interested agencies from next WGISS 3.Focus, as a start, on Best-practices/recommendations on: Preservation Workflow Persistent Identifiers Purge Alert 4.Update CEOS Purge Alert mailing list 5.Contribute to “GEO Data Management Principles Task Force (DMP TF)” activities which will support GEO in the definition and endorsement of common GEOSS Data Management Principles covering the entire data life cycle from planning, to acquisition, quality assurance, documentation, access, archiving and preservation.

ESA UNCLASSIFIED – For Official Use Thank you for your attention !!! Comments ? Questions ?