1. 1

2. SCIDIP-ES Objectives
To develop and deploy generic Services and Toolkits for data preservation. Validated in the ES domain
ES perspective: providing use cases, environment for S&T deployment, identify test datasets, SW testing, feedback and validation activities.
To analyse the state of the art/users needs and propose harmonisation activities for digital data preservation in the Earth Science domain:
Paving the way for the set-up of a shared and common approach for Long Term Preservation of Earth Science Data.

3. SCIDIP-ES partners Earth observation repositories
Earth Science repositories
Software house specialized in ES preservation systems (acs, cap, gim)
R&D in preservation Systems (APA, ENG, Forth, ICT, FTK)
University skilled in standards and preservation systems (UTV and JUB)

6. North pole Ice pack reduction
measured by EO
satellites between
1978 and 2010
This time series is used to show the progressive reduction of the sea ice in the Arctic area. Sea Ice melting is one of the Essential Climate Variables.
Ice melting at the north pole would significantly impact on our global environment (which we need to model and forecast). Moreover it would have an incredible impact on sea routes traffic (90% of the goods are transported on the sea) and therefore on the global economy.
It is essential that our scientist have the full access to all information related to this (and other) phenomena.
Unfortunately satellites (and/or experiments) are not forever. The video you see was realized unifying over 4 missions (i.e. 4 different satellites and sensor). This poses problems of file conversion, metadata harmonization. Which must be planned and have a cost.

7. Terabytes

8. THREATS Changes in: Format Software Hardware IPR Funding
Designated Communities
THREATS

9. GOCE turns out to be the first seismometer in orbit
Earth Science Change in data user community
User community is changing
Now scientists working with Earthquakes want this information to study earthquakes propagation in the atmosphere
GOCE turns out to be the first seismometer in orbit

10. LTDP WG
LTDP WG under ESA leadership formed in 2008 within the Ground Segment Coordination Body (GSCB) to better Coordinate European (and Canadian) efforts and activities in the domain of digital data preservation
Implemented the basic rules of a European LTDP Framework in Earth Observation: “LTDP Common Guidelines and Preserved Data Set Content”
Reflecting the consensus of the European EO Data providers
Reviewed at international level (GEO, CEOS, NASA)
Aligned with EO data quality standards (QA4EO)

11. State of the art (EO example)
Mission managers actually ask for document/software collection for data users.
Collection could be not properly organized or different from other mission
Not manageable in the long term
Not directly connected to catalogued data
Users will “Google” for information: not sure they will find the latest and/or original version

12. Practical problems
Mission concept
Mission design
Mission impl
Mission operation
Post/Preservation
Need to convince mission responsibles to include preservation from the start
Need not to change the normal routines/technologies in the archives
No technicalities (e.g. no OAIS training to understand the solution)
Need to show immediate benefits with almost no effort

13. THE SCIDIP-ES solution
WP 15
T. 33.3
WP 21
WP 12
MERIS Full Resolution- Level 2
MERIS IPF
BEAM
User Manual
Tech Details
Other docs
Level 1
Level 0

14. Building a Preserved Dataset Content
Defines what, why, how, to be preserved for an EO mission
Based on the EO LTDP defined by LTDP WG
Analysis of all dependencies (structural, semantics)
Extended to Earth Science (generic)
MERIS Full Resolution- Level 2
MERIS IPF
BEAM
User Manual
Tech Details
Other docs
Level 1
Level 0

15. Phases
Mission Preparation
Mission Operation
Post Mission
Framework for ES archivist - SCIDIP-ES developers communication

17. KNOWLEDGE BROWSER

18. DLR NDVI - Obtained from MERIS L3
MERIS Full Resolution- Level 3
MERIS L3 IPF
Tech Details
MERIS Full Resolution- Level 2
MERIS IPF
BEAM
User Manual
Tech Details
Other docs
Level 1
Level 0

20. Improvements (from feedbacks)
Improve the perception for SCIDIP-ES platform and S&T usability and understandability
Taking advantage of achievements in D33.3 (preservation ontology for ES data preservation used in the S&T)
De-coupling technology and knowledge of ISO standards details from effective content ingestion and data preservation actions
Final users perception would change dramatically: no need to understand OAIS, no need to fill in the registry, all data/knowledge immediately organized (semantically, structurally)
“Easy preservation” would facilitate adoption of preservation actions since the beginning of a mission or during its phases

22. Infrastructure take-up
As with any mature infrastructure – internet, mobile phone etc there is a need for
Small number of experts who set up the infrastructure
software, OAIS, terminology
Large number of end users for whom the experience is as painless as possible e.g.
Users (archivists etc) do not want to change their habits
You have to help them without complicating their life 
IN PROGRESS

23. Functionalities in a nutshell
Easy ingestion of ES-PDSC: users are guided through a simple wizard to recovery and upload data
Once updated it is easy to browse data-related knowledge
Easy to connect SCIDIP-ES registries to existing catalogues
SCIDIP-ES registries can be connected one to each other to share preservation effort between agencies
All services integrated in a single infrastructure via a common entry point
Critical mass of users

24. Preservation functionalities in a nutshell
Allow data managers to know something has changed
Facilitate their understanding of the implications of that change
Allow them to decide on the best course of action for preservation
Understand (and create) the metadata which is needed to fill the gaps (created by someone else or creating a new one)
If transformed: how to maintain data authenticity
Alternatively: hand it over to another repository (real case here)
Make sure data is now usable and close the process

26. Deployments APA APA will provide, after the project the:
“guarantor/exchange” nodes for the Registry etc
Build system
Interactive platform for users to try things out and provide feedback
This will be cleared each day.

27. Deployment at ESA VM1 VM2 ESA DASHBOARD RepInfo Registry GAP Identif
Service
Finding
Aids
Certification
ESA DASHBOARD
Happi
Storage
Packaging
Orchestration

28. The Seasat dataset
Seasat mission has significant historical value (being the first SAR) but low scientific impact (having completed only few passes). The data has been preserved so far on a best effort basis.
Perfect example for archive building
These data were on a tape and are being now reprocessed to generate products. All information have been collected and is now used as perfect example here.

29. The Nimbus-7 CZCS dataset
The Nimbus-7 spacecraft was turned off in 1994 after 16 years of service. During its long period of operation, the Nimbus-7 mission was regarded as the single most significant source of experimental data from Earth's orbit relating to atmospheric and oceanic processes.

30. What we were able to understand
Being the ESA registry based on the “Enterprise Version” and the APA based on the “Light Web Version” we (involuntarily) demonstrated we agree to the main principles of service orientation:  
standardized service contract: from the external view, the registry are the same
service loose coupling: no intra-components dependency
service autonomy: independent functional boundary and runtime environment (the switch-over was immediate and painless)

31. Future architecture at ESA (under consolidation)
ESA EO Data catalogue and dissemination system
Operational environment
ESA Earth Observation Knowledge Management System
ESA EO web portal
ENVRI Catalogue
SCIDIP-ES
ESA Dashboard
Research environment

32. ON top of these services
ESA is building a Virtual Research Environment e-infrastructure including different Earth Science Virtual Research Communities scenarios
Will make use of a subset of SCIDIP-ES services

33. DLR deployments
With this initial deployment, it is possible to achieve semantic connection between the DLR and ESA MERIS datasets

34. DLR – Screenshots

35. ISPRA Deployment structure
VMWare

36. BGS/BADC: example client applications
RIN’s created for key data types and managed via RIT (e.g. NetCDF, geophysical data, and 3D models)
On-line catalogue functionality enhanced by the provision of preservation information (via knowledge browser):
Earth Science Academic archive (ESAA - )
Strategic Environmental Assessment dataset catalogue (SEA -
BADC client
Next steps:
Deploy preservation assistant to assist day to day users load their data
Orchestration service to help us manage the LTDP of some of our more s/w depended data formats which represent a significant investment in data capture/ingestion (e.g. geophysical data)

37. INGV Scenarios & Dataset
INGV has identified two scenarios, and relative dataset, for using and testing SCIDIP-ES solutions in preservation tasks:
MOIST: Multidisciplinary Oceanic Information SysTem
(geomarine observation) -
DidYouFillIt : Earthquakes Maps -

38. Preservation
Virtual Machine

39. To close the loop opened with the survey for D15.1
Will show SCIDIP-ES results
For Data Managers, Data Providers, Data Users
Opened from Mid September 2014
Will contain evaluation methods and commitment to adopt our ideas
Will reach almost 10K persons in the ES domain
Results will form consistent part of final review discussion

40. Data managers: Data providers: End users:
The preservation infrastructure; E-PDSC; Federation model including access; LTDP MM
Data providers:
Preservation themes; E-PDSC; Federation Model including governance and policies; LTDP maturity model (20 questions)
End users:
Service federation - end user perspective; E-PDSC; Preservation infrastructure;