1. Big Data in Earth Observation
Cristiano Lopes
21/03/2017

2. Diversity of EO Data at ESA
ESA Missions (active + legacy)
ERS, ENVISAT, GOCE, Cryosat, SMOS, Swarm, Proba-1, Proba-V
Copernicus Missions (active)
Sentinel 1, Sentinel 2, Sentinel 3
Third Party Missions (active + legacy)
Landsat, SPOT, ALOS, OceanSat, Aura, OMI, DMC, IRS, KOMPSAT, etc, etc, etc
High-level Summary:
46+ Missions
68+ Instruments (with countless different modes)

3. Quantity of EO Data at ESA
ESA + TPM Missions (active + legacy)
Archive: ~3-5PB, ~5Million items
Distribution: ~3 PB, ~6 Million items
Copernicus Missions (active)
Archive: ~8 PB, ~4 Million items
Distribution: ~2 Million items
Rate of publication (last 3 months): ~470TB, ~520K items
Rate of download (last 3 months): ~6PB, ~8M items
~ 1CD (820MB) / second
~ 3 items / second
(Expected to increase even more once both A&B units are fully active in the 3 initial Sentinels.)

4. EO Data Different Perspectives
Different types of Sensor and Resolutions
Radar (X, L, C)
Optical (multispectral and panchromatic)
Altimetry
Atmospheric
LIDAR
Magnetometers
Application Domains / Earth Science Topics:
Agriculture
Atmosphere
Solid Earth
Water
Land
Oceans and Coasts
Snow and Ice
Natural Disasters

5. EO Data is only the start… More is needed.
Single EO image
Multiple EO images, processed
Plus population density…
Urban growth
Credits: Urban Thematic Exploitation Platform

6. How to fit it all? Ensuring interoperability
The challenge is to fit “Big”, “Linked” and “Geo” data in an interoperable manner so that it solves the problems of the end-user…
For the “Linked Data” domain we are still at a very early phase  learning and understanding
Progress made in the “Big Data” domain, leasing to an on-going shift in paradigm

7. “Bring the user to the data”
The old approach of getting the user to download a copy of satellite data to his environment is rapidly becoming un-manageable
The new paradigm is to have the user perform the analysis “on-line”, i.e. get the user to upload the algorithm instead of downloading the data
This involves not only Earth Observation data but also links with in-situ data, social-economic elements, etc
On-going initiatives like the Thematic-Exploitation Platforms, address these topic by providing a framework were different data co-exist
for example, EPOS (sysmologogic data, near fault observatories,

8. Big Linked Data and EO Exploitation Platform
Interoperability in this domain is mandatory, examples:
data discovery and access
(considering Linked Data, not only Spatial)
processing, algorithm executing
this must follow a write-once, run anywhere approach
User AuthN & AuthZ must be shared.
How can this all fit together?
What can ESA offer to make EO data more easily available to the Linked Data community?

9. Linked data – possibilities ?
Collection / Dataset discovery based on similarities, linking:
data from same type of sensors (e.g. SAR ERS,ENVISAT)
data with same topology (e.g. Optical Multispectral Medium Resolution, Landsat and S2)
complementary data for application (e.g. Land Use, S2 and Population)
Provenance linking:
Results to input data (imagery, social economics, etc)
Results to scientific background (and back)

10. Ground Segment System Engineer Earth Observation Programmes
Cristiano Lopes
Ground Segment System Engineer
Earth Observation Programmes
Phone: