1. Jordi Farres HMA-WG Meeting ESRIN, 23 Jan 2013
Setting the Scene
Jordi Farres
HMA-WG Meeting
ESRIN, 23 Jan 2013
ESA UNCLASSIED - For Offical Use /01/2013

2. A model of scientific exploitation – Earth Web
Agenda
Introduction
A model of scientific exploitation – Earth Web
A model of the scientific user services
Conclusions
ESA UNCLASSIED - For Offical Use /01/2013

3. Introduction
Increasing data volumes and data footprint for scientific projects
Open data policy and related expectations
High pressure for better dissemination and processing services
Surge of Cloud Computing
Workshop Models for Scientific Exploitation of EO data: Innovation, harmonization, technology
large consensus on the need to foster collaboration and further exploit the concept of “collaborative ground segment
Participants underlined the huge benefits of synergic exploitation of EO data collected from different missions
There is a clear need for testing, harmonising and standardising the user-infrastructures already in place, in order to establish the terms for evolution and further collaboration.
ESA UNCLASSIED - For Offical Use /01/2013

4. A model of scientific exploitation – Earth Web
Agenda
Introduction
A model of scientific exploitation – Earth Web
A model of the scientific user services
Conclusions
ESA UNCLASSIED - For Offical Use /01/2013

5. What is the preferred exploitation model?
An Earth Box
An Earth Web
Drop all data on large Dropbox
Place all data in the cloud
Users / service autonomously develop: Applications, Services and Research
Expose data on an exploitation platform
Federate services from different EO providers
Users develop the value added services on a common framework
ESA UNCLASSIED - For Offical Use /01/2013

6. Intro: An Earth Web A facility
for EO dissemination and processing services
geographically distributed
in a federated domain
largely scalable
with reliable QoS
ESA UNCLASSIED - For Offical Use /01/2013

7. ASAR, RA, GOME, GOMOS, MIPAS, ...
A case study: G-POD
Service ladder
ESA
ESRIN
UK-PAC
ESAC*
Cloud
6- Portal – User and Info mgt √
5- Service Mgt Deployment
4- Processing Services
>50 3
3- Data Mgt Discovery & Catalogue
2- Data Resources
ASAR, RA, GOME, GOMOS, MIPAS, ...
(A)ATSR, MERIS
SMOS
SAR subset
1- ICT Resources
2 Clusters
1 Cluster
IaaS
* Under construction
ESA UNCLASSIED - For Offical Use /01/2013

8. A case study: G-POD Service ladder ESA ESRIN UK-PAC ESAC* Cloud
6- Portal – User and Info mgt
5- Service Mgt Deployment
4- Processing Services
3- Data Mgt Discovery & Catalogue
2- Data Resources
1- ICT Resources
PORTAL
CATALOGUE
CATALOGUE
* Under construction
DATA
DATA
DATA
COMPUTING
COMPUTING
COMPUTING
ESA UNCLASSIED - For Offical Use /01/2013

9. A case study: G-POD Service ladder ESA ESRIN UK-PAC ESAC* Cloud
6- Portal – User and Info mgt
5- Service Mgt Deployment
4- Processing Services
3- Data Mgt Discovery & Catalogue
2- Data Resources
1- ICT Resources
PORTAL
PROCESSOR
PROCESSOR
* Under construction
COMPUTING
COMPUTING
COMPUTING
ESA UNCLASSIED - For Offical Use /01/2013

10. An Earth Web Model – Service ladder
Partner A
Location 1
Location 2
Partner B
Partner C
6- Portal – User and Info mgt √
5- Service Mgt Deployment
4- Processing Services
3- Data Mgt Discovery & Catalogue
2- Data Resources
1- ICT Resources
EO Community
Mgt
EO Service
Provider
EO Resources
Provider
Cloud Provider
ESA UNCLASSIED - For Offical Use /01/2013

11. An Earth Web Model – Interoperability
Service ladder
Technologies
Standards
HMA-AWG
6- Portal – User and Info mgt
SSO
SAML
Partial (Identity mgt)
5- Service Mgt Deployment PF
Virtual Machines
Web Services
IPF I/F
OVF
WPS-T NO
4- Processing Services
WPS
Partial
3- Data Mgt Discovery & Catalogue
CSW
YES
2- Data Resources
https
Metadata
File Formats
WCS
1- ICT Resources
IaaS
Grid brokers
OCCI, EC2
ESA UNCLASSIED - For Offical Use /01/2013

12. An Earth Web Model – Challenges
Execute processing X on data Y over server Z (portability)
Quantify user authorization in relation processing resources
Profile users in relation to QoS
Ability to quote “budget” for processing X on data Y
Accounting à la cloud for ICT resources
Harmonized authorization and accounting …
ESA UNCLASSIED - For Offical Use /01/2013

13. A model of scientific exploitation – Earth Web
Agenda
Introduction
A model of scientific exploitation – Earth Web
A model of the scientific user services
Conclusions
ESA UNCLASSIED - For Offical Use /01/2013

14. Requirements 1/2 Capabilities - Support EO research activities Power
Access to large data sources
Availability of best processing/visualization tools
Ability to scale from local processing to large processing campaigns
Control of sharing capabilities for results (data), processing services (WPS), software, documentation
Power
Scalability to large ICT capabilities; e.g. hundreds of TBs and cores
CDN-like capabilities for large dissemination
Multi-sourcing
ESA UNCLASSIED - For Offical Use /12/2012

15. Requirements 2/2 Usability Support Business Model
Simple (seamless?) deployment from WS to EP4x
Built-in collaboration capabilities
No entry-barriers - very simple start-up
Transparent accounting; e.g. single account for IaaS, data, sw licenses; pay-per-use model
Support Business Model
For Data Providers
For Software Vendors
For Cloud providers
ESA UNCLASSIED - For Offical Use /12/2012

16. Exploitation Model: Actors
Data Provider:
ESA and other EO data providers serving commercial or public data over common ordering protocols.
PI:
End-user of the platform. It can be a paying or a sponsored user.
Software Vendor
Software vendor providing software download and licensing on cloud resources. License could be Open Source or not. Free or not.
Software could be for usage on IaaS or also available also on PaaS/SaaS.
Cloud Provider
Provider for public IaaS, PaaS and SaaS services.
ESA UNCLASSIED - For Offical Use /12/2012

17. Technical pre-requisites for deployment
Application developed in line with specific development environment and/or compliant with established interfaces.
Embedded software licensed to run on the cloud platform
ESA UNCLASSIED - For Offical Use /12/2012

18. Pre-requisites for application Publishing:
Application usage accounting to be integrated in a pay-per-use function
Sub-licensing of embedded software
Integration with user support lines
ESA UNCLASSIED - For Offical Use /12/2012

19. ESA UNCLASSIED - For Offical Use 04/12/2012

20. ESA UNCLASSIED - For Offical Use 04/12/2012

21. ESA UNCLASSIED - For Offical Use 04/12/2012

22. Projects are an instrument for grouping and sharing objects.
Projects are aimed to process EO data for a certain purpose; so they are structured as:
A software project; ie. Including a sw repository, a Developer forum, an issue tracker
A set of reference data; ie. Including sample input data, reference ground-truth data.
A certain community of trial users; ie. A trial service, a user forum, faq
Plus documentation related to all previous three
Project objects are could be available in different repositories - in the cloud or somewhere else – under compatible identity/authorization management.
The trial service is expected to be an early deployment of the processing software to the processing platform.
ESA UNCLASSIED - For Offical Use /12/2012

23. EO Data Exploitation
ESA UNCLASSIED - For Offical Use /12/2012

24. New EO Service development
ESA UNCLASSIED - For Offical Use /12/2012

25. New EO Product development
ESA UNCLASSIED - For Offical Use /12/2012

26. A model of scientific exploitation – Earth Web
Agenda
Introduction
A model of scientific exploitation – Earth Web
A model of the scientific user services
Conclusions
ESA UNCLASSIED - For Offical Use /01/2013

27. Conclusions
To Be Written
ESA UNCLASSIED - For Offical Use /01/2013