Jordi Farres HMA-WG Meeting ESRIN, 23 Jan 2013 Setting the Scene Jordi Farres HMA-WG Meeting ESRIN, 23 Jan 2013 ESA UNCLASSIED - For Offical Use 23/01/2013

A model of scientific exploitation – Earth Web Agenda Introduction A model of scientific exploitation – Earth Web A model of the scientific user services Conclusions ESA UNCLASSIED - For Offical Use 23/01/2013

Introduction Increasing data volumes and data footprint for scientific projects Open data policy and related expectations High pressure for better dissemination and processing services Surge of Cloud Computing Workshop Models for Scientific Exploitation of EO data: Innovation, harmonization, technology large consensus on the need to foster collaboration and further exploit the concept of “collaborative ground segment Participants underlined the huge benefits of synergic exploitation of EO data collected from different missions There is a clear need for testing, harmonising and standardising the user-infrastructures already in place, in order to establish the terms for evolution and further collaboration. ESA UNCLASSIED - For Offical Use 23/01/2013

A model of scientific exploitation – Earth Web Agenda Introduction A model of scientific exploitation – Earth Web A model of the scientific user services Conclusions ESA UNCLASSIED - For Offical Use 23/01/2013

What is the preferred exploitation model? An Earth Box An Earth Web Drop all data on large Dropbox Place all data in the cloud Users / service autonomously develop: Applications, Services and Research Expose data on an exploitation platform Federate services from different EO providers Users develop the value added services on a common framework ESA UNCLASSIED - For Offical Use 23/01/2013

Intro: An Earth Web A facility for EO dissemination and processing services geographically distributed in a federated domain largely scalable with reliable QoS ESA UNCLASSIED - For Offical Use 23/01/2013

ASAR, RA, GOME, GOMOS, MIPAS, ... A case study: G-POD Service ladder ESA ESRIN UK-PAC ESAC* Cloud 6- Portal – User and Info mgt √ 5- Service Mgt Deployment 4- Processing Services >50 3 3- Data Mgt Discovery & Catalogue 2- Data Resources ASAR, RA, GOME, GOMOS, MIPAS, ... (A)ATSR, MERIS SMOS SAR subset 1- ICT Resources 2 Clusters 1 Cluster IaaS * Under construction ESA UNCLASSIED - For Offical Use 23/01/2013

A case study: G-POD Service ladder ESA ESRIN UK-PAC ESAC* Cloud 6- Portal – User and Info mgt 5- Service Mgt Deployment 4- Processing Services 3- Data Mgt Discovery & Catalogue 2- Data Resources 1- ICT Resources PORTAL CATALOGUE CATALOGUE * Under construction DATA DATA DATA COMPUTING COMPUTING COMPUTING ESA UNCLASSIED - For Offical Use 23/01/2013

A case study: G-POD Service ladder ESA ESRIN UK-PAC ESAC* Cloud 6- Portal – User and Info mgt 5- Service Mgt Deployment 4- Processing Services 3- Data Mgt Discovery & Catalogue 2- Data Resources 1- ICT Resources PORTAL PROCESSOR PROCESSOR * Under construction COMPUTING COMPUTING COMPUTING ESA UNCLASSIED - For Offical Use 23/01/2013

An Earth Web Model – Service ladder Partner A Location 1 Location 2 Partner B Partner C 6- Portal – User and Info mgt √ 5- Service Mgt Deployment 4- Processing Services 3- Data Mgt Discovery & Catalogue 2- Data Resources 1- ICT Resources EO Community Mgt EO Service Provider EO Resources Provider Cloud Provider ESA UNCLASSIED - For Offical Use 23/01/2013

An Earth Web Model – Interoperability Service ladder Technologies Standards HMA-AWG 6- Portal – User and Info mgt SSO SAML Partial (Identity mgt) 5- Service Mgt Deployment PF Virtual Machines Web Services IPF I/F OVF WPS-T NO 4- Processing Services WPS Partial 3- Data Mgt Discovery & Catalogue CSW YES 2- Data Resources https Metadata File Formats WCS 1- ICT Resources IaaS Grid brokers OCCI, EC2 ESA UNCLASSIED - For Offical Use 23/01/2013

An Earth Web Model – Challenges Execute processing X on data Y over server Z (portability) Quantify user authorization in relation processing resources Profile users in relation to QoS Ability to quote “budget” for processing X on data Y Accounting à la cloud for ICT resources Harmonized authorization and accounting … ESA UNCLASSIED - For Offical Use 23/01/2013

A model of scientific exploitation – Earth Web Agenda Introduction A model of scientific exploitation – Earth Web A model of the scientific user services Conclusions ESA UNCLASSIED - For Offical Use 23/01/2013

Requirements 1/2 Capabilities - Support EO research activities Power Access to large data sources Availability of best processing/visualization tools Ability to scale from local processing to large processing campaigns Control of sharing capabilities for results (data), processing services (WPS), software, documentation Power Scalability to large ICT capabilities; e.g. hundreds of TBs and cores CDN-like capabilities for large dissemination Multi-sourcing ESA UNCLASSIED - For Offical Use 04/12/2012

Requirements 2/2 Usability Support Business Model Simple (seamless?) deployment from WS to EP4x Built-in collaboration capabilities No entry-barriers - very simple start-up Transparent accounting; e.g. single account for IaaS, data, sw licenses; pay-per-use model Support Business Model For Data Providers For Software Vendors For Cloud providers ESA UNCLASSIED - For Offical Use 04/12/2012

Exploitation Model: Actors Data Provider: ESA and other EO data providers serving commercial or public data over common ordering protocols. PI: End-user of the platform. It can be a paying or a sponsored user. Software Vendor Software vendor providing software download and licensing on cloud resources. License could be Open Source or not. Free or not. Software could be for usage on IaaS or also available also on PaaS/SaaS. Cloud Provider Provider for public IaaS, PaaS and SaaS services. ESA UNCLASSIED - For Offical Use 04/12/2012

Technical pre-requisites for deployment Application developed in line with specific development environment and/or compliant with established interfaces. Embedded software licensed to run on the cloud platform ESA UNCLASSIED - For Offical Use 04/12/2012

Pre-requisites for application Publishing: Application usage accounting to be integrated in a pay-per-use function Sub-licensing of embedded software Integration with user support lines ESA UNCLASSIED - For Offical Use 04/12/2012

ESA UNCLASSIED - For Offical Use 04/12/2012

ESA UNCLASSIED - For Offical Use 04/12/2012

ESA UNCLASSIED - For Offical Use 04/12/2012

Projects are an instrument for grouping and sharing objects. Projects are aimed to process EO data for a certain purpose; so they are structured as: A software project; ie. Including a sw repository, a Developer forum, an issue tracker A set of reference data; ie. Including sample input data, reference ground-truth data. A certain community of trial users; ie. A trial service, a user forum, faq Plus documentation related to all previous three Project objects are could be available in different repositories - in the cloud or somewhere else – under compatible identity/authorization management. The trial service is expected to be an early deployment of the processing software to the processing platform. ESA UNCLASSIED - For Offical Use 04/12/2012

EO Data Exploitation ESA UNCLASSIED - For Offical Use 04/12/2012

New EO Service development ESA UNCLASSIED - For Offical Use 04/12/2012

New EO Product development ESA UNCLASSIED - For Offical Use 04/12/2012

A model of scientific exploitation – Earth Web Agenda Introduction A model of scientific exploitation – Earth Web A model of the scientific user services Conclusions ESA UNCLASSIED - For Offical Use 23/01/2013

Conclusions To Be Written ESA UNCLASSIED - For Offical Use 23/01/2013