RSS support to scientific exploitation of EO data Giancarlo Rivolta giancarlo.rivolta@esa.int ESRIN, 11 October 2012

Summary RSS Overview RSS G-POD process RSS flexible resources for on-demand processing Success stories Future directions GPOD demo Questions & Answers

RSS Overview Research and Service Support Let’s focus on G-POD The ESA Research and Service Support (RSS) service provides resources for supporting Earth Observation (EO) data exploitation RSS users are EO Principal Investigators and EO Service Providers The environments made available to RSS users are: Grid Processing On-Demand (G-POD) Service Support Environment (SSE) E-collaboration environment (Join&Share) OGC services (WMS, WFS, WCS) Virtual Reality Theatre (VRT) Knowledge-based Earth Observation (KEO) Reference Data Sets (RDS) EO Ontology search engine More than 3500 people are registered as RSS users On the industry side Logica delivers the RSS service as part of the ESRIN O&M Framecontract More information available from the RSS Portal (rssportal.esa.int ) Let’s focus on G-POD

Research Process

RSS G-POD process steps Principal Investigators EO algorithms delivery Data type and range indication Output validation On-demand EO data processing Use of produced data (projects delivery, publications, etc) RSS Data are made available in the RSS catalogue Algorithm porting Integration design Implementation Test and validation (involving the PI) On-demand EO data processing Delivery

RSS Flexible Resources On-demand processing service: EO Scientists Principal Investigators Process delivery EO data Application Technology Infrastructure ESRIN - 172 cores - 400 TB UK-PAC - 88 cores - 300 TB Flexible Infrastructure - 10-200 cores - 1-10 TB Flexible Infrastructure satisfies: HW requirements Connectivity requirements SLA (HA, help desk, ticketing systems, etc.)

RSS G-POD Team On-demand Processing: team composition and responsibility The G-POD team is composed by 4 selected highly skilled resources Key areas of responsibility are: Interaction with Principal Investigators Algorithm Understanding and proposal evaluation Resources planning (data, storage, integration, capacity, etc) Algorithm integration (porting, design, implementation, test) Data management Processing environment set-up User support (for on-demand processing) Bulk Processing management Key skills are: Remote sensing background / EO data exploitation expertise Software development /software integration Virtualization technology Grid / Cloud technology Operations and Maintenance Scientific goal orientation

Success Story: MGVI MERIS Global Vegetation Index Cat-1 project Objective: Development of MERIS derived products based on the Fraction of Absorbed Photosynthetically Active Radiation (FAPAR) algorithm IPR: MGVI algorithm developed and owned by the Joint Research Centre Evolution and time span: Several GPOD-MGVI versions integrated since 2007, bulk re-processings over the entire ENVISAT mission (2002-2012) Outcomes/Benefits: Access to online data for bulk re-processing Systematic global products published on ESA Earthnet Portal Regional products distributed via RSS SSE and RSS OGC service as well On-demand MGVI regional available MGVI Europe / Africa distributed to JRC for creating products for desertification monitoring Several scientific papers published MGVI delivered as complementary information to Fire detection products over Senegal From JRC contribution to the paper “A Model for the Scientific Exploitation of Earth Observation Missions”: RSS supported the process of algorithm development, test and deployment over a general purpose processing on demand environment dedicated to the scientific exploitation of EO data generated by ESA missions, as well as the provision of tools and environments in support to that process.

Success Story: KLIMA-IASI The KLIMA-IASI ESA Cat-1 project was initially structured along 3 phases 1- The KLIMA-IASI application is integrated in G-POD The data set used consists of: IASI data levels L1B, L1C and L2, in EPS format; for the month of July 2008; The results of the application on this data set are used to determine if results produced from Level L1B data are better than the results produced from Level L1C; The phase is concluded with the decision to use Level L1B or Level L1C data; Also, during this phase, processing performance is measured and the processing time for the qualification campaign is estimated 2- IASI Level L1 (B or C) and L2 are ingested for the whole year 2009, in EPS format, up to 5Tb As a "qualification campaign", the KLIMA-IASI application is run upon the year 2009 data and results are used to validate the quality of the algorithm in relation to Gosat measurements; Also, during this phase processing performance is measured and the exploitation scenario defined 3- Full IASI Mission data and EumetCast NRT are systematically ingested in G-POD The KLIMA-IASI application performance is optimized in line with NRT requirements; The KLIMA-IASI application is enhanced to accept BUFR data format; KLIMA-IASI application is provided as a service to the User Community IPR: KLIMA-IASI algorithm developed and owned by the IFAC-CNR, Florence Evolution and time span: Several GPOD-MGVI versions integrated since 2011, bulk processing on dataset from 1 year IASI data (Mar 2010 – Feb 2011) Outcomes/Benefits: On-demand KLIMA-IASI service available to PI for pre-analysis Access to cloud resources for flexible capacity to meet the deadline Access to online data for bulk re-processing (11 out of 12 months processed) Several scientific papers published and presentations at conferences Process speeded-up by 2 orders of magnitude (flexible)

Success Story: SMOS SMOS testbed Objective: Provide a flexible test environment to support ESA GQ users for L1 calibration and developers for L2 OS Evolution and time span: Several SMOS L1 and L2 OS re-processing campaigns with custom configuration (recently processed sample SMOS L1 products over 25 months (from June 2010 to June 2012, 1728 L1A/B + 6576 CRS products), SMOS L2 OS 1 month) Outcomes/Benefits: Fast integration of new versions (if ICD unchanged) Access to online data for bulk re-processing Access to flexible cloud resources for meeting deadlines On-demand SMOS L1 available for GQ On-demand SMOS L2 OS available for developer SMOS L0 NRT ingestion chain set-up for L1 NRT custom re-processing Testbeds required for other sensors as well Open call for proposal published on EOPI

Future directions In the light of the RSS success story the following should be considered RSS model as basis for the definition of a harmonized model of research and service support Federation of payload data ground segment services to the scientific community

Contacts & further information Giancarlo Rivolta giancarlo.rivolta@esa.int RSS Team rss_team@esa.int GPOD Team eo-gpod@esa.int G-POD Portal gpod.eo.esa.int Join&Share wiki.services.eoportal.org

GPOD demo Now the GPOD Team will give a 15-minute GPOD demo GPOD Portal: gpod.eo.esa.int

Questions and Answers Q&A?

Thank you Giancarlo Rivolta Logica xxx Visitor address Street Postcode City Country www.logica.com Contact: Name Function T: +XX (0) 000 00 000 000 F: +XX (0) 000 00 000 000 E: address@logica.com