Earth Science GRID on Demand & e-collaboration The ESRIN GRID team - Grid and e-collaboration for the Space Community ESA-ESRIN, 2 February 2005

Summary Earth Science and GRID Earth science and e-collaboration Present infrastructure at ESRIN Medium term vision Example of applications

GRID for Earth Science means: –Single stop shop for data access –Number-crunching, e.g EO data reprocessing –e-collaboration, e.g. access and sharing of data sources, tools, means, models, algorithms … Earth Science and Grid May 2004 June 2004 July 2004 August 2004September 2004 October 2004 MERIS global composite > 1 Terabyte data processed – 1 Km res.

e-collaboration technologies Earth Science community analysis The Internet and improved communication Web-Services Grid Semantic Technologies and Ontologies Sensor Web Wireless Technologies Workflow Management Technologies Virtual Reality or Augmented Reality Standards (OGC, W3C, …)

Part of operational ESA EO environment SOAP MUIS ESA Catalogue AMS ESA Data Archive ESA EO operational environment WCS WFS / WMS Catalogue (CSS) Reference infrastructure GLOBUS+? Computing Elements Storage Elements Local Grid DATAGRID Computing Elements Storage Elements European Grid Client Applications (JAVA / Linux / Windows) Web Portal Client Applications OGC Web Services GRID Engine LCG + ? Computing Elements Storage Elements ESA EO GRID MEGALAB LCG + ? Computing Elements Storage Elements EGEE Computing Elements Storage Elements European Grid ESA EO planned operational environment geant GRID & EO Web ESA

ESA actions and 2y vision Mid term vision – 2 years –Target: mid –2007 –EO missions Ground Segment operations Satellites data routine acquisition, processing, archiving, dissemination –Earth Science User environment Science and commercial applications

EO Grid infrastructure G/S vision Transfer to operation of present GRID capabilities: –Support large scale reprocessing requirements Demonstrated already for L1 L2 GOMOS Planned for GOMOS / MERIS in 2005 New EO processors based on Linux –Enable new classes of large-scale applications (e.g. processing of global MERIS Level 3) –Innovative solution for the data access and exchange –Increased exploitation of available resources GRID interconnection of ESA and other operational facilities –Reference infrastructure for new missions G/S studies –Take advantage of EC e-infrastructure initiative

Earth Science User Segment Vision Set up of the EO Science Grid User Segment –Include support to science data processing / analysis based on Envisat tools (e.g. BEAM) –Platform for e-collaboration in Earth Science (e.g. multi data source CAL/VAL…) Goal 1: Move algorithms and analysis tools to data –data assimilation … integration in models… –Support applications within the ENVISAT AO, e.g. MERIS data assimilation for chlorophyll forecast… Goal 2: extend approach to wide science user community –Wish to define a dedicated AO for next year –Contribute to the definition of GMES User segment

Status of applications in Grid environment Ready for operations –On Demand: Ozone GOME validation, Global and regional MERIS L3 products (MGVI, Chlorophyll …) –Validation of MGVI with Science users Under development in house –Preparation of ENVISAT (GOMOS, MERIS ) reprocessing –ASAR Global Monitoring Antarctica (sea ice, icebergs, …) –AATSR fire detection and SST; RA2 Lake and river altimetry –Direct access to NASA archives (CEOS) Under development with external partners –e-collaboration for ozone (GOMOS, SCIAMACHY, MIPAS, …) –Objective analysis SST and chlorophyll in the Med (AATSR, MODIS, MERIS…) –MERIS applications: case 2 waters, data assimilation for chlorophyll forecast, … –CNR – integration of MSG for fire detection; –Uni Tor Vergata – reprocessing of ERS-2 SAR for oil spill statistics

Portal Grid Services Job selection, chaining, launching and live status Direct result visualization in WMS (using open source quickWMS client) Join personal data folder as a WMS

Chlorophyll BEAM processing May 2004June 2004July 2004 Input Data: Gb in 862 files Output Files: 700 Kb (JPEG) 28 Mb (TIFF) 36 Mb (DIM-MAP) Input Data: Gb in 1643 files Output Files: 700 Kb (JPEG) 28 Mb (TIFF) 36 Mb (DIM-MAP) Input Data: Gb in 1828 files Output Files: 700 Kb (JPEG) 28 Mb (TIFF) 36 Mb (DIM-MAP)

Task chaining in the Science Grid on Demand environment. Based on L2 products COLLECT FUNCTION Searches and retrieves the requested files from the storage elements BINNING FUNCTION Projects the Envisat files to a given datum and projection and according to the requested geographical bounding box AGGREGATION FUNCTION Assembles the resulting files from the binning processes Earth Observation Visualization Function Generic functions for image composition and format conversion (JPEG/TIFF/…) DATA REGISTRY FUNCTION Registries the data files and the corresponding metadata in the defined storage element RETRIVE FUNCTION Generic function for file transfer (FTP, SFTP, gridFTP,…) Chlorophyll processing on GRID BEAM code is embedded here

Alternate MERIS MGVI algorithms Arithmetic Mean Algorithm fapar (JRC) Algorithm Testing different temporal aggregation methods for MERIS MGVI

MERIS ASAR Global mode mapping May 04 June 04 July 04 Extension of Sea Ice Pack Displacement of icebergs

Participation to external projects GRID – EC funded: –Participation to EGEE – NA4 –DILIGENT– Digital Libraries in GRID environt started in Sept 04 ESA in charge of Environmental User scenario Very interesting technology for Long Term Data and Knowledge Preservation e-collaboration – EC funded: –BRAINBRIDGE – to KO in Mar 05 define road map for e-collaboration initiative in FP7 Infrastructure – Regional funding: –MEGALAB GRID infrastructure across Frascati-Tor Vergata- Rome area

Grazie!