1. INFSO-RI-508833 User Forum 1-3 March 2006 Enabling Grids for E-sciencE www.eu-egee.org Worldwide ozone distribution by using Grid infrastructure ESA: L. Fusco, J. Linford IPSL: C. Boonne, S. Godin-Beekmann, M. Petitdidier, D. Weissenbach KNMI: W. Som de Cerff SCAI: J. Kraus, H. Schwichtenberg UTV: F. DelFrate, M; Iapaolo

2. Enabling Grids for E-sciencE INFSO-RI-508833 User Forum 1-3 March 2006 CERN 2 INTRODUCTION Earth Observation by satellite: large community –Space Agencies –Scientists in many fields –Organisation in charge of natural risk alerts –SMEs for elaborated products GOME total ozone assimilation Grid Technology seems very well adapted to a lot of applications related to Earth Observations ESA/ESRIN, IPSL and KNMI have started to port applications in DataGrid Figure from ESRIN

3. Enabling Grids for E-sciencE INFSO-RI-508833 User Forum 1-3 March 2006 CERN 3 EARTH OBSERVATION BY SATELLITE [1] Typical satellite case: large number of files 7 years of Ozone from GOME/ERS2 experiment CHALLENGE PRODUCTION: Complex Algorithms: neural algorithm and inversion VALIDATION: For a given day find 10 profiles among 26000.

4. Enabling Grids for E-sciencE INFSO-RI-508833 User Forum 1-3 March 2006 CERN 4 ESA(IT) – KNMI(NL) Processing of raw GOME data to ozone profiles. 2 alternative algorithms ~28000 profiles/day IPSL(FR) Validate some of the GOME ozone profiles (~10 6 /y) Coincident in space and time with Ground-Based measurements Visualization & Analyze EGEE environment Level 2 (example of 1 day total O 3 ) Level 1 Raw satellite data from the GOME instrument (~75 GB - ~5000 orbits/y) The EGEE Ozone processing and validation test-case [2] Additional GRID demonstration: GOMOS, OMI, CEOS, …

5. Enabling Grids for E-sciencE INFSO-RI-508833 User Forum 1-3 March 2006 CERN 5 Ozone by SATELLITE [3] Production of Ozone profiles from satellite data GOME/ESR by UTV (Italie), ESRIN (Italie), KNMI(Hollande) Validation of the profiles with lidar data by IPSL, UTV NNO: Algorithm « Neural network » 2 versions (F. del Frate, M. Iapaolo, S. Casadio) 8 years (1995-2003) on EGEE: 38489 files/algorithm with ~ 1800 profiles d’ozone/file Database: metadata taking into accountthe orbits footprint OPERA: Algorithm – Inversion (R. van derA et al., 2002) Several months onEGEE: 76000 Files 1 profil/file (W. Som de Cerff) Capability to run complex algorithms and to handle large amount of files

6. Enabling Grids for E-sciencE INFSO-RI-508833 User Forum 1-3 March 2006 CERN 6 Metadata In ES, Metadata and data bases are very common as large data sets are handled by different teams. The RDBMS generally used are MySQL, PostgreSQL or Oracle Many bases already exist and the aim is at least to make a copy of them. If they are developed to be used on EGEE they need to be useful outside Grid. Another specific point is the satellite data that are stored by orbit. The orbits are described by their footprint, expressed in geographical coordinates. Many queries concern collocation in time and/or space Some metadata and data are only accessible to authorized persons Others available on web site have rules for publications(acknowledgement, co-author)

7. Enabling Grids for E-sciencE INFSO-RI-508833 User Forum 1-3 March 2006 CERN 7 http://www.scai.fraunhofer.de ES Metadata Management (SCAI; IPSL) Enabling Grids for E-sciencE ES meta data clients OGSA-DAI Service SE EGEE UI ES meta data client query lfns data X 509 User Proxy use ES meta data client EGEE Job on WN X 509 User Proxy submits lfns use

8. Enabling Grids for E-sciencE INFSO-RI-508833 User Forum 1-3 March 2006 CERN 8 http://www.scai.fraunhofer.de ES Metadata Management Enabling Grids for E-sciencE What do we need? - RDBMS support OpenGIS conform data types What is PostGIS? - PostGIS adds support for geographic objects to Postgres: http://postgis.refractions.net/ Why PostGIS? - MySQL spatial extensions only support convex polygons Database Aspects (Spatial Ext.) Relational DataBase Management System

9. Enabling Grids for E-sciencE INFSO-RI-508833 User Forum 1-3 March 2006 CERN 9 CONCLUSION In EGEE : Feasibility studies carried out on relatively simple applications Needs to pass from simple to complex applications and operational chain –for near-real time like daily previson of ozone. –Cascade of jobs –Inclusion of web services –………………. Some solutions in gLite Others investigations via SSA DEGREE(Dissemination and Exploitation of GRids in Earth SciencE)