INFSO-RI-508833 User Forum 1-3 March 2006 Enabling Grids for E-sciencE www.eu-egee.org Worldwide ozone distribution by using Grid infrastructure ESA: L.

Slides:

Advertisements

Similar presentations

ESA Data Integration Application Open Grid Services for Earth Observation Luigi Fusco, Pedro Gonçalves.

Advertisements

European Commission Directorate-General Information Society Unit F2 – Grid Technologies INSERT PROJECT ACRONYM HERE BY EDITING THE MASTER SLIDE (VIEW /

DataGrid Kimmo Soikkeli Ilkka Sormunen. What is DataGrid? DataGrid is a project that aims to enable access to geographically distributed computing power.

Using The EDG Testbed The European DataGrid Project Team

INFSO-RI Enabling Grids for E-sciencE ES applications in EGEEII – M. Petitdidier –11 February 2008 Earth Science Application Overview.

EU 2nd Year Review – Jan – WP9 WP9 Earth Observation Applications Demonstration Pedro Goncalves :

DataGrid is a project funded by the European Commission under contract IST Earth Observation applications in EU DataGrid (EDG) and follow on.

Catania, 9-11 January M Earth Science Activity David Weissenbach (IPSL) EGEE is funded by.

INFSO-RI Enabling Grids for E-sciencE FloodGrid application Ladislav Hluchy, Viet D. Tran Institute of Informatics, SAS Slovakia.

EDG Application The European DataGrid Project Team

KNMI Applications on Testbed 1 …and other activities.

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks The EGEE Infrastructure and Remote Instruments.

ES Metadata Management Enabling Grids for E-sciencE ES metadata OGSA-DAI NA4 GA Meeting, D. Weissenbach, IPSL, France.

1 Satellite data assimilation for air quality forecast 10/10/2006.

INFSO-RI Enabling Grids for E-sciencE Project Gridification: the UNOSAT experience Patricia Méndez Lorenzo CERN (IT-PSS/ED) CERN,

INFSO-RI Enabling Grids for E-sciencE ES applications in EGEEII – M. Petitdidier –11 February 2008 Earth Science session Wrap up.

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks LOFAR Archive Information System Kor Begeman.

INFSO-RI Enabling Grids for E-sciencE Supporting legacy code applications on EGEE VOs by GEMLCA and the P-GRADE portal P. Kacsuk*,

WP9 – Earth Observation Applications – n° 1 WP9 report to Plenary ESA, KNMI, IPSL Presented by M. Petitdidier, IPSL DataGrid Plenary Session 5 th Project.

EU 2nd Year Review – Jan – WP9 WP9 Earth Observation Applications

Earth Observation Virtual Organisation applications: a proposal to EGEE EGEE NA4 Generic Application.

ESA DataGrid Review 1. Welcome and Introduction Frascati, 10 June 2002.

Enabling Grids for E-sciencE EGEE-III INFSO-RI I. AMGA Overview What is AMGA Metadata Catalogue of EGEE’s gLite 3.1 Middleware Main Feature of.

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks INTRODUCTION Monique Petitdidier & Horst.

INFSO-RI Enabling Grids for E-sciencE SA1 and gLite: Test, Certification and Pre-production Nick Thackray SA1, CERN.

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks NA2 – Dissemination, Outreach and Communication.

NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.

INFSO-RI Enabling Grids for E-sciencE OGSA DAI Data Access and Integration Marek Ciglan Institute of Informatics, Slovac Academy.

WP9 – Earth Observation Applications – n° 1 Planning for EO Application Deployment DataGrid WP9 EO Applications Parallel Session 5 th Project Conference.

Replica Management Services in the European DataGrid Project Work Package 2 European DataGrid.

Sciamachy features and usage with respect to end-users The typical fate of retrieval people dealing with large datasets… C. Frankenberg, SRON team, IUP.

INFSO-RI Enabling Grids for E-sciencE eGY Meeting Boulder – M. Petitdidier –March 2008 EGEE – Grid project M. Petitdidier (IPSL/CETP)

INFSO-RI Enabling Grids for E-sciencE SALUTE – Grid application for problems in quantum transport E. Atanassov, T. Gurov, A. Karaivanova,

EGEE-II INFSO-RI Enabling Grids for E-sciencE The GILDA training infrastructure.

INFSO-RI Enabling Grids for E-sciencE A service oriented framework to create, manage and update metadata for earth system science.

INSERT PROJECT ACRONYM HERE BY EDITING THE MASTER SLIDE (VIEW / MASTER / SLIDE MASTER) Monique Petitdidier IPSL/CETP CEOS/WGISS.

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Grid Web Portal for Chemists M. Sterzel,

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE’09 SSC Workshop SAFE Project Proposal.

INFSO-RI Enabling Grids for E-sciencE Intelligent Distributed Data Management in Earth System Science S. Kindermann, DKRZ, Germany.

EDG Applications The European DataGrid Project Team

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks The GILDA t-Infrastructure Roberto Barbera.

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGI Operations Tiziana Ferrari EGEE User.

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Going from EGEE-NA ES cluster to EGI SSC.

GENESI-DR: Ground European Network for Earth Science Interoperations - Digital Repositories Grant agreement no: EGEE 4 th.

EC Review – 01/03/2002 – WP9 – Earth Observation Applications – n° 1 WP9 Earth Observation Applications 1st Annual Review Report to the EU ESA, KNMI, IPSL,

Testing and integrating the WLCG/EGEE middleware in the LHC computing Simone Campana, Alessandro Di Girolamo, Elisa Lanciotti, Nicolò Magini, Patricia.

2. WP9 – Earth Observation Applications ESA DataGrid Review Frascati, 10 June Welcome and introduction (15m) 2.WP9 – Earth Observation Applications.

On the D4Science Approach Toward AquaMaps Richness Maps Generation Pasquale Pagano - CNR-ISTI Pedro Andrade.

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Evaluating Metadata access strategies with.

EGEE-II INFSO-RI Enabling Grids for E-sciencE Introduction to P-GRADE Portal hands-on Miklos Kozlovszky MTA SZTAKI

Open Grid Services for Earth Observation Pedro Gonçalves.

ESA DataGrid Review – 10 June 2002 – n° 1 Summary item 3  ESA and WP9 part 1 (45m)  DataGrid Frascati infrastructure (AT, 7.5m)  ESA GOME application.

Tecnologías Grid aplicadas a Ciencias de la Tierra Manuel López-Puertas (IAA, Granada) Page 1 1ª Reuni ó n e-CA, Granada, Junio 2007.

EGEE-II INFSO-RI Enabling Grids for E-sciencE Practical using WMProxy advanced job submission.

INFSO-RI Enabling Grids for E-sciencE gLite Test and Certification Effort Nick Thackray CERN.

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE: Enabling grids for E-Science Bob Jones.

Casablanca Septembre Earth Science Applications Monique Petitdidier (IPSL) Coordinator Earth Science in the Grid.

DataGrid is a project funded by the European Commission under contract IST rd EU Review – 19-20/02/2004 WP9 Earth Observation applications.

The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) gLite Grid Introduction Salma Saber Electronic.

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks GOCDB4 Gilles Mathieu, RAL-STFC, UK An introduction.

Applications and the Grid EDG CERN Ingo Augustin CERN DataGrid HEP Applications.

Earth Observation inputs to ATF Annalisa Terracina EU-DataGrid Project Work Package 9 – EO Applications April 2003 CERN.

DataGrid France 12 Feb – WP9 – n° 1 WP9 Earth Observation Applications.

INFSO-RI Enabling Grids for E-sciencE Earth Science Applications M. Petitdidier (CNRS/IPSL-Paris)

INFSO-RI Enabling Grids for E-sciencE ESR Database Access K. Ronneberger,DKRZ, Germany H. Schwichtenberg, SCAI, Germany S. Kindermann,

WP 9.4 Use Case (ESA-ESRIN, IPSL, KNMI)

EO Applications Parallel Session

Earth Observation and Climate Applications for EGEE

EDG Final Review Demonstration

Dissemination and Exploitation of GRids in Earth sciencE

Presentation transcript:

INFSO-RI User Forum 1-3 March 2006 Enabling Grids for E-sciencE Worldwide ozone distribution by using Grid infrastructure ESA: L. Fusco, J. Linford IPSL: C. Boonne, S. Godin-Beekmann, M. Petitdidier, D. Weissenbach KNMI: W. Som de Cerff SCAI: J. Kraus, H. Schwichtenberg UTV: F. DelFrate, M; Iapaolo

Enabling Grids for E-sciencE INFSO-RI User Forum 1-3 March 2006 CERN 2 INTRODUCTION Earth Observation by satellite: large community –Space Agencies –Scientists in many fields –Organisation in charge of natural risk alerts –SMEs for elaborated products GOME total ozone assimilation Grid Technology seems very well adapted to a lot of applications related to Earth Observations ESA/ESRIN, IPSL and KNMI have started to port applications in DataGrid Figure from ESRIN

Enabling Grids for E-sciencE INFSO-RI User Forum 1-3 March 2006 CERN 3 EARTH OBSERVATION BY SATELLITE [1] Typical satellite case: large number of files 7 years of Ozone from GOME/ERS2 experiment CHALLENGE PRODUCTION: Complex Algorithms: neural algorithm and inversion VALIDATION: For a given day find 10 profiles among

Enabling Grids for E-sciencE INFSO-RI User Forum 1-3 March 2006 CERN 4 ESA(IT) – KNMI(NL) Processing of raw GOME data to ozone profiles. 2 alternative algorithms ~28000 profiles/day IPSL(FR) Validate some of the GOME ozone profiles (~10 6 /y) Coincident in space and time with Ground-Based measurements Visualization & Analyze EGEE environment Level 2 (example of 1 day total O 3 ) Level 1 Raw satellite data from the GOME instrument (~75 GB - ~5000 orbits/y) The EGEE Ozone processing and validation test-case [2] Additional GRID demonstration: GOMOS, OMI, CEOS, …

Enabling Grids for E-sciencE INFSO-RI User Forum 1-3 March 2006 CERN 5 Ozone by SATELLITE [3] Production of Ozone profiles from satellite data GOME/ESR by UTV (Italie), ESRIN (Italie), KNMI(Hollande) Validation of the profiles with lidar data by IPSL, UTV NNO: Algorithm « Neural network » 2 versions (F. del Frate, M. Iapaolo, S. Casadio) 8 years ( ) on EGEE: files/algorithm with ~ 1800 profiles d’ozone/file Database: metadata taking into accountthe orbits footprint OPERA: Algorithm – Inversion (R. van derA et al., 2002) Several months onEGEE: Files 1 profil/file (W. Som de Cerff) Capability to run complex algorithms and to handle large amount of files

Enabling Grids for E-sciencE INFSO-RI User Forum 1-3 March 2006 CERN 6 Metadata In ES, Metadata and data bases are very common as large data sets are handled by different teams. The RDBMS generally used are MySQL, PostgreSQL or Oracle Many bases already exist and the aim is at least to make a copy of them. If they are developed to be used on EGEE they need to be useful outside Grid. Another specific point is the satellite data that are stored by orbit. The orbits are described by their footprint, expressed in geographical coordinates. Many queries concern collocation in time and/or space Some metadata and data are only accessible to authorized persons Others available on web site have rules for publications(acknowledgement, co-author)

Enabling Grids for E-sciencE INFSO-RI User Forum 1-3 March 2006 CERN 7 ES Metadata Management (SCAI; IPSL) Enabling Grids for E-sciencE ES meta data clients OGSA-DAI Service SE EGEE UI ES meta data client query lfns data X 509 User Proxy use ES meta data client EGEE Job on WN X 509 User Proxy submits lfns use

Enabling Grids for E-sciencE INFSO-RI User Forum 1-3 March 2006 CERN 8 ES Metadata Management Enabling Grids for E-sciencE What do we need? - RDBMS support OpenGIS conform data types What is PostGIS? - PostGIS adds support for geographic objects to Postgres: Why PostGIS? - MySQL spatial extensions only support convex polygons Database Aspects (Spatial Ext.) Relational DataBase Management System

Enabling Grids for E-sciencE INFSO-RI User Forum 1-3 March 2006 CERN 9 CONCLUSION In EGEE : Feasibility studies carried out on relatively simple applications Needs to pass from simple to complex applications and operational chain –for near-real time like daily previson of ozone. –Cascade of jobs –Inclusion of web services –………………. Some solutions in gLite Others investigations via SSA DEGREE(Dissemination and Exploitation of GRids in Earth SciencE)