EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarksEGEE-III INFSO-RI Astro-Wise and EGEE Fokke Dijkstra Donald Smits Center for Information Technology University of Groningen, NL In collaboration with OmegaCEN, Kapteyn institute, University of Groningen, NL 23 September - EGEE conference Istanbul

Enabling Grids for E-sciencE EGEE-III INFSO-RI Introduction

Enabling Grids for E-sciencE EGEE-III INFSO-RI Astro-Wise setup

Enabling Grids for E-sciencE EGEE-III INFSO-RI TARGET diagram Extreme data lineage!

Enabling Grids for E-sciencE EGEE-III INFSO-RI Astro-WISE: Services

Enabling Grids for E-sciencE EGEE-III INFSO-RI Lofar in short Large distributed radio telescope In NL 2008: initial roll out 20 stations –Full scale aperture synthesis array, extends 100 km Fibre network, Software Correlator Run as a broad common-user observatory –Making the transition now from “project” to “observatory” Extended with European partners Up to 4 Petabytes/year to archive

Enabling Grids for E-sciencE EGEE-III INFSO-RI Grid user EGEE Blue Gene Tier 0 Nearline storage & processing Tier 1 New Tier 1 location Jülich - Germany Amsterdam CIT / Tape Oracle DB Export by file Meta data and Files Tier 2 Astro-WISE view CIT – Groningen Lofar Communities SAS Archive Code base Euro VO Astro-WISE Communities - Groningen - Amsterdam - Leiden - Nijmegen - Jülich Global FS view LoWise C=IT/O=INFN /L=CNAF /CN=Pinco Palla /CN=proxy VOMS pseudo- cert C=IT/O=INFN /L=CNAF /CN=Pinco Palla /CN=proxy VOMS pseudo- cert

Enabling Grids for E-sciencE EGEE-III INFSO-RI Astro-Wise and Grid User authentication –VOMS proxies –Proxy server –Preferably client installation  Python only!  Web portal (should work) –Astro-Wise system will obtain delegated proxy –Extend to Oracle database Job submission –Manage collection of tasks Storage –Support SRM

Enabling Grids for E-sciencE EGEE-III INFSO-RI Astro-Wise job submission Astro-Wise Database Astro-Wise user tasks DPUProxy server credentials tasks connects gLite UI tasks Linux cluster VOMS server VOMS attributes connects

Enabling Grids for E-sciencE EGEE-III INFSO-RI Problems encountered Preferrably no gLite-UI installation for all users Currently only myproxy-init really necessary –Python implementation needed  OMII-UK (NDG) seems to have one, earlier tests unsuccessful –Other option java webstart version  CCLRC version works

Enabling Grids for E-sciencE EGEE-III INFSO-RI Job submission Image processing inherently parallel –e.g. Processing per single CCD image –One step requires global communication  Communication through database MPI job type used for processing –No real MPI program used –Independent processes started using ssh AdvantagesDisadvantages Reduces the amount of data that is shipped around Single job, easier management No problem with varying system speeds MPI support limited Shared directory necessary (currently) Depends on password-less ssh between WNs

Enabling Grids for E-sciencE EGEE-III INFSO-RI Future plans Investigate current state of python myproxy implementation as used by NDG Use of SRM for Lofar storage (StoRM on global file system) Further integration of GSI with Astro-Wise and Oracle Allow Astro-Wise to store data through SRM

Enabling Grids for E-sciencE EGEE-III INFSO-RI Final remarks Grid job submission relatively easy Credential management stumble block for users –Obtaining certificate –Managing (proxy) certificates –Renewing certificates Lack of portable tools –Cannot ship current gLite UI to everyone –UI portals may be a solution Advantages of Grid –Standards!  Job submission  Data management  Security

Enabling Grids for E-sciencE EGEE-III INFSO-RI Acknowledgments Kor Begeman (implementation in Astro-Wise) Willem Jan Vriend (implementation in Astro-Wise) Danny Boxhoorn Andrey Belikov All from:OmegaCEN Kapteyn Institute University of Groningen, NL