1. EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE www.eu-egee.org EGEE and gLite are registered trademarksEGEE-III INFSO-RI-222667 Astro-Wise and EGEE Fokke Dijkstra Donald Smits Center for Information Technology University of Groningen, NL In collaboration with OmegaCEN, Kapteyn institute, University of Groningen, NL 23 September - EGEE conference Istanbul

2. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Introduction

3. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Astro-Wise setup

4. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 TARGET diagram Extreme data lineage!

5. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Astro-WISE: Services

6. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Lofar in short Large distributed radio telescope In NL 2008: initial roll out 20 stations –Full scale aperture synthesis array, extends 100 km Fibre network, Software Correlator Run as a broad common-user observatory –Making the transition now from “project” to “observatory” Extended with European partners Up to 4 Petabytes/year to archive

7. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Grid user EGEE Blue Gene Tier 0 Nearline storage & processing Tier 1 New Tier 1 location Jülich - Germany Amsterdam CIT / Tape Oracle DB Export by file Meta data and Files Tier 2 Astro-WISE view CIT – Groningen Lofar Communities SAS Archive Code base Euro VO Astro-WISE Communities - Groningen - Amsterdam - Leiden - Nijmegen - Jülich Global FS view LoWise C=IT/O=INFN /L=CNAF /CN=Pinco Palla /CN=proxy VOMS pseudo- cert C=IT/O=INFN /L=CNAF /CN=Pinco Palla /CN=proxy VOMS pseudo- cert

8. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Astro-Wise and Grid User authentication –VOMS proxies –Proxy server –Preferably client installation  Python only!  Web portal (should work) –Astro-Wise system will obtain delegated proxy –Extend to Oracle database Job submission –Manage collection of tasks Storage –Support SRM

9. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Astro-Wise job submission Astro-Wise Database Astro-Wise user tasks DPUProxy server credentials tasks connects gLite UI tasks Linux cluster VOMS server VOMS attributes connects

10. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Problems encountered Preferrably no gLite-UI installation for all users Currently only myproxy-init really necessary –Python implementation needed  OMII-UK (NDG) seems to have one, earlier tests unsuccessful –Other option java webstart version  CCLRC version works

11. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Job submission Image processing inherently parallel –e.g. Processing per single CCD image –One step requires global communication  Communication through database MPI job type used for processing –No real MPI program used –Independent processes started using ssh AdvantagesDisadvantages Reduces the amount of data that is shipped around Single job, easier management No problem with varying system speeds MPI support limited Shared directory necessary (currently) Depends on password-less ssh between WNs

12. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Future plans Investigate current state of python myproxy implementation as used by NDG Use of SRM for Lofar storage (StoRM on global file system) Further integration of GSI with Astro-Wise and Oracle Allow Astro-Wise to store data through SRM

13. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Final remarks Grid job submission relatively easy Credential management stumble block for users –Obtaining certificate –Managing (proxy) certificates –Renewing certificates Lack of portable tools –Cannot ship current gLite UI to everyone –UI portals may be a solution Advantages of Grid –Standards!  Job submission  Data management  Security

14. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Acknowledgments Kor Begeman (implementation in Astro-Wise) Willem Jan Vriend (implementation in Astro-Wise) Danny Boxhoorn Andrey Belikov All from:OmegaCEN Kapteyn Institute University of Groningen, NL