1. INFSO-RI-508833 Enabling Grids for E-sciencE www.eu-egee.org iASTRO MC MEETING&WORKSHOP, 27-30, APRIL, 2005,SOFIA, BULGARIA Introduction to Grid Technologies in EGEE Emanouil Atanassov, Aneta Karaivanova and Todor Gurov Institute for Parallel Processing - BAS

2. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 2 Overview Evolvement of Grids What is Grid? Grid Services Goals of the EGEE project Building a production Grid for e-Science Grid applications in EGEE and SEE-GRID The Grid Challenges

3. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 3 Evolvement of Grids Historical perspective Local Computing –All computing resources at single site. –People move to resources to work. Remote Computing –Resources accessible from distance. –All significant resources still centralized. Distributed Computing –Resources geographically distributed. –Specialized access; largely data transfers. Grid Computing –Resources and services geographically distributed. –Standard interfaces; transfers of computations and data. Web Services and Grid Computing – Grid Services –Industry adopts Grid technology

4. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 4 What is GRID? “Coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations” (I.Foster) –Resources are controlled by their owners –The Grid infrastructure provides access to collaborators A Virtual Organization is: –People from different institutions working to solve a common goal –Sharing distributed processing and data resources Enabling People to Work Together on Challenging Projects –Science, Engineering, Medicine… - e-Science, e-Health –Public service, commerce… - e-Government, e-Business The Grid could be the “new age” Internet –‘[The Grid] intends to make access to computing power, scientific data repositories and experimental facilities as easy as the Web makes access to information.’, UK PM, 2002

5. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 5 The GRID vision On one hand: –Researchers/employees perform their activities regardless of geographical location, interact with colleagues, share and access data On the other hand: –Scientific instruments and experiments provide huge amount of data, incl. national databases And in the middle: –The Grid: networked data, processing centres and ”grid middleware” as the “glue” of resources.

6. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 6 Grid Services Basic unit of computation – job Basic unit of storage – file Information systems – BDII, Globus-mds, R-GMA, file catalogues, metadata catalogues Authorization, authentication, accounting (AAA)– based on PKI (Public key infrastructure) Every Grid site provides basic Grid services Advanced Grid Services: MPI jobs, Mass Storage Facilities accessed via SRM, Fine grained AAA (VOMS, DGAS).

7. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 7 Grid Services - schema

8. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 8 Grid Services in gLite

9. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 9 EGEE Partner Federations All work in EGEE will be carried out by the 70 partners grouped in 12 federations.

10. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 10 Goals of the EGEE project Goal in one sentence: –Allow scientists from multiple domains to use, share, and manage geographically distributed resources transparently. The EGEE project brings together experts from over 27 countries with the common aim of building on recent advances in Grid technology and developing a service Grid infrastructure, available to scientists 24 hours-a- day. The project aims to provide researchers in academia and industry with access to major computing resources, independent of their geographic location. The EGEE project will also focus on attracting a wide range of new users to the Grid.

11. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 11 Scientific disciplines to run Grid applications EGEE aims to establish production quality sustained Grid services –3000 users from at least 5 disciplines –integrate 50 sites into a common infrastructure –offer 5 Petabytes (10 15 ) storage Demonstrate a viable general process to bring other scientific communities on board Pilot New

12. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 12 EGEE – building a production Grid for e-Science Operations Management Centre (OMC): –At CERN – coordination etc Core Infrastructure Centres (CIC) –Manage daily grid operations – oversight, troubleshooting –Run essential infrastructure services –Provide 2 nd level support to ROCs –UK/I, Fr, It, CERN, + Russia (M12) –Taipei also run a CIC Regional Operations Centres (ROC) –Act as front-line support for user and operations issues –Provide local knowledge and adaptations –One in each region – many distributed User Support Centre (GGUS) –In FZK – manage PTS – provide single point of contact (service desk) –Not foreseen as such in TA, but need is clear

13. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 13 Components of a production Grid A production Grid consists of stable interoperating Grid sites (Resource centres), which enable access to Grid users from various Virtual Organizations Every Grid site provides basic Grid services and follows strict operational procedures. Monitoring allows fast detection of problems and their resolution or isolation.

14. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 14 BG01-IPP setup UI - PKI X.509 certificate keys - JDL files Terminals enter Grid enter Grid enter Grid enter Grid UI WN RB/II CE SE BDII

15. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 15 Structure of EGEE operations The grid is flat, but Hierarchy of responsibility –Essential to scale the operation CICs act as a single Operations Centre –Operational oversight (grid operator) responsibility –rotates weekly between CICs –Report problems to ROC/RC –ROC is responsible for ensuring problem is resolved –ROC oversees regional RCs ROCs responsible for organising the operations in a region –Coordinate deployment of middleware, etc CERN coordinates sites not associated with a ROC CIC RC ROC RC ROC RC ROC RC ROC OMC RC = Resource Centre

16. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 16 Operations monitoring maps In LCG-2: 137 sites, 34 countries >12,000 cpu ~5 PB storage Includes non-EGEE sites: 9 countries, 18 sites In LCG-2: 137 sites, 34 countries >12,000 cpu ~5 PB storage Includes non-EGEE sites: 9 countries, 18 sites

17. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 17 Selection of Monitoring tools GIIS MonitorGIIS Monitor graphs Sites Functional Tests GOC Data Base Scheduled Downtimes Live Job Monitor GridIce – VO viewGridIce – fabric viewCertificate Lifetime Monitor Note: Those thumbnails are links and are clickable.

18. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 18 Example: LHC at CERN

19. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 19 CMS LHC Experiment

20. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 20 Example biomedical app: gPTM3D One data set is –DICOM files: 100MB – 1GB –One radiological image: 20MB – 500MB Complex interface: optimized graphics and medically- oriented interactions Physician interaction is required at and inside all steps Poorly discriminant data, pathologies, medical windowing Interaction RenderExploreAnalyseInterpretAcquire

21. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 21 Figures Small body Medium body Large body Lungs Dataset 87MB 210MB 346MB 87MB Input data 3MB 18KB/slice 9.6 MB 25KB/slice 15MB 22KB/sclice 410KB 4KB/slice Output data 6MB 106KB/slice 57MB 151KB/slice 86MB 131KB/slice 2.3MB 24KB/slice Tasks 169 378 676 95 Standalone Execution 5min15s 1min54s 33min 11min5s 18min 36s EGEE Execution 14 procs. 37s 18s 2min30s 1min15s 2min03 24s

22. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 22 Example: The MAGIC Telescope Ground based Air Cerenkov Telescope Gamma ray: 30 GeV - TeV LaPalma, Canary Islands ( 28° North, 18° West ) 17 m diameter operation since autumn 2003 (still in commissioning) Collaborators: IFAE Barcelona, UAB Barcelona, Humboldt U. Berlin, UC Davis, U. Lodz, UC Madrid, MPI München, INFN / U. Padova, U. Potchefstrom, INFN / U. Siena, Tuorla Observatory, INFN / U. Udine, U. Würzburg, Yerevan Physics Inst., ETH Zürich Physics Goals: Origin of VHE Gamma rays Active Galactic Nuclei Supernova Remnants Unidentified EGRET sources Gamma Ray Burst

23. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 23 ~ 10 km Particle shower Ground based γ-ray astronomy ~ 1 o Cherenkov light ~ 120 m Gamma ray GLAST (~ 1 m 2 ) Cherenkov light Image of particle shower in telescope camera reconstruct: arrival direction, energy reject hadron background

24. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 24 MAGIC – Hadron rejection Based on extensive Monte Carlo Simulation –air shower simulation program CORSIKA –Simulation of hadronic background is very CPU consuming  to simulate the background of one night, 70 CPUs (P4 2GHz) needs to run 19200 days  to simulate the gamma events of one night for a Crab like source takes 288 days. –At higher energies (> 70 GeV) observations are possible already by On-Off method (This reduces the On-time by a factor of two) –Lowering the threshold of the MAGIC telescope requires new methods based on Monte Carlo Simulations

25. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 25 BG application in SEE-GRID VO - SALUTE The Problem: ultra-fast semiconductor carrier transport femtosecond relaxation of hot electrons by phonon emission in presence of electric field. Barker-Ferry equation and Monte Carlo approach Application in nanotechnologies: innovative results for GaAs: collision broadening and memory effects of quantum kinetic model; Intra-collision field effect: quantum scattering - retarding and accelerating field. “NP-hard” problem concerning the evolution time Parallel and Grid implementation

26. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 26 Wigner function 800 x 260 points 150 fs

27. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 27 Energy relaxation process: collisional broadening Accumulation From 10 fs up to 250 fs

28. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 28 BG application in ESR VO – air pollution prediction Under development by Tzvetan Ostromsky from IPP Transition from HPC to Grid computing

29. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 29 Challenges before new sites Install middleware and follow security and middleware upgrades in a timely fashion Present valuable resource to the Virtual Organizations that the site supports Participate in the various challenges. So far we have seen the HEP and the Biomed VO challenges, and the security challenges Participate in innovation efforts – development of middleware and/or grid applications Attract new users The Grid is about people

30. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Day Nis 31 Jan 2006 30 BG Grid support centre contact information Contact persons: Emanouil Atanassov, SA1 Activity Leader, emanouil@parallel.bas.bg Aneta Karaivanova, NA2 Activity Leader, anet@parallel.bas.bg Todor Gurov, Alternate EGEE SEE-ROC and SEE-GRID manager, gurov@parallel.bas.bg Ivan Dimov, EGEE & SEE-GRID Project manager for BG ivdimov@bas.bg http://www.grid.bas.bg/