1. Enabling Grids for E-sciencE INFSO-RI-508833 Dr. Rüdiger Berlich Forschungszentrum Karslruhe ruediger@berlich.de Introduction to Grid Computing Christopher Jung Forschungszentrum Karslruhe Christopher.jung@iwr.fzk.de Slides contributed by Rüdiger Berlich, Dave Berry and Christopher Jung

2. Enabling Grids for E-sciencE INFSO-RI-508833 Forschungszentrum Karlsruhe Institute for Scientific Computing (IWR) Part of the “Helmholtz Gemeinschaft“ One of the largest independent German research institutions Many different research areas ranging from environmental studies over nano technology to Grid Computing

3. Enabling Grids for E-sciencE INFSO-RI-508833 The GridKa Cluster (1)

4. Enabling Grids for E-sciencE INFSO-RI-508833 The GridKa Cluster (2) Status and future of ressources 12 %475375280Tape [TB] 50 % 1010*2Internet [Gb/s] 18 %270220160Disc [TB] 12 %1290920580Compute Power / kSI2k 30 %1.2801.070680Processors % of 2008Apr 2005Okt 2004Apr 2004 As of 10/2004: largest Linux cluster in the German science community largest online storage of a single installation in Germany fastest Internet connection in Germany part of a Grid with ca. 80 other European installations routing (full 10 Gbps): GridKa – DFN (Karlsruhe) – DFN (Frankfurt) – Géant (Frankfurt) – Géant (Paris) – Géant (Genf) – CERN * still being tested What for ?

5. Enabling Grids for E-sciencE INFSO-RI-508833 Usage of GridKa Cluster January-December 2003 2004 Prozessor Usage [h]1.980.0004.182.000 Number of Jobs 328.0001.442.000 LHC 34% non-LHC 66%

6. Enabling Grids for E-sciencE INFSO-RI-508833 LHC / CMS In LHC: Expect data rates of 1 Petabyte (??) per experiment per year. But: trivial to run in parallel...

7. Enabling Grids for E-sciencE INFSO-RI-508833 Data acquisition at CMS 100 KHz (150 GB/sec) 150 Hz (225 MB/sec) High Level Trigger – PCs Data  40MHz (  60TB/sec) 1 event:  1.5 MB data recording Online-System Offline-Analysis Level 1 Trigger – special hardware multi-level trigger to: filter out uninteresting events reduce data volume

8. Enabling Grids for E-sciencE INFSO-RI-508833 Distributed Collaborations 70 53 8 27 460 3 637 55 22 87 10 Europe: 267 Institutes, 4603 Users Other: 208 Institutes, 1632 Users Over 6000 LHC Scientists worldwide Want transparent and quick access (very rightly so). Interested more in physics results, than computing revolutions

9. Enabling Grids for E-sciencE INFSO-RI-508833 The LHC Computing Grid LCG helps the experiments’ computing projects Phase 1 – 2002-05 prepare and deploy the environment for LHC computing Phase 2 – 2006-08 acquire, build and operate the LHC computing service SC2 – Software & Computing Committee SC2 includes the four experiments, Tier 1 Regional Centres SC2 identifies common solutions and sets requirements for the project PEB – Project Execution Board PEB manages the implementation organising projects, work packages coordinating between the Regional Centres

10. Enabling Grids for E-sciencE INFSO-RI-508833 The MONARC* study and Tier-1 centers Basic idea: hierarchical distribution of tasks Idea accepted by the LHC Computing Grid (responsible for planning and management of LHC computing) Tier-0: Initial reconstruction and storage of raw events, distribution to Tier-1 Tier-1: Data-heavy analysis, reprocessing of data, regional support Tier-2: Managed disk storage, simulation of PP events, computing * MONARC == Models Of Networked Analysis at Regional Centers Today: “The Grid”

11. Enabling Grids for E-sciencE INFSO-RI-508833 Distributed Computing and PP Distributed computing and particle physics go well together, because: PP analysis is trivial to parallelize (just run each job on a separate machine and collect the results) PP collaborations are distributed by design, as modern experiments usually cannot be financed by a single country anymore Distributed resources already exist and can be used to lower the cost of new experiments physicists in general are willing to set up a production system for distributed computing, not only interest in theory of computing Governments like to spend their money locally (billion dollar investments...)

12. Enabling Grids for E-sciencE INFSO-RI-508833 Requirements Need: transparent access to data replication, virtualisation, global filesystems,... secure storage, authentication and authorisation access control (Unix...), PKI infrastructure, CA, agreed policies, VO accounting (computing costs money) not really solved training, support GGUS, EGEE Workpackages fast networks (low latency, high bandwidth) Geant, DFN,.... Need: (a) software layer “middleware”, (b) fast networks, (c) common policies and (d) services

13. Enabling Grids for E-sciencE INFSO-RI-508833 The Grid: Definition A Virtual Organisation is: People from different institutions working to solve a common goal Sharing distributed processing and data resources Not too different from Unix rights management (access control) “Grid computing is coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations” (I.Foster) Genealogy: The term “Grid Computing” comes from the analogy to the electrical power grid - “computing power from a plug in the wall”

14. Enabling Grids for E-sciencE INFSO-RI-508833 Distributed applications – today and tomorrow Existing distributed applications: –tend to be specialised systems –intended for a single purpose or user group Grids go further and take into account: –Different kinds of resources  Not always the same hardware, data and applications –Different kinds of interactions  User groups or applications want to interact with Grids in different ways –Dynamic nature  Resources and users added/removed/changed frequently

15. Enabling Grids for E-sciencE INFSO-RI-508833 The Grid? There is not only one grid! Many grid initiatives Different requirements of different sciences (physics, bioinformatics, meteorology, disaster management,...) and industry Most important ‘grids’ in High Energy Physics: –LCG –NorduGrid –Grid 3 –SAM Those are incompatible (to at least a high degree)

16. Enabling Grids for E-sciencE INFSO-RI-508833 Grid Projects http://www.cordis.lu/ist/grids/project s.htm Many brilliant people with many brilliant (but incompatible) ideas

17. Enabling Grids for E-sciencE INFSO-RI-508833 The diverse world of grid computing Just to illustrate the diversities of ideas in grid computing, I‘ll show you an excerpt of program items at last year‘s „International Summer School of Grid Computing“ Grids, Middleware and Application DAGMAN, Condor-G and Stork Web Services Community Grids Labroatory Grid Portals Application Grids Boat trip to Amalfi (okay, no grid computing) Workflow OGSA-DAI Unicore Commercial Grids ….

18. Enabling Grids for E-sciencE INFSO-RI-508833 The world of LCG … and still growing

19. Enabling Grids for E-sciencE INFSO-RI-508833 Main misunderstandings There are some general misunderstadings about grid computing: Computing power and disk/tape space for free Grid software is easy and fast to install Users will jump at it as soon as they can My favorite Linux flavor is XYZ; for sure it will be easy to install grid software on it I can install all basic services on just one machine Documentation is great Only small amount of man-power is needed for grid administration

20. Enabling Grids for E-sciencE INFSO-RI-508833 „Outlook“: Why do grid computing now? Grid computing is quite easy for the user (you will see this later) At the moment the grid is everything but crowded by users, so there is much computing power available (and much more will become available by 2007) Grid developers need feedback to do further improvements on software Today‘s experiments can already profit from grid computing Who wants to spend time in 2007 on learning grid computing, if there is interesting physics to be done?

21. Enabling Grids for E-sciencE INFSO-RI-508833 Questions? Thanks to the German Federal Ministry of Education and Research, BMB+F, as well as Forschungszentrum Karlsruhe / Germany for their continuous interest and support !