Enabling Grids for E-sciencE INFSO-RI Dr. Rüdiger Berlich Forschungszentrum Karslruhe Introduction to Grid Computing Christopher Jung Forschungszentrum Karslruhe Slides contributed by Rüdiger Berlich, Dave Berry and Christopher Jung

Enabling Grids for E-sciencE INFSO-RI Forschungszentrum Karlsruhe Institute for Scientific Computing (IWR) Part of the “Helmholtz Gemeinschaft“ One of the largest independent German research institutions Many different research areas ranging from environmental studies over nano technology to Grid Computing

Enabling Grids for E-sciencE INFSO-RI The GridKa Cluster (1)

Enabling Grids for E-sciencE INFSO-RI The GridKa Cluster (2) Status and future of ressources 12 % Tape [TB] 50 % 1010*2Internet [Gb/s] 18 % Disc [TB] 12 % Compute Power / kSI2k 30 % Processors % of 2008Apr 2005Okt 2004Apr 2004 As of 10/2004: largest Linux cluster in the German science community largest online storage of a single installation in Germany fastest Internet connection in Germany part of a Grid with ca. 80 other European installations routing (full 10 Gbps): GridKa – DFN (Karlsruhe) – DFN (Frankfurt) – Géant (Frankfurt) – Géant (Paris) – Géant (Genf) – CERN * still being tested What for ?

Enabling Grids for E-sciencE INFSO-RI Usage of GridKa Cluster January-December Prozessor Usage [h] Number of Jobs LHC 34% non-LHC 66%

Enabling Grids for E-sciencE INFSO-RI LHC / CMS In LHC: Expect data rates of 1 Petabyte (??) per experiment per year. But: trivial to run in parallel...

Enabling Grids for E-sciencE INFSO-RI Data acquisition at CMS 100 KHz (150 GB/sec) 150 Hz (225 MB/sec) High Level Trigger – PCs Data  40MHz (  60TB/sec) 1 event:  1.5 MB data recording Online-System Offline-Analysis Level 1 Trigger – special hardware multi-level trigger to: filter out uninteresting events reduce data volume

Enabling Grids for E-sciencE INFSO-RI Distributed Collaborations Europe: 267 Institutes, 4603 Users Other: 208 Institutes, 1632 Users Over 6000 LHC Scientists worldwide Want transparent and quick access (very rightly so). Interested more in physics results, than computing revolutions

Enabling Grids for E-sciencE INFSO-RI The LHC Computing Grid LCG helps the experiments’ computing projects Phase 1 – prepare and deploy the environment for LHC computing Phase 2 – acquire, build and operate the LHC computing service SC2 – Software & Computing Committee SC2 includes the four experiments, Tier 1 Regional Centres SC2 identifies common solutions and sets requirements for the project PEB – Project Execution Board PEB manages the implementation organising projects, work packages coordinating between the Regional Centres

Enabling Grids for E-sciencE INFSO-RI The MONARC* study and Tier-1 centers Basic idea: hierarchical distribution of tasks Idea accepted by the LHC Computing Grid (responsible for planning and management of LHC computing) Tier-0: Initial reconstruction and storage of raw events, distribution to Tier-1 Tier-1: Data-heavy analysis, reprocessing of data, regional support Tier-2: Managed disk storage, simulation of PP events, computing * MONARC == Models Of Networked Analysis at Regional Centers Today: “The Grid”

Enabling Grids for E-sciencE INFSO-RI Distributed Computing and PP Distributed computing and particle physics go well together, because: PP analysis is trivial to parallelize (just run each job on a separate machine and collect the results) PP collaborations are distributed by design, as modern experiments usually cannot be financed by a single country anymore Distributed resources already exist and can be used to lower the cost of new experiments physicists in general are willing to set up a production system for distributed computing, not only interest in theory of computing Governments like to spend their money locally (billion dollar investments...)

Enabling Grids for E-sciencE INFSO-RI Requirements Need: transparent access to data replication, virtualisation, global filesystems,... secure storage, authentication and authorisation access control (Unix...), PKI infrastructure, CA, agreed policies, VO accounting (computing costs money) not really solved training, support GGUS, EGEE Workpackages fast networks (low latency, high bandwidth) Geant, DFN,.... Need: (a) software layer “middleware”, (b) fast networks, (c) common policies and (d) services

Enabling Grids for E-sciencE INFSO-RI The Grid: Definition A Virtual Organisation is: People from different institutions working to solve a common goal Sharing distributed processing and data resources Not too different from Unix rights management (access control) “Grid computing is coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations” (I.Foster) Genealogy: The term “Grid Computing” comes from the analogy to the electrical power grid - “computing power from a plug in the wall”

Enabling Grids for E-sciencE INFSO-RI Distributed applications – today and tomorrow Existing distributed applications: –tend to be specialised systems –intended for a single purpose or user group Grids go further and take into account: –Different kinds of resources  Not always the same hardware, data and applications –Different kinds of interactions  User groups or applications want to interact with Grids in different ways –Dynamic nature  Resources and users added/removed/changed frequently

Enabling Grids for E-sciencE INFSO-RI The Grid? There is not only one grid! Many grid initiatives Different requirements of different sciences (physics, bioinformatics, meteorology, disaster management,...) and industry Most important ‘grids’ in High Energy Physics: –LCG –NorduGrid –Grid 3 –SAM Those are incompatible (to at least a high degree)

Enabling Grids for E-sciencE INFSO-RI Grid Projects s.htm Many brilliant people with many brilliant (but incompatible) ideas

Enabling Grids for E-sciencE INFSO-RI The diverse world of grid computing Just to illustrate the diversities of ideas in grid computing, I‘ll show you an excerpt of program items at last year‘s „International Summer School of Grid Computing“ Grids, Middleware and Application DAGMAN, Condor-G and Stork Web Services Community Grids Labroatory Grid Portals Application Grids Boat trip to Amalfi (okay, no grid computing) Workflow OGSA-DAI Unicore Commercial Grids ….

Enabling Grids for E-sciencE INFSO-RI The world of LCG … and still growing

Enabling Grids for E-sciencE INFSO-RI Main misunderstandings There are some general misunderstadings about grid computing: Computing power and disk/tape space for free Grid software is easy and fast to install Users will jump at it as soon as they can My favorite Linux flavor is XYZ; for sure it will be easy to install grid software on it I can install all basic services on just one machine Documentation is great Only small amount of man-power is needed for grid administration

Enabling Grids for E-sciencE INFSO-RI „Outlook“: Why do grid computing now? Grid computing is quite easy for the user (you will see this later) At the moment the grid is everything but crowded by users, so there is much computing power available (and much more will become available by 2007) Grid developers need feedback to do further improvements on software Today‘s experiments can already profit from grid computing Who wants to spend time in 2007 on learning grid computing, if there is interesting physics to be done?

Enabling Grids for E-sciencE INFSO-RI Questions? Thanks to the German Federal Ministry of Education and Research, BMB+F, as well as Forschungszentrum Karlsruhe / Germany for their continuous interest and support !