NorduGrid and LCG middleware comparison ARC vs LCG2 NorduGrid and LCG middleware comparison Oxana Smirnova Lund/ATLAS August 2, 2004, CERN

Setting the field NorduGrid LCG Name at birth Nordic Testbed for Wide Area Computing and Data Handling Large Hadron Collider Computing Grid When Started in spring 2001 Started in spring 2002 Who Nordic universities: ATLAS teams, others CERN: IT division and LHC experiments Members and associates Dozens Hundreds Why Provide researchers with adequate computing power by harnessing distributed resources using Grid technologies 2004-08-02

Main activity areas NorduGrid LCG Nordic countries Certificate Authority Grid middleware (Advanced Resource Connector, ARC) development Applications support (mainly ATLAS) Support of resources (those in the universities) User support Grid deployment (over the available resources) includes user and applications support Grid Middleware (took over from EDG, collaboration with EGEE ) Applications (GEANT4, POOL, generators,…) Distributed Analysis Fabric and networking (mainly CERN, regional centers) 2004-08-02

Basic systems and services (figures from the NG Monitor and GridIce) NorduGrid LCG Computing resources Standalone stations, Linux clusters, pools, other UN*X systems PBS, SGE, Condor,… All shared ~40 sites Linux (RH7.3, soon: Scientific Linux) clusters PBS, LSF Some shared ~70 sites Storage resources Disk storage ~30 TB Disk and tape storage ~40 TB Certification Resources and users must be certified by a CA authorized by the common Security Team Resource management Up to resource owners Centrally coordinated User management ~500 users Centralized, VO-based ? Users ? SWEGRID computers count towards both; deploy only ARC so far 2004-08-02

Relation to Middleware NorduGrid LCG Started deploying EDG middleware in 2001 Started deploying EDG middleware in 2002 Started developing own middleware (ARC) in February 2002 Reason: EDG m/w could not support ATLAS DC1 production Tailored EDG middleware (LCG1 and LCG2) Reason: EDG m/w could not support Data Challenges by LHC experiments Uses ARC for production Satisfactory performance for ATLAS DC2, CMS (in Estonia), all kinds of Nordic researchers Uses LCG2 for Data Challenges Not quite satisfactory performance so far Will continue developing ARC, no need to switch to anything else willing to offer ARC to LCG Will develop and deploy gLite, together with EGEE EGEE requires Nordic sites to run LCG2/gLite 2004-08-02

Middleware: general ARC LCG2 License GPL EDG Platforms GNU Linux (runs on RH*, SuSE*, Mandrake*, Debian3.0 etc), tests on Mac OS X (GT2 issues) RedHat 7.3, in future: Scientific Linux, Mac OS X Grid middleware Globus 2.4.3, own patches Globus 2.2.4, patches by VDT GT3 RLS EDG RLS GACL - GSOAP - (?) Condor-G EDG VOMS Core developers 6 ± 2 Some dozens ? 2004-08-02

ARChitecture Goal: no single point of failure 2004-08-02

ARC-Geography 2004-08-02

Components overview (1/2) ARC LCG2 Computing element Grid Manager: resides on a master node Interface to LRMS Input data stage-in (LFN resolution etc) Output data stage-out and eventual registration Job management (kill, restart, logging, clean up etc) Pluggable (e.g. accounting plug-in, benchmarking...) Provides info for infosystem (stateful) CE: resides on a master node Includes some data management tools (not integrated) Storage element « Simple »: stripped down computing element (gridftp server + GACL) « Smart »: Reliable file transfer Communication with various data indexing services Asynchronous data manipulation « Classic »: gridftp server « SE »: SRM-enabled 2004-08-02

Components overview (2/2) ARC LCG2 Worker node Not needed Resides at every WN; mostly data management functionality User Interface A lightweigt client Security: certificate request, proxy... Job submission: matchmaking, scheduling, upload Job management: monitor, kill, proxy renewal etc Grid file movement: copy/erase/register etc Central service Job submission: RB contact Job management: monitor, kill etc GIIS Part of the Computing Element service (« server » package) Separate service (BDII) Resource Broker Not needed (UI’s functionality) Job submission: matchmaking, scheduling, sandbox transfer, logging 2004-08-02

Information System: MDS ARC LCG2 LDAP tree Own Cluster-oriented Glue LDAP schema GRIS At each cluster and SE GIIS No data caching Strict hierarchical system (Grid - Country or an international body - National body - Cluster) Multi-rooted at top and for big countries Caching on Some hierarchy Top-level GIISes replaced by BDII Clients Broker/User Interface Monitor Broker 2004-08-02

No caching, only registrants lists ARC MDS details No caching, only registrants lists Cluster sub-tree 2004-08-02

Grid workload management ARC LCG2 Job description Globus RSL + extensions JDL: Condor classAds User Interface (UI) Local to user or shared, unlimited amount (unknown, often several per user) Typically shared, several per testbed Matchmaking, scheduling Decentralized (local to every client), integrated into UI, relies on Infosys Dedicated Resource Broker (RB), approx. one per VO, relies on infosys Job description transaction UI uploads files directly to chosen clusters Sandbox is transferred via RB Job monitoring, management Direct queries to clusters Queries via the submission RB 2004-08-02

Grid data management ARC LCG2 Replica services Replica Catalog (Globus + patches), RLS (Globus) RLS (EDG) Input data LFN or URL defined in job description, staged by the server, cached If LFN or GUID is defined in job description, the job is steered to the location; otherwise files must be moved by the job (hence necessary to install LCG2 on every worker node) Output data LFN or URL defined in job description, uploaded and registered by the server FN (+LFN +SE) specified in job description, uploaded by the server Automatic, bulk replication Not supported (yet?) 2004-08-02

User management ARC LCG2 VO membership Several options: LDAP servers VOMS Plain text lists (served by HTTP) VO-based functionality - Local user mapping Resource availability Tags publishing Databases Monitoring 2004-08-02

Other services ARC LCG2 Monitoring Based entirely on Information System, no special sensors Uses additional sensors apart of GRISes Bookkeeping Under development (user-friendly interface is missing), Web-service Part of the WMS Distribution Available via FTP and Web: nightly snapshots, weekly tags, stable releases (RPM, tgz) Available via anonymous CVS checkout, LCFG (RPM) Support Web site, support ticket system, Bugzilla, discussion mailing list Web site (!), support ticket system (?), Savannah (?), plenty of mailing lists 2004-08-02

Summary A major difference: ARC concentrates most job and data management functionality in the cluster front-end (the Grid Manager) LCG2 requires installation on each WN; ARC does not Another major difference: Information System Wrong/outdated/insufficient information is the most destructive thing on Earth Another difference: ARC is oriented towards particular persons and small teams, while LCG2 is for large organizations ARC has a lightweight client, LCG2 has none ARC user can monitor each job on the Web, LCG can not ARC is portable to many systems, LCG is not LCG has tape storage facilities, NorduGrid has none LCG has common policies, NorduGrid has none Plenty of other differences, but many commonalities as well Certification Absence of a reliable accounting system No fine-grained information access Ultimately, the common goal – to help them physicists to find the Higgs  2004-08-02