1. LHCb Applications and GRID Integration Domenico Galli Catania, April 9, 2001 1 st INFN-GRID Workshop

2. LHCb Applications and GRID Integration. 2 Domenico Galli Status of LHCb-Italy Computing LHCb-Italy is now building its first computing farm. Already purchased and installed at CNAF: Ethernet switch. 1 TB RAID-5 Network Attached Storage. 1 bi-processor rack-mounted PC box. Purchased but still not installed: Standard rack. Remote controlled (via IP) power switches. In course of purchasing: 8 bi-processor rack-mounted PC boxes. Planned for 2001. 16 more bi-processor rack-mounted PC boxes. Goal: set up as soon as possible a working computing farm for MC production: Cheap (commodity components wherever possible). Reliable (network boot, diskless, RAM errors debug). Able to work unattended (monitoring, remote control).

3. LHCb Applications and GRID Integration. 3 Domenico Galli LHCb Short Term Use Case: Distributed MC Production At present MC production is performed at CERN, RAL, Liverpool, and Lyon. It is planned to extend this to Nikhef and INFN/Bologna in 2001 and to Glasgow/Edinburgh in 2002. Current MC code (SICBMC) and reconstruction code (SICBDST) are Fortran based. Events are written out in the form of Zebra banks. A (smooth) transition to OO software (Brunel, Gaudi) is foreseen, but this should be transparent to the usage of the GRID. The present distributed MC production system doesn’t use any GRID facilities. LHCb proposes to convert this system to GRID technology in a gradual manner.

4. LHCb Applications and GRID Integration. 4 Domenico Galli 1. Production started by filling out a Web form:  Version of software  Acceptance cuts  Detector database  Physics channel  Number of events to generate  Number of jobs to run  Centre where the jobs should run 2. Web form calls a java servlet that:  Creates a job script (one per job).  Creates a card file (one-three per job) with random number seeds and job options (The card files need to be accessible by the running job).  Issues a job submit command to run script in batch  WP1. 3. Script does the following:  Copies executable, detector database and card files  Executes executable  Executable creates output dataset  Output copied to local mass store  WP5  Log file copied to web browsable area  Script calls java program (see 4) 4. Java program calls servlet at CERN to:  Transfer data back to CERN  WP2  Update book-keeping meta-database at CERN (currently Oracle)  WP2 LHCb Use Case for GRID (MC Production)

5. LHCb Applications and GRID Integration. 5 Domenico Galli Main issue: We are forced to copy all data back to CERN. Reasons for this: Standard cataloguing tools do not exist — so we cannot keep track of the data where they are produced. Absence of smart analysis job-submission tools that move executables to the centre where the input data are stored. Steps that make the production difficult: Authorisation (jobs can be submitted only from trusted machines). Copy data (generated both inside & outside CERN) into the (CERN) mass store (many fragile steps). Problems of Current Production System

6. LHCb Applications and GRID Integration. 6 Domenico Galli Where can the GRID help?  Very transparent way of authorizing users on remote computers.  Data set cataloguing tools to avoid unnecessary replication.  If replication is required, provide fast and reliable tools.  Analysis job submission tools that interrogate the data set catalogue and specify where the job should be run (the executable may need to be sent to the data).

7. LHCb Applications and GRID Integration. 7 Domenico Galli Test-bed 1 Requirements and Month 9 Deliverables LHCb is deliberately conservative in its hopes for M9 to increase the chances of success. For Test-bed 1 (September 2001) LHCb intends to test: The basic GRID job submission facilities. The GRID file copier facilities. LHCb will be happy to test intermediate version of middleware before September, if they are available. That will avoid a big discontinuity taking a lot of a new software at the same time. LHCb support the decoupling of M9 deliverables from the long term architecture planning.

8. LHCb Applications and GRID Integration. 8 Domenico Galli LHCb Requirements for Short Term Use Case  GRID software must be callable from (in order of priority): Linux command prompt. Scripts (shell, perl, python). Java programs. C++ programs. A single certificate should be necessary to utilise the entire testbed. Jobs can run for a great length of time (several weeks). We need lasting security tokens for all resources that the job may access during this time. 12 hours is not enough.

9. LHCb Applications and GRID Integration. 9 Domenico Galli LHCb Requirements for Short Term Use Case (II) The middleware giving us access to mass storage should be robust enough to handle many tens of simultaneous requests. In case of error the system should interact with the batch facility and automatically put jobs in hold until the problem is fixed. The identification of files on the mass store should be possible via a hierarchical path and filenames of arbitrary length. There should be a common API used to read and write data at every site. In addition, there should be a common set of commands to move data locally and over the network.

10. LHCb Applications and GRID Integration. 10 Domenico Galli LHCb Requirements for Short Term Use Case (III) A standard mechanism is required for transferring/replicating datasets over the grid employing the fastest possible techniques (e.g. parallel transfers). Any data transfer mechanism should be robust and have the ability to recover from errors and transmission breaks. (Not strictly GRID but coupled with GRID infrastructure) A single public Web server with the following properties: Servlets are allowed to run. Write access for servlets to a disk area (visible by a running job, and web-browsable) to store scripts, card files and executables.

11. LHCb Applications and GRID Integration. 11 Domenico Galli Longer Term Requirements The meta data base should be distributed (i.e. as soon as it is updated somewhere, the whole GRID knows about it). It should be integrated with the job submission system so that when an input dataset is required, the job submission system sends the script to where the data resides. It should have similar performance and accessibility as Oracle.

12. LHCb Applications and GRID Integration. 12 Domenico Galli Gridi-fying the LHCb Production Tools First test (in these weeks): Submitting bb-bar inclusive jobs at RAL CSF (pbs) using the globus-run command. Second test: Modify production tools (e.g. java job submission tool) to allow submission of general jobs to pbs. Gradually replace data-move commands in the scripts by GRIDftp commands. Third test (September): Try the modified tools on test-bed 1.