1. Implementation of remote job submission over GRID with IMPALA/BOSS CMS MC production tools * A. Edunov, JINR, Dubna, U. Gasparini, INFN, Padua University and INFN Padua A. Kryukov, SINP Moscow State University, Moscow, S. Lacaprara, INFN, Padua University and INFN Padua M. Verlato, INFN, Padua University and INFN Padua ACAT-2002, 24-28 June A.Kryukov, SINP MSU

2. Outline ● Introduction ● Computational model ● IMPALA/BOSS overview ● Integration of IMPALA/BOSS with GRID ● DOLLY: The package to clone of CMS Production jobs ● Conclusions and results ACAT-2002, 24-28 June A.Kryukov, SINP MSU

3. Introduction ● CMS MC Production: SPRING-2002 – Tools: ● IMPALA ● BOSS – Stages ● CMKIN ● CMSIM ● OOHit formatting ● Digitization ACAT-2002, 24-28 June A.Kryukov, SINP MSU

4. Computational Model ObjDB ROOT, NTPL ObjDB NTPL FZ NTPL Pythia ORCA ooHITS GEANT 3 ORCA Digi ORCA User analysis Pythia ORCA ooHITS GEANT 3 ACAT-2002, 24-28 June A.Kryukov, SINP MSU

5. IMPALA/BOSS overview Input Parameters (Stored in IMPALA) Objectivity DB Runs to Process Parameters Interface Input Source Discovery Declare Script Create Script Submit Script Tracking Interface MySQL Based Tracking File Based Tracking Batch System Pre Process Run Job Script Post Process Input Parameters (Obtained with wget) Local Files to Process ACAT-2002, 24-28 June A.Kryukov, SINP MSU

6. IMPALA/BOSS integration with grid CMKIN Job Executer Jobs BOSS IMPALA Batch Manager CE WN1 WN2 Wnn UI MySQL DB Reference DB in CERN Environment ACAT-2002, 24-28 June A.Kryukov, SINP MSU

7. IMPALA/BOSS integration with GRID (continue) GRID CMKIN IMPALA Job Executer Gate Keeper Batch Manager BOSS Dolly WN1 WN2 Wnn UI CE Recource Broker Job Jobs MySQL DB CERN RefDB Environment ACAT-2002, 24-28 June A.Kryukov, SINP MSU

8. DOLLY: The package to clone CMS production jobs ● DOLLY_CE – Modified IMPALA ● DOLLY_CE.patch – DollyEnv.sh ● DOLLY_UI – Modified GetXXX_fromRefDB.sh – dolly –.dollyrc – Dolly-service ACAT-2002, 24-28 June A.Kryukov, SINP MSU

9. DOLLY: The package to clone CMS production jobs (continue) ● Three steps: – Installation: ● Just unpack: tar xzvf dolly-xxxxxx.tgz – Initialization working directory: ● Create working directory ● Modify of dolly-env and run it ● Run dolly-init command – Running ● Dolly commands. ACAT-2002, 24-28 June A.Kryukov, SINP MSU

10. ● Usage (briefly) dolly job_type [-debug] ● job_type is kin, sim, oohit. Oodigi in progress. Dolly-service utilitie for reinitialized some date. ● dolly-service job_type [-clean] [-r run_number] [-j job_number] [-s set_name] [-o newowner_number] Some additional utilities for retrieving data and monitoring jobs. DOLLY: The package to clone CMS production jobs (continue) ACAT-2002, 24-28 June A.Kryukov, SINP MSU

11. Conclusions and results ● There was designed and development new scheme of integration of IMPALA/BOSS production tools with GRID – Minimal modification in IMPALA. – Standard BOSS (version 3.x). ● The set of scripts (DOLLY) to clone the CMS production jobs on the UI side was developed. ● Two steps of production (CMKIN and CMSIN) was tested completely. ● The oodigi formating step is under development ACAT-2002, 24-28 June A.Kryukov, SINP MSU

12. Conclusions and results (continue) ● IMPALA problems – Single user application ● Require the write access to impala directories ● Use fixed names for temporally files (like /tmp/xxxx) – Two ways to pass initial data and parameters. ● Through the files (pythia cards,...) ● Through global environment ● BOSS – Direct access to MySQL DB in the world. ACAT-2002, 24-28 June A.Kryukov, SINP MSU

13. JDL file ########################################################### # # ----- kin_2006.jdl ----- # ########################################################### Executable = "kin_2006.sh"; StdOutput = "kin_2006.out"; StdError = "kin_2006.err"; InputSandbox = {"kin_2006.sh","kin_2006.tgz"}; OutputSandbox = {"kin_2006.out","kin_2006.err","kin_2006_out.tgz"}; ACAT-2002, 24-28 June A.Kryukov, SINP MSU

14. Script file ########################################################### # # ----- kin_2006.sh ----- # ########################################################### tar xfz kin_2006.tgz export defaultCMKINNumberOfEventsUI=10 export CMS_ProdData=CMS_ProdData./DollyCMKINJob tar cfz kin_2006_out.tgz * ACAT-2002, 24-28 June A.Kryukov, SINP MSU

15. Job file #!/bin/sh #Globals: export DATA_SET_NAME=PD_MB_test export REGIONAL_CENTER=PD_INFN_TEST export DATA_SET_DIR=`pwd`/${CMS_ProdData}/tracking/$REGIONAL_CENTER/cmkin/$DATA_SET_NAME export RUN_DATA_DIR=`pwd`/${CMS_ProdData}/rundata export IMPALA_ROOT=/home/impalabo/cms/impala export PROD_RESOURCES=/home/impalabo/cms/impala/scripts export ROOT_OF_WORK_PATH=`pwd`/${CMS_ProdData} export PATH=$PATH:$PROD_RESOURCES if [ -z $defaultCMKINNumberOfEventsUI ]; then export defaultCMKINNumberOfEventsUI=500 fi export EDGUpdate=true mkdir -p $DATA_SET_DIR DeclareCMKINJobs.sh -v $DATA_SET_NAME 1 CreateCMKINJobs.sh -v $DATA_SET_NAME 1 RunJob.sh -v -j cmkin $DATA_SET_NAME 1