Progress on Design and Implement of Job Management System Suo Bing, Yan Tian, Zhao Xianghu

Outline Design principle and goals Data Flow Diagram Inspiration from Ganga Profile of JMS Integration of JUNO Integration of CEPC

Design principle and goals for users to split and submit jobs to backend common platform supporting multiple experiments: BES3, CEPC, JUNO, LHAASO, etc. general, simple, easy to expand Python 2.7

Data Flow Diagram Users Receive Input split submitexecute BackendSE steering file app inputsub job input output sandboxoutput data input +

Inspiration from Ganga Reference (advantages)  classical architecture of “Job, Application, Splitter, Handler, Backend”  common interface in GPIDev for expanding  strong, rigorous Abandonment (disadvantages)  complex wrapped class (schema, item)  confusing attributes (inputsandbox, inputdata)

JMS architecture ApplicationSplitter Adapter Backend specify and set input app params sub job params backend params user

JMS class diagram

JMS directory structure job subjob para ms status input app backend job input

JUNO Several script are supplied(by Lin Tao) for simulation and reconstruction in “offline/Examples/Tutorial/share” Run directly:  $ source …/setup.sh  $ python tut_detsim.py [--gdml run] Integrate to JMS: job input tut_detsim.py input 0 job.sh

JUNO test script #Import Statements j = Job() app = JunoApp() app.setCommand('tut _detsim.py') app.setArgs('--seed gdml gun')#start-seed and other args j.setApplications([app ]) splitter = SplitterByEvent(evtm ax = 20,evtTotal = 66) j.setSplitter(splitter) backend = Dirac() backend.setSite('CLO UD.IHEP- OPENNEBULA.cn') j.setBackend(backen d) j.prepare() j.submit()

JUNO parameters app_params job_params

JUNO parameters backend_params

JUNO Sequence Diagram

CEPC Generation + Simulation + Reconstruction  Generation→.stdhep→Simulation/Reconstruction  Steeringfiles (init.macro,event.macro) for simulation  Steeringfiles(init.xml,PandoraSettingsDefault.xml...) for reconstruction Integrate to JMS: job input job.sh input 0 init.macro event.macro init.xml PandoraSettingsDefault.xml PandoraLikelihoodData9EBin.xml Parsers read users' templates and generate input files for per job

CEPC test script app = CepcApp() app.setInitMacroPath('/home/cc/suob/tem plate/init.macro') app.setEventMacroPath('/home/cc/suob/te mplate/event.macro') #not flexible app.setInitXmlPath('/home/cc/suob/templa te/init.xml') app.setPandoraSettingPath('/home/cc/suo b/template/PandoraSettingsDefault.xml') app.setPandoraDataPath('/home/cc/suob/t emplate/PandoraLikelihoodData9EBin.xml ') j.setApplications([app]) splitter = SplitterByFile(evtmax = 20,seedStart = 50000) splitter.setInputFileDir('/besfs/groups/grid/ besdirac/cepc/stdhep/input_0010/') splitter.setInputFileLFN('LFN:/test/cepc/std hep/input_0010/') j.setSplitter(splitter) backend = Dirac() backend.setSite('CEPC.WHU.cn') backend.setGroup('suob-cepc-1219') backend.setOutputSE('WHU-USER') backend.setOutputPath('/test/cepc/1219') j.setBackend(backend)

Summary Evaluation – Inspired by gangaBOSS and ILCDIRAC, we designed a general Job Management System for BESIII, CEPC, JUNO and LHAASO. – A simple prototype has been coded and implemented. – CEPC and JUNO are supported and tested in this prototype, BESIII will be added soon. Future work  integrate more experiments and cases to improve versatility  add workflow module for application  consider realizing other functions (monitoring, tracking)