1. GangaTasks Automating of job submission wit Athena/AthenaMC in Ganga sw week, 24.10.2007 johannes ebke, tariq mahmoud

2. tariq mahmoud atlas sw week, cern, 24102007 automating of job submission in Athena/AthenaMC outline I. introduction, concept of tasks II. starting and submitting a task III. automating of job submission IV. job interconnection V. conclusion

3. tariq mahmoud atlas sw week, cern, 24102007 automating of job submission in Athena/AthenaMC I. motivation aim: automate job submission. job chaining (interconnection) the user gives few parameters … and gets RESULTS

4. tariq mahmoud atlas sw week, cern, 24102007 automating of job submission in Athena/AthenaMC I. concept task few parameters: task parameters: float, name, number of events etc. process parameters: datasets, process name, geometry etc. input/output specifications Athena version AbstractJob generated automatically: name, task, status, necessary, prerequisites, run limit, number of attempts and attempt jobs, … AbstractJob job Ganga jobs

5. tariq mahmoud atlas sw week, cern, 24102007 automating of job submission in Athena/AthenaMC I. GangaTasks MCTask MCJob AnaTask AnaJob Task AnaJob repository/tools.data TaskList(GangaObject ) GPI.tasks

6. tariq mahmoud atlas sw week, cern, 24102007 automating of job submission in Athena/AthenaMC II: starting and submitting a new task running numbering on all tasks created by the same user if the task is 'new', make some settings permanent and set the status to 'running‘

7. tariq mahmoud atlas sw week, cern, 24102007 automating of job submission in Athena/AthenaMC III: automating Task()AnaTask() AnaJob() II: starting and submitting a new task auto submitting info resubmitting

8. tariq mahmoud atlas sw week, cern, 24102007 automating of job submission in Athena/AthenaMC III: automating s=AnaTask() s.name=" PythiaZmumu " s.float= 10 s.application_option_file=‘ $HOME/athena/…./run/AnalysisSkeleton_topOptions.py ' s.inputdata_dataset= " trig1_misal1_csc11.005145.PythiaZmumu.recon.AOD.v12000601_tid005999 " s.requirements_sites=[ 'LRZ', 'FZK', 'LYON‘ ] s.outputdata_outputdata=[“ AnaPythiaZmumu.root "] s.submit()

9. tariq mahmoud atlas sw week, cern, 24102007 automating of job submission in Athena/AthenaMC III: info and more

10. tariq mahmoud atlas sw week, cern, 24102007 automating of job submission in Athena/AthenaMC III: resubmitting

11. tariq mahmoud atlas sw week, cern, 24102007 automating of job submission in Athena/AthenaMC IV: jobs interconnections Task()MCTask() MCJob() II: starting and submitting a new task III: auto submitting info resubmitting chaining task steps interconnection, prerequisites

12. automating of job submission in Athena/AthenaMC IV: chaining task steps mc.total_events = 200; mc.events_per_job={"evgen": 50, "simul": 5, "recon":10} 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 13597 246 10 8 13597 246 8 13597 246 8 13597 246 8 2341 tariq mahmoud atlas sw week, cern, 24102007 evgen:i-0 simul:i-j recon:i-k

13. mc.total_events = 200 mc.events_per_job= {"evgen": 50, "simul": 5, "recon":10} mc.float=10 AbstractJob.prerequisites() tariq mahmoud atlas sw week, cern, 24102007 automating of job submission in Athena/AthenaMC IV: interconnections, prerequisites (1) Task.float: does NOT apply for subjobs

14. tariq mahmoud atlas sw week, cern, 24102007 automating of job submission in Athena/AthenaMC AnaTask: optimise sites CE= ‘ lcg-lrz-ce.lrz-muenchen.de:2119/jobmanager-sge-atlas ’ requirements_sites= ['LRZ', 'FZK', 'LYON'] optimize_sites=False inputdata_match_ce_all =True inputdata_min_num_files=100 sites_optimized=[]

15. tariq mahmoud atlas sw week, cern, 24102007 automating of job submission in Athena/AthenaMC V. conclusion GangaTask package: automates job management (small scale production system) works with Ganga Athena and Ganga AthenaMC. wiki in progress: https://twiki.cern.ch/twiki/bin/view/Atlas/GangaTasks To do: 1. gather the largest number of files possible of an in- complete dataset. 2. test everything on Lxplus “and elsewhere”. 3. soon available in one of the next in Ganga releases the user gives few parameters … and gets RESULTS