ADA Job Builder A Graphical Approach to Job Building ATLAS Software and Computing Workshop May 2005 Chun Lik Tan

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Agenda The current situation. A very quick tour of command-line job building What the ADA Job Builder hopes to achieve. One more time... this time graphically! Life after job submission... Current status. Future plans. Questions?

25th May 2005ADA Job Builder - A Graphical Approach to Job Building The current situation Release 1.10 of DIAL is the basis for the current ADA system. Job building is currently a command line affair. ADA/DIAL services are currently accessible from a ROOT client which provides the user with a comprehensive suite of DIAL commands... all within the ROOT environment. More recently, with PyDial ( developed by Karl Harrison at Cambridge ), the user can now work in the Python environment with.

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Command-line Job Building Search for a dataset: root [0] print(dsc.query("level = 'TOP' and owner='rome' and name like '%recov10%'", 30)) List has 6 entries: rome recov10.J2_Pt_35_70.AOD rome recov10.AcerMC_ttbb_l.AOD rome recov10.T1_McAtNLO_top.AOD rome recov10.SU2_Jimmy_focus.AOD rome recov10.SU3_Jimmy_bulk.AOD rome recov10.SU4_Jimmy_lowmass.AOD At the ROOT client command line:

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Command-line Job Building Check if dataset is suitable: root [1] print(dsc.attributes("rome recov10.SU3_Jimmy_bulk.AOD")) Row has 10 entries: level = TOP modtime = name = rome recov10.SU3_Jimmy_bulk.AOD nevt = nfile = 1441 nsub = 40 owner = rome type = AOD uid = virtual = 0 At the ROOT client command line:

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Command-line Job Building At the ROOT client command line: Note dataset’s name, retrieve it’s id and extract the dataset from the dataset repository: root [2] did = dsc.id("rome recov10.SU3_Jimmy_bulk.AOD"); root [3] pdst = dr.extract(did); root [4] pprint(pdst) EventMergeDataset with no parent is locked and not empty Content includes 1 block: Dataset content block: Dataset type: AtlasPoolEventDataset Content label: AOD Content ID list has 36 entries: type BJetContainer with with key BCandidates type ElectronContainer with with key ElectronCollection type INavigable4MomentumCollection with with key MuonboyTrackParticles type INavigable4MomentumCollection with with key StacoTrackParticles type INavigable4MomentumCollection with with key TrackParticleCandidate type INavigable4MomentumCollection with with key TrackParticleCandidateXK type JetTagContainer with with key BJetCollection...

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Selecting a task: root [5] tid = tsc.id("atlasopt_example_zll"); root [6] ptsk = tr.extract(tid); root [7] pprint(ptsk) Task has 2 files: atlas_release jo.py You will probably want to modify the task so extract the embedded files: root [8] ptsk->write_files("mytask", true) (const int)0 At this point, you open a shell and edit the relevant files. Once you are done with the modifications, resume the session. root [9] ptsk = new dial::Task("atlas_release jo.py", "mytask"); root [10] pprint(ptsk) Task has 2 files: atlas_release jo.py Command-line Job Building At the ROOT client command line:

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Command-line Job Building Do the same for application and then you are ready to submit! root [11] aid = asc.id("atlasopt"); root [12] papp = ar.extract(aid); root [13] pprint(papp) Application has 2 files: build_task run root [14] submit() Application Task Dataset *** Submitting job *** Submitted job status: CompoundJob is initializing Application: Task Dataset with events Job preferences ID 0-0 Run host: adial01.usatlas.bnl.gov... At the ROOT client command line: There is an easier way!

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Aims The graphical ADA Job Builder aims to: hide away the intricacies of the command line, present the user with a convenient and user friendly environment, provide job monitoring capabilities, manage job results, and be fully integrated with GANGA.

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Graphical Job Building

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Graphical Job Building SQL Query Builder constructing a dataset query:

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Graphical Job Building

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Graphical Job Building

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Graphical Job Building

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Graphical Job Building At this point, with application, task, dataset (and preferences) objects configured, the user can proceed to: save the configuration for future reference, and/or submit the job to the ADA Job scheduler. Submit

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Job Monitoring

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Job Monitoring Under the bonnet Based on the Twisted Framework ( Client and server model. Connection-oriented asynchronous communications. Push architecture using function callbacks. Server polls the ADA Job Scheduler and pushes updated job status information to subscriber clients (e.g. the ADA Job Builder). Based on GANGA v4 prototype monitoring component.

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Current Status Application selection & modification Task selection & modification Dataset selection Job submission Loading & saving of job configuration (job template) Job monitoring Sub-job monitoring Job result retrieval ADA Job Scheduler selector GSI certificate authentication Implement native DIAL web service client

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Future plans Short term (next few weeks) First beta release Documentation Job result retrieval Sub-job monitoring Job scheduler selector Middle term (next few of months) Integration with GANGA v4 GSI certificate authentication Native DIAL web service client

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Questions ?

25th May 2005ADA Job Builder - A Graphical Approach to Job Building Thank you!