1. K.Harrison CERN, 6th March 2003 GANGA: GAUDI/ATHENA AND GRID ALLIANCE - Aims and design - Progress with low-level software - Progress with Graphical User Interface - Conclusions and next steps

2. 6th March 20032 Aims - The Indian goddess Ganga descended to Earth to flow as a river (English: Ganges) that carried lost souls to salvation - Ganga software is being developed jointly by ATLAS and LHCb to provide an interface for running Gaudi/Athena applications on the Grid  Deal with all phases of a job life cycle: configuration, submission monitoring, error recovery, output collection, bookkeeping  Carry jobs to the Grid underworld, and hopefully bring them back - Idea is that Ganga will have functionality analogous to a mail system, with jobs having a role similar to mails  Make configuring a Gaudi/Athena job and running it on the Grid as easy as sending a mail

3. 6th March 20033 Ganga: schematic representation Gaudi/Athena application Ganga GUI/CLI JobOptions Algorithms Collective & Resource Grid Services Histograms Monitoring Results

4. 6th March 20034 Design considerations - Ganga should not reproduce what already exists, but should make use of, and complement, work from other projects, including AtCom, AthASK, DIAL and Grappa in ATLASAthASKDIALGrappa  Should also follow, and contribute to, developments in Physicist Interface (PI) project of LCGPhysicist Interface (PI) - The design should be modular, and the different modules should be accessed via a thin interface layer implemented using a scripting language, with Python the current choice - Ganga should provide a set of tools that can be accessed from the command line (may be used in scripts), together with a local GUI and/or a web-based GUI that simplifies the use of these tools - Ganga should allow access to local resources as well as to the Grid

5. 6th March 20035 Tentative Ganga architecture Server EDG UI PYTHON SW BUS XML RPC server XML RPC module GANGA Core Module OS Module Gaudi /Athena GaudiPython PythonROOT PYTHON SW BUS GUI DB Remote user (client) LAN/WAN GRID LRMS Local Job DB Production DB Bookkeeping DB Job Configuration DB

6. 6th March 20036 Low-level software - In Ganga, components used to define a job are mapped to Python classes - The low-level software needs to provide representations of the most basic components, and must allow flexibility in manipulating them - Users will have access to low-level data and methods at the command line, but in most cases shouldn’t need them  Common tasks will be simplified through higher-level functionality, available both at the command line and through a GUI - Low-level software presented here works, but is preliminary  Details of component representations are still being discussed by development team

7. 6th March 20037 Job components - In the preliminary implementation, the component breakdown is as follows: 6 Job: a WorkFlow and a set of methods for submitting a script to a particular batch system  For now, consider LSF  Grid will be modelled as just another batch system 6 WorkFlow: a series of WorkSteps, defining the sum of the actions to be performed when the job is run 6 WorkStep: a set of sub-components providing all information necessary to run one instance of an executable 6 WorkStep sub-components: Command, InputFile, OutputFile, CMTPackage (produces a library), CMTApplication (has an executable associated with it)  Other sub-components to follow

8. 6th March 20038 Defining and running jobs - For now, consider case where experiment’s standard software is pre-installed at the site where the job runs (as for EDG), and user software has been configured (i.e. cmt requirements and job options are defined)  Ganga should help with configuration in the future, but user will always have to do the work of developing his/her own analysis code - User software does not have to be pre-installed on remote site - Command sequences may be entered at Ganga prompt, or may be used in a Python script that imports the Ganga classes: from GangaKernel.GangaImport import 

9. 6th March 20039 Atlas example (Atlfast) atlasSetup = GangaCommand(“source /afs/cern.ch/user/h/harrison/public/atlasSetup.sh”) atlfast = GangaCMTApplication(“TestRelease”, “TestRelease-00-00-15”,”athena.exe”, “run/AtlasfastOptions.txt”) atlfastOutput = GangaOutputFile(“atlfast.ntup”) workStep1 = GangaWorkStep([atlasSetup,atlfast,atlfastOutput]) workFlow = GangaWorkFlow([workStep1]) lsfJob = GangaLSFJob(“atlfastTest”,workFlow) lsfJob.build() lsfJob.run()  At this level, there is less built-in intelligence than in ASK, but a lot of flexibility

10. 6th March 200310 LHCb example (DaVinci) lhcbSetup = GangaCommand(“source /afs/cern.ch/lhcb/scripts/lhcbenv.sh”) daVinciSetup = GangaCommand(“source /afs/cern.ch/lhcb/scripts/ProjectEnv.sh DaVinci v7r2”) daVinci = GangaCMTApplication(“Phys/DaVinci”, “v7r2”,”DaVinci.exe”, “options/DaVinci.opts”) daVinciOutput = GangaOutputFile(“dvhistos.hbook”) workStep1 = GangaWorkStep([lhcbSetup,daVinciSetup,daVinci, daVinciOutput]) workFlow = GangaWorkFlow([workStep1]) lsfJob = GangaLSFJob(“daVinciTest”,workFlow) lsfJob.build() lsfJob.run()  Command sequences for Atlas and LHCb are very similar

11. 6th March 200311 Graphical User Interface - GUI development has been undertaken by A.Soroko - Implementation is based on wxPython extension module - Current version is mainly illustrative  Right buttons are there; full functionality will come later

12. 6th March 200312 GUI: basic design Embedded Python interpreter

13. 6th March 200313 GUI: job creation Create a new job

14. 6th March 200314 GUI: specifying WorkFlow Give name and version of the application

15. 6th March 200315 GUI: defining job options Job (application) options downloaded from DB

16. 6th March 200316 GUI: defining batch system Define batch system

17. 6th March 200317 GUI: defining resource requirements Define necessary resources

18. 6th March 200318 GUI: job submission Submit the job (after configuration)

19. 6th March 200319 GUI: notification of job submission Job has been submitted successfully

20. 6th March 200320 Conclusions and next steps - Low-level tools that allow (pre-configured) jobs to be handled in a similar manner for Atlas and LHCb have been developed - Implementation of GUI is at an advanced stage - Jobs have been submitted successfully through Ganga to Cern LSF  Ready to start experimenting with job submission to Grid - C.L.Tan is working to understand how AtCom and Ganga can best complement one another - C.E.Tull has started investigating possibilities for exploiting Grid Monitoring Architecture in Ganga - Ganga abstract has been accepted for presentation at CHEP03