K. Harrison CERN, 25th September 2003 GANGA: GAUDI/ATHENA AND GRID ALLIANCE - Project news - Ganga release 1 - Work towards Ganga release 2 - Interaction with other projects - Conclusions

25th September Project news (1) - Ganga tutorial given at BNL (K.Harrison and C.L.Tan)  Ganga session included in one-day software tutorial at US-Atlas Computing and Physics Meeting (BNL, August)  Tutorial attended by about 50 physicists  Not able to cover everything in short-time available, but all participants successfully launched Ganga GUI, and were able to submit jobs to local LSF queue - Ganga workshop held at LBNL, 1-6 September  Contributors from US and UK brought together for extended discussions and some hands-on sessions  Started along path towards Ganga release 2  With D.Adams, agreed first steps for Ganga-DIAL integration

25th September Project news (2) - ITR (Information Technology Research) proposal from LBNL turned down by US National Science Foundation  LBNL will be unable to maintain current high level of involvement in Ganga  C.E.Tull has been a driving force in defining the project; hope that we don’t lose completely his expertise  W.Lavrijsen will continue to work on AthASK and the Python bus, both directly relevant to Ganga - Ganga presentation given by R.W.L.Jones at UK eScience All-Hands Meeting (Nottingham, 1-2 September) - Ganga Savannah Portal for bug reporting and tracking set up:

25th September Ganga release 1: availability - Ganga release 1 was placed in Gaudi CVS at beginning of July - Several small modifications and improvements have been made, leading to minor releases (latest tagged version is Ganga ) - Code can be obtained using:  cvs –d login [password: CERNuser]  cvs –d co –P Ganga - GUI is launched by executing:  Ganga/GangaMain/src/Ganga.py (Requires that wxPython be available – no problems on lxplus)

25th September Ganga release 1: functionality - Ganga release 1 includes:  GUI  Command-line access to underlying tools (but not user oriented)  Job-options editor (optimised for Atlfast)  Job submission to different batch systems (localhost, LSF, PBS)  Mechanism for splitting/cloning jobs (usability to be improved)  Job catalogue  Automatic monitoring  On-line help

25th September Ganga release 1: first usage - Ganga has been installed and run at a number of sites: Birmingham, BNL, Cambridge, CERN, Imperial, Oxford, LBNL - Ganga has been used to submit jobs for a variety of applications  Atlfast, where Ganga specialisations help user  Atlsim and Athena reconstruction, currently without Ganga specialisations  Applications of LHCb and BaBar - Test users have provided valuable feedback to developers

25th September Basic GUI Job tree Main panel Toolbar Python interpreter

25th September Job creation

25th September Job-parameters panel

25th September Job-options editor: sequences

25th September Job-options editor: options

25th September Job submission

25th September Examination of job output

25th September Ganga release 2: refactorisation - Plans for Ganga release 2 include a refactorisation  Improve breakdown into components, to simplify the integration in Ganga of external components, and the use externally of Ganga components  Collect information that may be used by Ganga in a layer of databases or database-like structures  Ensure clear mapping onto services identified by LCG RTAG11 (ARDA) - Python bus will be introduced to mediate communication between components, as opposed to simply using the Python interpreter  Improve configurability: bus can determine component to load on basis of functionality requested  Prototype (PyBus) has been written (W.Lavrijsen)PyBus

25th September Ganga release 2: refactorisation diagram

25th September Ganga release 2: other improvements - High priority will be given to user-oriented CLI - Unit testing will be introduced  Help ensure quality of software - Interfaces to Magda and AMI will be added  Python bindings have been written (C.L.Tan) - Job-options handling will be more sophisticated  Editor will be more generic; options validator is to be added - Remote job submission will be enabled  First tests of procedures for issuing commands remotely from Python have been carried out (K.Harrison,W.Lavrijsen) - Ganga will become more Grid-oriented  Submission to EDG is already possible in principle, but procedure needs to be checked; submission to NorduGrid is being added (A.Soroko)

25th September Interaction with other projects (1) - AthASKAthASK  W.Lavrijsen will be migrating most (all?) AthASK functionality into PyBus components that can be harvested by GangaPyBus - DIALDIAL  DIAL, with focus on interactive analysis, and Ganga, with focus on batch processing, are complementary (even if divide between interactive and batch processing is hazy)  First steps for Ganga-DIAL integration agreed with D.Adams  Idea is to wrap C++ classes of DIAL using SWIG or Boost.Python,SWIGBoost.Python and so make them available from Ganga  User will be able to use DIAL directly, or will be able to use it transparently from Ganga

25th September Interaction with other projects (2) - PacmanPacman  Meeting on installation and packaging held this morning  Will start looking at ways to take advantage of Pacman, in particular for packaging user code on the client side and unpackaging the code at a remote (Grid) site - AtCom and DC2AtCom  C.L.Tan is a member of DC2 Task Force; other Ganga people also following developments  Role Ganga might play in DC2 still to be defined: Production request tool? Production definition tool? Supervisor? Other?

25th September Conclusions - A lot of work has been done on Ganga since the May software workshop, with a strong emphasis on dealing with Atlas-specific requirements - Ganga release 1 is available from Gaudi CVS repository  This isn’t production quality, but gives a feeling of how things should work - Ganga has been used to run Atlsim, Atlfast and Athena reconstruction at both European and US sites - Work has started on interfacing Ganga to other Atlas tools: AMI, DIAL, Magda, Pacman - Good progress is being made towards Ganga release 2 (refactorised design, remote submission, other improvements)