The Computing Environment

Outline Software Management –CVS –CMT –Tag-Collector –Savannah Data Structures Grid & Local cluster Athena –Projects & Packages –Versions –Python / C++ –Algorithms & Tools HyperNews Atlas Computing twiki page:

CVS - Concurrent Version System The software environment in Atlas (and in the whole HEP community) is always changing. CVS is a program the keep track of the evolution of the software. That way one can always return to a previous stable version, or advance to an exciting new version. The bookkeeping of the version is done using “tags”. Each version is associated with a specific tag, and it is identified by it. ViewVC: atlas.cgi/offline/ atlas.cgi/offline/

CMT - Configuration Management Tool An environment for packaged based development framework. The framework is divided into packages. Each package has a specific goal, and compiled into its own library. The CMT provide a simple way for configuring and building (gmake) the environment using simple configuration files (requirements) It is installed over the CVS system, and the check-out/in of the packages is done inside the cmt environment. LXR: Doxygen: computing/links/nightlyDocDirectory/allpackages.htmlhttp://alxr.usatlas.bnl.gov/ computing/links/nightlyDocDirectory/allpackages.html

Tag Collector A web interfaced database application which is an essential part of the management of ATLAS software releases. Sites: gCollectorInAtlas gCollectorInAtlas

Savannah Savannah is a tracking system used to track bugs and tasks Sites: twareDevelopmentWorkBookSavannah search=soft&words=%%&type=7 twareDevelopmentWorkBookSavannah search=soft&words=%%&type=7

Data Structures HepMC – A C++ event record for Monte-Carlo Generators. BS / PRD(PrepRawData) / RDO (Raw Data Object) – Low level encoding of detector information. Has the complete information of the hits in the detector, but has no reconstructed information. Size ~2MB/evt ESD (Event Summary Data) – Contains reconstructed objects together with hit clusters in each detector. Size ~500kB/evt AOD (Analysis Object Data) – A reduced sized files. Contains only reconstructed objects and their associated hit clusters and tracks in the detector. Size ~100kB/evt D n PD (Derived Physics Data) – Reduced size file. Contains derived objects (like Z/H) and “pre-analysis step” like overlap-removal, Slimming, thinning etc. Each group defines their own DPDs. Most of the DPDs are usually pool files. However the last DPD can be flat ntuple. (will be discussed on the advanced tutorial) TAG – List of identifiers are extracted to index the events. Size ~1kB/evt (?) Formats – –Ntuple – The well known flat ntuple. Several enhancement were made to include back navigation to AOD (Athena Aware Ntuple – AAN). Advantage – very fast analysis. Disadvantage – not very flexible. –Pool files – Containers of C++ objects. Advantage – very flexible. Disadvantage – complex and slow.

Grid & Local Cluster Local Cluster (TAU, Tech, WIS): –~140 a site a site –Half of the resources are for the Grid, and half is for local use. –Use it for small production, simple analysis and small tasks. Grid: –Institutes all over the world share their computing power and storage facilities in a big cloud called the Grid (see later talk by Lorne) –Use it to run your analysis on real data or centrally produced MC, heavy analysis jobs and long tasks that can be divided into parallel jobs

Athena See: A framework for Atlas software –A skeleton of an application into which developers plug in their code –Provides most of the common functionality and communications between different components –Embodies the underlying design and philosophy of the software –Encourages a common approach –Factors out common functionality for re-use Athena is a hybrid beast of python and C++. Its engine is python, and it is controlled by python scripts (also known as “JobOptions”), and it is capable of running C++ algorithms and tools. Divided into projects and packages. Each is associated with an overall release tag. Atlas Work Book

Athena Projects Some of the projects are: –AtlasAnalysis - This contains Tools, Algorithms and Services associated with physics analysis, monitoring and the event display. It depends upon the AtlasTrigger project. –AtlasCore - This contains core components and services (e.g. Athena and StoreGate). It depends upon the Gaudi framework project, and the LCGCMT project which provides links to LCG supported external software packages such as POOL and ROOT. –AtlasOffline - This is the project that provides the default entrypoint into the project hierarchy for interactive use. It depends upon the AtlasAnalysis and AtlasSimulation projects via explicit dependencies. By default it contains essentially no packages, but is a placeholder whereby bugfix releases may be created. The release number of this project is the ATLAS offline release number. –AtlasProduction - This is a top-level project for production use of the ATLAS offline release. It depends upon all other projects. Only a few packages are assigned to this project, in particular those that allow global testing of the complete release in a production enironment (e.g. KitValidation). By convention the release number of this project is the same as the ATLAS offline release number, although a forth digit (i.j.k.l) denotes a patch that is applied for production use only. See also WorkingWithPatchedReleases

Athena Package A package is a building block in Athena. Each package is created under a specific project, and has a certain goal. A package has the following directory structure: –cmt : contains requirements + setup files. This is where you build your code. – : contains the C++ header files. –src : contains the C++ source files. –share : contains job option (.py) files. –run : where you run your jobs. –python : any additional python source.

Athena Versions Athena has versions for releases and nightly. Nightly – Each night pre-defined versions of the code are build and the latest development is tested. The nightlies are rotated between rel_0 – rel_6, so you can always return to up to a seven days old nightly. Nightlies can be installed on the local cluster if necessary (I think???), however – I have no experience with it, and I think it is better to work with the local builds on lxplus. mary mary Releases – from time to time a new “stable” version of Athena is out to the market. In order to see the content of each release take a look at: computing/projects/releases/status/ computing/projects/releases/status/

Python / C++ Athena is handled in a python environment using jobOptions python scripts. Each C++ packages is compiled and a dynamic link library is created (.so files). The C++ inherits from a common interface (Tool, Algorith,…) and is represented using a python class defined automatically in genConf “some” things will become clearer after the tutorial later today

Algorithms & Tools Algorithm – Inherits from Algorithm. Has: initialize, execute and finalize. The method execute is called once per events Tool – Inherits from AlgTool. Can be instantiated many times. Service – Provides common services like StoreGate, message services, histograms etc.

How To Get Help 1 st advantage in big collaborations – Someone knows the answer so: Ask your friend 2 nd advantage in big collaboration – there is always someone that is dumber than you. Search for his dumb questions on the Hypernews, Savannah or google it. If you can’t find the answer, put your question on the Hypernews. Usually, the problem will be solved.