CMT Christian Arnault – CMT tutorial – dec CMT Tutorial How to use CMT in Atlas Christian Arnault

CMT Christian Arnault – CMT tutorial – dec Introduction Presentation of the tutorial –It is composed of three short lectures (~ 20’ each) The Atlas work model and conventions Working on Atlas packages Advanced features –It includes materials for documented examples (rather than exercises) with screen dumps Goal, scope –It is really aimed at Atlas context rather than being a general purpose CMT tutorial –The expected audience ranges from the normal software operator up to the more expert developer Announce –CMT workshop at LAL Feb-28/March

CMT Christian Arnault – CMT tutorial – dec Plan Lecture #1 :The Atlas work model and conventions –Introduction –Work model, conventions –How to setup CMT –The CMT user interface –The CMT architecture Lecture #2 : Working on Atlas packages –Working with a package –Creating a new package –Running an application –Examples in Atlas Lecture #3 : Advanced features –Policy and patterns in Atlas –Package types and areas –Details on configuration parameters

CMT Christian Arnault – CMT tutorial – dec Lecture #1 The Atlas work model and conventions

CMT Christian Arnault – CMT tutorial – dec Work model, conventions (1) The roles –So far, we only consider “developers” and “package coordinators” Developers work (edit, commit, tag, test, register) on individual packages Package coordinators accumulate package tags and validate The scenarios –Working on a package in a private development area –Connecting to the official release We are talking about the current practices in Atlas rather than the standard or the final definitions (since this is not yet fully defined / understood / adopted). A working group is currently being setup to define : the roles (developer, coordinators, librarian, etc…) the scenarios (developing, tagging, testing, integrating, installing, deploying, …)

CMT Christian Arnault – CMT tutorial – dec Work model, conventions (2) The software organization –The software base is a hierarchy of packages: LArCalorimeter LArCalorimeter/LArBookkeeping LArCalorimeter/LArCellRec LArCalorimeter/LArClusterRec LArCalorimeter/LArCnv LArCalorimeter/LArCnv/LArAthenaRoot LArCalorimeter/LArCnv/LArHitZebraCnv LArCalorimeter/LArConditions LArCalorimeter/LArDetDescr LArCalorimeter/LArDetDescrObjyDDL LArCalorimeter/LArDigitization LArCalorimeter/LArEvent LArCalorimeter/LArEventObjyDDL LArCalorimeter/LArExample LArCalorimeter/LArExample/LArBookkeepingExample LArCalorimeter/LArExample/LArConditionsExample LArCalorimeter/LArExample/SimpleCellBuilder LArCalorimeter/LArExample/TestLArDigit LArCalorimeter/LArExample/TestLArHardwareID containers

CMT Christian Arnault – CMT tutorial – dec Work model, conventions (3) The software organisation –Packages may have particular semantics Normal packages provide software or documents Container packages manage other packages Policy packages hold Atlas specific conventions, policies, patterns Interfaces convert conventions and policies of external packages –We must differentiate The physical hierarchy (the directory structure) The logical hierarchy (relationships between packages) –Those two hierarchies are totally independent with each other –A release is the unique and exhaustive set of all [package-version] duets validated as consistently built and cooperating. (see next screen) It is materialized by one container package AtlasRelease

CMT Christian Arnault – CMT tutorial – dec use InDetSpacePoint InDetSpacePoint InnerDetector/InDetRec use SiClusterization SiClusterization InnerDetector/InDetRec use SiTrackerCluster SiTrackerCluster InnerDetector/InDetRec use SiTrackerSpacePoint SiTrackerSpacePoint InnerDetector/InDetRec use TRT_Rec TRT_Rec InnerDetector/TRT use LArBookkeeping LArBookkeeping LArCalorimeter use LArCellRec LArCellRec LArCalorimeter use LArClusterRec LArClusterRec LArCalorimeter use LArAthenaRoot LArAthenaRoot LArCalorimeter/LArCnv use LArHitZebraCnv LArHitZebraCnv LArCalorimeter/LArCnv use LArConditions LArConditions LArCalorimeter use LArDetDescr LArDetDescr LArCalorimeter use LArDetDescrObjyDDL LArDetDescrObjyDDL LArCalorimeter use LArDigitization LArDigitization LArCalorimeter use LArEvent LArEvent LArCalorimeter use LArEventObjyDDL LArEventObjyDDL LArCalorimeter use LArBookkeepingExample LArBookkeepingExample LArCalorimeter/LArExample use LArConditionsExample LArConditionsExample LArCalorimeter/LArExample use SimpleCellBuilder SimpleCellBuilder LArCalorimeter/LArExample use TestLArDigit TestLArDigit LArCalorimeter/LArExample use TestLArHardwareID TestLArHardwareID LArCalorimeter/LArExample use TestLArHECTBCnv TestLArHECTBCnv LArCalorimeter/LArExample use TestLArTBCnv TestLArTBCnv LArCalorimeter/LArExample use LArGraphics LArGraphics LArCalorimeter use LArROD LArROD LArCalorimeter use LArSim LArSim LArCalorimeter use LArHECTBAna LArHECTBAna LArCalorimeter/LArTestBeam use LArHECTBCnv LArHECTBCnv LArCalorimeter/LArTestBeam use LArHECTBPed LArHECTBPed LArCalorimeter/LArTestBeam use LArTBCnv LArTBCnv LArCalorimeter/LArTestBeam use MagneticFieldAge MagneticFieldAge MagneticField use MagneticFieldAgeHeader MagneticFieldAgeHeader MagneticField use MagneticFieldAthena MagneticFieldAthena MagneticField use MagneticFieldAthenaHeader MagneticFieldAthenaHeader MagneticField use MagneticFieldCore MagneticFieldCore MagneticField use MagneticFieldCoreHeader MagneticFieldCoreHeader MagneticField use MagneticFieldStand MagneticFieldStand MagneticField

CMT Christian Arnault – CMT tutorial – dec Work model, conventions (4) The software organisation –A release can be frozen or under development The latter will likely become the integration release –Releases are maintained in ATLAS_DIST_AREA/ATLAS_RELEASE Where –ATLAS_DIST_AREA takes one specific value per site where the Atlas software is installed (CERN, LBL, LBNL, laptops, …) –ATLAS_RELEASE is a 3-number tag describing the release level % ls /afs/cern.ch/atlas/software/dist/

CMT Christian Arnault – CMT tutorial – dec Work model, conventions (5) The internal work model of CMT –Each package is identified by a triplet [package - version - location] –Packages are located and searched for in package areas –The search list for all possible areas is specified in one path-like environment variable named CMTPATH software/dist/2.4.0 % cmt show path # Add path /afs/cern.ch/atlas/offline/external/Gaudi/0.8.3 from ${CMTPATH} # Add path /afs/cern.ch/atlas/software/dist/2.4.0 from ${CMTPATH} # Add path /afs/cern.ch/sw/contrib from default path # /afs/cern.ch/atlas/offline/external/Gaudi/0.8.3:/afs/cern.ch/atlas/software/dis t/2.4.0:/afs/cern.ch/sw/contrib software/dist/2.4.0 % echo $CMTPATH /afs/cern.ch/atlas/offline/external/Gaudi/0.8.3:/afs/cern.ch/atlas/software/dis t/2.4.0 Gaudi area Release area

CMT Christian Arnault – CMT tutorial – dec Work model, conventions (6) The internal work model of CMT –Version specification accepts a semantics based on a 3-level identifier scheme: [ [ ]] Major-id describes major evolutions (that imply code change to client packages) Minor-id describes minor evolutions (backward compatible change to the interface, only imply re-compilation) Patch-id describes simple patches (i.e. no change to any interface) LArDetDescrObjyDDL LArDigitization LArEvent LArEventObjyDDL LArBookkeepingExample LArConditionsExample SimpleCellBuilder TestLArDigit v1 v10 v10r1 v13r10 v1r0 v1r24p0 v1r6p2 v2 v2001 v2r0p2

CMT Christian Arnault – CMT tutorial – dec Work model, conventions (7) The internal work model of CMT –The resulting directory structure of a package reflects this organization: LArCalorimeter/LArCellRec/ LArCalorimeter/LArCellRec/LArCellRec LArCalorimeter/LArCellRec/LArCellRec /LArCellRec LArCalorimeter/LArCellRec/LArCellRec /ChangeLog LArCalorimeter/LArCellRec/LArCellRec /PACKAGE LArCalorimeter/LArCellRec/LArCellRec /configure LArCalorimeter/LArCellRec/LArCellRec /src LArCalorimeter/LArCellRec/LArCellRec /cmt LArCalorimeter/LArCellRec/LArCellRec /cmt/Makefile LArCalorimeter/LArCellRec/LArCellRec /cmt/requirements LArCalorimeter/LArCellRec/LArCellRec /share LArCalorimeter/LArCellRec/LArCellRec /Linux-gcc-dbg LArCalorimeter/LArCellRec/LArCellRec /Linux-gcc-opt Location Package Version Branches Header files Sources CMT Binary files

CMT Christian Arnault – CMT tutorial – dec Work model, conventions (8) The internal work model of CMT –Each package holds its own entire configuration specification into one single text file named requirements This file must reside in the cmt sub-directory –By default everything belonging to the package (source files, or generated files) will stay within the context of the package Customizations are possible (eg to install generated files elsewhere) –There is one single relationship established between packages, qualified by the use concept. (see next screens) –CMT relies on this structure to address used packages, resolving complete address from The name The required version expression The possible access paths from CMTPATH

CMT Christian Arnault – CMT tutorial – dec LArCellRec/LArCellRec /cmt % cat requirements package LArCellRec author RD Schaffer author Srini Rajagopalan use AtlasPolicy AtlasPolicy-* use CxxFeatures CxxFeatures-00-* Utilities use GaudiInterface GaudiInterface-01-* External use StoreGate StoreGate-02-* Control use Identifier Identifier-00-* DetectorDescription use AtlasDetDescr AtlasDetDescr-00-* DetectorDescription use EventAthena EventAthena-00-* Event use EventStructure EventStructure-00-* Event use ZebraTDREvent ZebraTDREvent-00-* Event use EventRawData EventRawData-00-* Event use LArDetDescr LArDetDescr-00-* LArCalorimeter use LArEvent LArEvent-00-* LArCalorimeter use LArROD LArROD-01-* LArCalorimeter use LArSim LArSim-00-* LArCalorimeter use CaloRec CaloRec-01-* Calorimeter

CMT Christian Arnault – CMT tutorial – dec (Excerpt from the graphical representation of the use hierarchy by CMT)

CMT Christian Arnault – CMT tutorial – dec How to setup CMT (1) The bootstrap operation: how to connect for the very first time with CMT –Call the setup script of CMT itself. –This selects one version of CMT –This only needs to be done once until the next change of CMT version For switching to another version of CMT, see later … user/c/carnault % source /afs/cern.ch/sw/contrib/CMT/v1r10p /mgr/setup.sh user/c/carnault % cmt version V1r10p user/c/carnault % echo $CMTROOT /afs/cern.ch/sw/contrib/CMT/v1r10p

CMT Christian Arnault – CMT tutorial – dec How to setup CMT (2) Setting up an Atlas environment 1.Install a HOME requirements file eg. in ${HOME}/cmtdir 2.Install in this file: a.The site identification ( CMTSITE ) b.The location to the distribution area ( ATLAS_DIST_AREA ) c.The selected release level ( ATLAS_RELEASE ) d.The connection to the standard AtlasSettings definitions 3.… user/c/carnault % mkdir cmtdir user/c/carnault % cd cmtdir user/c/carnault/cmtdir % vi requirements set CMTSITE CERN macro ATLAS_DIST_AREA "/afs/cern.ch/atlas/software/dist“ #macro ATLAS_RELEASE "2.3.1" macro ATLAS_RELEASE "2.4.0" use AtlasSettings v* $(ATLAS_DIST_AREA) 1 2-a 2-b 2-c 2-d

CMT Christian Arnault – CMT tutorial – dec How to setup CMT (3) Setting up Atlas environment in various sites –Change or select CMTSITE –Provide the possible values for ATLAS_DIST_AREA (some sites are already defined) –A proper SITEROOT env. variable is expected for other sites. This will provide logical root entry point towards various software user/c/carnault/cmtdir % vi requirements set CMTSITE CERN #set CMTSITE BNL #set CMTSITE LBNL #set CMTSITE EXTSITE macro ATLAS_DIST_AREA "/afs/cern.ch/atlas/software/dist" \ CERN "/afs/cern.ch/atlas/software/dist" \ BNL "/afs/rhic/usatlas/software/dist" \ LBNL "/auto/atlas/local/software/dist" \ EXTSITE "${SITEROOT}/software/dist“ ${SITEROOT}/cern => Cernlib ${SITEROOT}/external => Atlas external software ${SITEROOT}/ROOT => ROOT ${SITEROOT}/sw => CERN Anaphe etc… ${SITEROOT}/software => Atlas software

CMT Christian Arnault – CMT tutorial – dec How to setup CMT (4) Setting up an Atlas environment … 3. Configure for CMT by cmt config (this creates the setup and cleanup scripts) 4. Apply the setup script (only this last step has to be redone at each session startup). c/carnault/cmtdir % cmt config Configuring environment for standalone package. CMT version v1r10p System is Linux-gcc-dbg Creating setup scripts. Creating cleanup scripts. user/c/carnault/cmtdir % source setup.sh 3 4

CMT Christian Arnault – CMT tutorial – dec How to setup CMT (5) Setting up an Atlas environment … 5. Declare a private development area into the HOME requirements file 6. Create if needed the development area 7. Apply this definition to upgrade CMTPATH c/carnault/cmtdir % vi requirements... path_remove CMTPATH ${HOME} path_prepend CMTPATH ${HOME}/devarea user/c/carnault/cmtdir % mkdir –p../devarea user/c/carnault/cmtdir % source setup.sh user/c/carnault/cmtdir % cd../devarea 5 6 7

CMT Christian Arnault – CMT tutorial – dec How to setup CMT (6) Understanding what is the current settings 1. cmt version (the version of CMT) 2. cmt show path (the CMTPATH entries) 3. cmt show tags (the active configuration tags) 4. cmt show packages | grep dist (all reachable package-versions) c/carnault/cmtdir % cmt version v1r10p c/carnault/cmtdir % cmt show path # Add path /afs/cern.ch/atlas/offline/external/Gaudi/0.8.3 from initialization # Add path /afs/cern.ch/atlas/software/dist/2.4.0 from initialization # Add path /afs/cern.ch/sw/contrib from default path # /afs/cern.ch/atlas/offline/external/Gaudi/0.8.3:/afs/cern.ch/atlas/software/dist/2.4.0:/afs/cern.ch/s w/contrib c/carnault/cmtdir % cmt show tags Linux (from uname) package AtlasSettings implies [gcc debug] i386_linux22 (from CMTCONFIG) package CMT implies [Linux] CERN (from CMTSITE) debug (from package CMT) package AtlasSettings excludes [opt optimized] Default (from Default) gcc (from package AtlasSettings) excludes [egcs] Linux-gcc-dbg (from package AtlasSettings) package AtlasSettings implies [debug] Linux-gcc (from CMTCONFIG in package AtlasSettings) 1 2 3

CMT Christian Arnault – CMT tutorial – dec The CMT user interface Principle : one command driver : cmt –All CMT actions are operated via this command line driver –gmake internally makes a wide use of it Basic syntax cmt [ - = ] Main commands cmt config re-configure a package cmt create … create a new package cmt show … monitors configuration parameters cmt broadcast … launch shell command over several packages cmt checkout … interface to CVS’ checkout operation cmt help get the list of possible commands source cleanup.sh undo the source setup

CMT Christian Arnault – CMT tutorial – dec The CMT user interface Parameter monitoring cmt show … path package search paths uses the use tree macros all macro definitions sets all set and path definitions tags currently active tags macro complete definition for a macro set complete definition for a set constituents all constituents for this package fragments all available make fragments clients all direct clients of this package packages all reachable packages

CMT Christian Arnault – CMT tutorial – dec The CMT architecture The database –The requirements file It is the unique source of configuration parameters for all packages CMT automatically connects requirements files of all used packages (belonging to the current use tree) The use relationship –The unique relationship between packages managed by CMT –Parameterized by the version –Similar to inheritance (includes private or public qualification) The configuration parameters –Inherited parameters Use relationships : relate packages to each other Symbols : define make macros, environment variables, aliases Tags : define value selectors for symbols Patterns : factor out recurrent configuration schemes Make fragments : extend the CMT behaviour –Local definitions (never inherited) Constituents : for libraries, applications, document generators

CMT Christian Arnault – CMT tutorial – dec End of lecture #1