INFSOM-RI Training: Build using the ETICS Service The ETICS Build Process Alberto Di Meglio ETICS Project

INFSOM-RI ETICS Training - Build - 24 September CERN 2 Building Software with ETICS ETICS can build your code and create distribution packagesETICS can build your code and create distribution packages It provides great flexibility and allows implementing different types of scenarios, from almost completely automated to almost completely user-definedIt provides great flexibility and allows implementing different types of scenarios, from almost completely automated to almost completely user-defined However the system relies on the information you register as a user and on a good understanding of the ETICS data modelHowever the system relies on the information you register as a user and on a good understanding of the ETICS data model It consists essentially of three main steps:It consists essentially of three main steps: BUILDING  PACKAGING  PUBLISHING

INFSOM-RI ETICS Training - Build - 24 September CERN 3 BUILDING

INFSOM-RI ETICS Training - Build - 24 September CERN 4 The etics-build command The ETICS command to perform builds is called etics- build. The syntax of the command is:The ETICS command to perform builds is called etics- build. The syntax of the command is: etics-build [options] [ ] It requires that the configurations to build have been previously checked out with the etics-checkout command etics-build [options] [ ] It requires that the configurations to build have been previously checked out with the etics-checkout command

INFSOM-RI ETICS Training - Build - 24 September CERN 5 Command options (1) OptionDescription -h, --helpShow the usage instructions -c, --config Define a specific configuration to be used instead of the default one, where default is the first configuration found in the store. This is normally not required unless multiple or non-default configurations of the selected module have been checked out -p, --property This option allows passing properties to the build process. If the property is already defined, its value is overridden by the value specified in the command-line. To pass multiple properties, use multiple –p options -e, --env This option allows passing environment variables to the build process. If the variable is already defined, its value is overridden by the value specified in the command-line. To pass multiple variables, use multiple –e options -t, --target Execute the build stopping at the specified target, If not specified the publish target is executed. Allowed targets are: clean, init, checkstyle, compile, test, doc, packager, publish, install --platform Overwrite the local platform (useful for testing or when the local platform is not a valid ETICS platform, but it’s compatible with one).

INFSOM-RI ETICS Training - Build - 24 September CERN 6 Command options (2) OptionDescription --nobuildDo not perform the build, just print the sequence --continueonerrorDo not stop building if an error is found --nodepsOnly build the currently specified module (do not build children and dependencies) --forceForce the build of unmodified modules --verbosePrint verbose messages --versionReturn the current client version number.

INFSOM-RI ETICS Training - Build - 24 September CERN 7 Basic build scenario The standard command to build a configuration is:The standard command to build a configuration is: etics-build [options] [ ] This command builds the current configuration of the specified module.This command builds the current configuration of the specified module. When run by itself, the command builds the configuration of the current project:When run by itself, the command builds the configuration of the current project: etics-build  etics-build etics-build  etics-build

INFSOM-RI ETICS Training - Build - 24 September CERN 8 Build Targets Target nameDescriptionMandatory cleanCommand to cleanNo initCommand to perform initialization operations before compiling No checkstyleCommand to perform code checking operations (coding conventions) No compileCommand to compile codeNo testCommand to perform static tests like unit tests or coverage tests No docCommand to generate documentationNo packagingCommand to generate distribution packagesNo publishCommand to publish build artifacts to standard distribution formats No installCommand to install the packageNo (but no packaging can be done if this is not defined)

INFSOM-RI ETICS Training - Build - 24 September CERN 9 How Targets are Evaluated The etics-build command executes the target in the following order:The etics-build command executes the target in the following order: init  checkstyle  compile  test  doc  packaging  publish The cycle stops after the execution of the target specified with the –t option or after publish if –t is not used.The cycle stops after the execution of the target specified with the –t option or after publish if –t is not used. The clean and install target are executed separately using the –t option.The clean and install target are executed separately using the –t option. If a target is not defined, it is normally simply skipped (but see later for packaging and publish).If a target is not defined, it is normally simply skipped (but see later for packaging and publish).

INFSOM-RI ETICS Training - Build - 24 September CERN 10 The Stage Target ETICS defines an internal target called ‘stage’ETICS defines an internal target called ‘stage’ This target uses the module’s ‘install’ target to install the module in a common stage areaThis target uses the module’s ‘install’ target to install the module in a common stage area The stage area can be used by other modules to locate interface files and libraries during the build processThe stage area can be used by other modules to locate interface files and libraries during the build process The ‘stage’ location is available to your build commands using the built-in ${stageDir} property that can be passed as a parameter or used to define environment variables. Ex if mod2 relies on MOD1_HOME, just attach the following env var to mod1, so that it will be available to mod2The ‘stage’ location is available to your build commands using the built-in ${stageDir} property that can be passed as a parameter or used to define environment variables. Ex if mod2 relies on MOD1_HOME, just attach the following env var to mod1, so that it will be available to mod2 In mod1: MOD1_HOME = ${stageDir}

INFSOM-RI ETICS Training - Build - 24 September CERN 11 How to define build commands

INFSOM-RI ETICS Training - Build - 24 September CERN 12 How to define build commands etics-configuration add/modifyetics-configuration add/modify

INFSOM-RI ETICS Training - Build - 24 September CERN 13 How to define build commands etics-configuration –i etics-configuration –i

INFSOM-RI ETICS Training - Build - 24 September CERN 14 PACKAGING

INFSOM-RI ETICS Training - Build - 24 September CERN 15 The ETICS Packaging System The ETICS Client comes with a built-in packaging system that allows to build distribution packages on all supported platforms in different formats (tarballs, RPMS, debs, MSIs, etc)The ETICS Client comes with a built-in packaging system that allows to build distribution packages on all supported platforms in different formats (tarballs, RPMS, debs, MSIs, etc) In order to use the ETICS Packager the install target has to be defined, since it is used internally to identify what the content of the package should be.In order to use the ETICS Packager the install target has to be defined, since it is used internally to identify what the content of the package should be. In order to activate the ETICS Packager, leave the packaging target undefinedIn order to activate the ETICS Packager, leave the packaging target undefined

INFSOM-RI ETICS Training - Build - 24 September CERN 16 Why Should You Use the EPS? It saves you from having to code packaging code in your modulesIt saves you from having to code packaging code in your modules It is cross-platform, once the metadata is defined it can be used to produce different kinds of packages of various platformsIt is cross-platform, once the metadata is defined it can be used to produce different kinds of packages of various platforms It allows to enforce consistent conventions for all packages across a projectIt allows to enforce consistent conventions for all packages across a project

INFSOM-RI ETICS Training - Build - 24 September CERN 17 Naming conventions Binary tarballs: -x.y.x-r.tar.gzBinary tarballs: -x.y.x-r.tar.gz Source tarballs: -x.y.x-r.src.tar.gzSource tarballs: -x.y.x-r.src.tar.gz Binary RPMS: -x.y.x-r...rpmBinary RPMS: -x.y.x-r...rpm Binary RPMS: -x.y.x-r.src.rpmBinary RPMS: -x.y.x-r.src.rpm Distribution is a standard OS distribution name like slc3, rhel4, fc4Distribution is a standard OS distribution name like slc3, rhel4, fc4 Architecture is a cpu architecture name like i386, i686, x86_64, noarch, etc.Architecture is a cpu architecture name like i386, i686, x86_64, noarch, etc.

INFSOM-RI ETICS Training - Build - 24 September CERN 18 Packaging Options OptionDescription --createsource Create source tarball and RPMS in addition to binary ones (this slows down the build due to the extra compilation steps --createdebug Create debug RPMS in addition to binary ones (this slows down the build due to the extra compilation steps [1] [1] --nodistname Do not append the distribution name to the RPM revision (it has no effect if --userspec is used) --usetimestamp Append a timestamp to the RPM revision number (it has no effect if --userspec is used) --versioneddeps Use version information when setting package dependencies in RPMS (ex: etics-client >= 1.0.0). Cannot be used together with –strictversioneddeps --strictversioneddeps Use strict version information when setting package dependencies in RPMS (ex: etics-client = 1.0.0). Cannot be used together with –versioneddeps --userspec Use a user-defined spec file. The spec file must be placed in /project. If this option is used all other packaging options are ignored [1] [1] Not supported in the current release

INFSOM-RI ETICS Training - Build - 24 September CERN 19 Packaging Properties Property nameDefault value if not specified TypeDescription package.prefix/opt/module-namestringThe default installation prefix for the generated RPMS package.buildarchLocal platform/distribution specifier (slc3, rhel4, etc) stringIt can be used to override the default specifier. It is necessary to set it to ‘noarch’ to generate noarch RPMS package.providesAutodetected if autoreqprov is set to yes (default) stringThe list of provides entries for the RPM spec file (overrides the autodetected list) package.requiresAutodetected if autoreqprov is set to yes (default) stringThe list of requires entries for the RPM spec file (overrides the autodetected list) package.obsoletesstringThe list of obsoletes entries for the RPM spec file package.conflictsstringThe list of conflicts entries for the RPM spec file

INFSOM-RI ETICS Training - Build - 24 September CERN 20 Packaging Properties Property nameDefault value if not specified TypeDescription package.autoreqprovyesstringCan be ‘yes’ or ‘no’. If it is set to yes, rpm will try to automatically detect requires and provides from the packaged files and libs, otherwise will only use specified entries Additional properties are available to set.pre/.post scripts and other various spec files optionsAdditional properties are available to set.pre/.post scripts and other various spec files options

INFSOM-RI ETICS Training - Build - 24 September CERN 21 How to Tell the Packager What to Package The packager relies on two elements from your metadataThe packager relies on two elements from your metadata –The install target –The installation prefix There are various conventions about how to pass this parameter depending on which installation method is used (autotools, ant, python distutils, etc)There are various conventions about how to pass this parameter depending on which installation method is used (autotools, ant, python distutils, etc) ETICS defines the following generic conventionETICS defines the following generic convention –whatever method is used to pass the prefix, the value of the prefix must be set in the commands as ${prefix}. The ETICS Packager will then resolve this value to an internal location of its own –Note that the same method is used to stage the files in the stage area

INFSOM-RI ETICS Training - Build - 24 September CERN 22 Installation Instructions Examples AutotoolsAutotools –init =./configure --prefix=${prefix} –compile = make –install = make install python distutilspython distutils –compile = python setup.py build –install = python setup.py --prefix=${prefix} Custom install scriptCustom install script –install =./my_installation_script --install_location=${prefix}

INFSOM-RI ETICS Training - Build - 24 September CERN 23 PUBLISHING

INFSOM-RI ETICS Training - Build - 24 September CERN 24 The ETICS Publisher The ETICS Client comes with pre-defined publishing conventions that allow harvesting build artifacts (packages, logs, etc) from the various modules into a common location. The ETICS Publisher uses the following conventions:The ETICS Client comes with pre-defined publishing conventions that allow harvesting build artifacts (packages, logs, etc) from the various modules into a common location. The ETICS Publisher uses the following conventions: –All checkout and build logs are stored in the reports directory of the workspace. The main log file is called: build-status.xml and contains summary checkout and build information for each module in the current build run. Detailed logs of the checkout and build operations of each modules are saved in files of the format: [CHECKOUT | BUILD]- -.log –All packages are stored in the directory dist

INFSOM-RI ETICS Training - Build - 24 September CERN 25 Packages Locations The packages in dist are organized in a tree that mirrors the structure of the ETICS software repository: / / / / The packages in dist are organized in a tree that mirrors the structure of the ETICS software repository: / / / / The packages are taken by default from the following locations in each module root:The packages are taken by default from the following locations in each module root: –(S)RPMS: /RPMS –(src.)tar.gz: /tgz

INFSOM-RI ETICS Training - Build - 24 September CERN 26 Packaging Properties used by the Publisher Property nameDefault value if not specified TypeDescription package.tgzLocation${location}/tgzstringThe default location where generated tarballs are stored in each module at the end of a build and where the Publisher looks for them package.SRPMSLocation${location}/RPMSstringThe default location where generated source RPMS are stored in each module at the end of a build and where the Publisher looks for them package.RPMSLocation${location}/RPMSstringThe default location where generated binary RPMS are stored in each module at the end of a build and where the Publisher looks for them