NIGHTLY BUILD SCRIPT FOR THE LEVEL-1 TRIGGER ONLINE SOFTWARE 14th October 2010 Wolfgang Schlichtner

Introduction Aim: Build and test all the source code together Execute automatically every night Show the results via web interface > So the developers can take a look at the build results of the whole system every morning 14th October 2010 Wolfgang Schlichtner

Conditions Based on LCG Nightly Builds Runs on SLC4 32bit, gcc-3.4 Language: Python 14th October 2010 Wolfgang Schlichtner

Steps of the nightly script • Setup of the environment for build and installation • Checkout of the source code from the CVS Repository • Installation of XDAQ • Compilation of the source codes • Generation of RPMs (Red Hat Package Manager software packages) • Installation of the RPMs • Execution of SOAP-tests • Uninstallation of everything 14th October 2010 Wolfgang Schlichtner

Incorporate other L1TS subsystems Requirements: makefile to compile code and make RPMs test scripts (e.g. SOAP-Tests) 14th October 2010 Wolfgang Schlichtner

Client / Server structure One configuration.xml file How LCG nightlies work ... Client / Server structure One configuration.xml file Clients send their capabilities (architecture, OS, …) to the server, server starts the build jobs according to the configuration … 14th October 2010 Wolfgang Schlichtner

specify projects, platforms,… to build configuration.xml specify projects, platforms,… to build : <slot name="CMS-l1ts" description="CMS L1 Online Software"> <paths> <path name="builddir" value="/build/nightlies/%SLOT%/%DAY%" /> <path name="buildersdir" value="/build/nightlies/%SLOT%/%DAY%/LCGCMT..." /> <path name="releasedir" value="/afs/cern.ch/sw/lcg..." /> <path name="wwwdir" value="/afs/cern.ch/sw/lcg/app/nightlies/www..." /> </paths> <platforms> <platform name="slc4_ia32_gcc34" priority="1"/> </platforms> <days mon="true" tue="true" wed="true" thu="true" fri="true" ... /> <projects> <project name="CMS-l1ts" tag="CMS-l1ts-HEAD" /> </projects> </slot> 14th October 2010 Wolfgang Schlichtner

Ignore warnings/errors <warning value="PUT HERE WARNING MESSAGES IF YOU WANT TO IGNORE THEM"/> <error value="PUT HERE ERROR MESSAGES IF YOU WANT TO IGNORE THEM"/> </ignore> 14th October 2010 Wolfgang Schlichtner

Implementation to the LCG scripts There is just one main Builder class for all the LCG projects Therefore Karol from LHCb changed the scripts, so that external Builder classes can be used, this work is still in progress No documentation!  try & error 14th October 2010 Wolfgang Schlichtner

How to get our builder loaded by the server... export NIGHTLIES_EXTENSIONS=L1TS_Nightlies (name of the builder-module) export PYTHON_PATH=/path/to/the/builder/module:$PYTHON_PATH add line: __lcgnightlies_extenions__ = {“builder”}: [Name-of-builder-class]} at the bottom of the builder file method canBuild is needed to specify which projects the builder class can build to start the build-process, the server starts the buildProject() method of the builder class 14th October 2010 Wolfgang Schlichtner

after the build steps finished: checkLogFiles.checkBuildLogs: creates the build-html file and a summary file with the numbers of warnings/errors for the website WebUpdater.updateWeb: copies the html, summary and test-log file to the afs-dir for the webpage WebUpdater.updateDB: sends the results to the DB of the webpage, also checkLogFiles.checkTestLogs is called from within here checkTestLogs: had to be changed for our needs checkBuildLogs: I rewrote and simplified this html generator 14th October 2010 Wolfgang Schlichtner

How to start the nightly builds ... by acrontab: 0 0 * * * machine exec /afs-path/check_server.sh <machine> <port> /afs-path/configuration.xml 0 2 * * * machine exec /afs-path/resetServer.py –-machine=<machine> –-port=<port> 0 5 * * * machine exec /afs-path/launch_client.sh <machine> <port> the build-user (which defined the acron-jobs) needs rights for all of the used afs-dirs, the build-dir on the build-machine and a sudo without password request 14th October 2010 Wolfgang Schlichtner

The results of the nightly build ... are sent to the developers by email: 2010-08-26 15:23:30,400 INFO (WebUpdater.py:122) Errors found in log file for CMS-l1ts-HEAD slot CMS-l1ts while building for slc4_ia32_gcc34 Number of Warnings 0 Number of Errors 0 Number of Make Errors 0 Number of CMT  Errors 0 are shown on the nightly webpage 14th October 2010 Wolfgang Schlichtner

Thanks for your attention! 14th October 2010 Wolfgang Schlichtner