1. AtlasSetup & Evolution
Shuwei YE
BNL
June 26, 2019
Atlas S&C Week at NYU

2. Why new asetup?
Because the old asetup, designed for cmt releases, is complicated,
unnecessary for cmake releases, and difficult to maintain/improve.
Thus a new simplified asetup was re-designed for cmake releases.
Here are some comparison numbers: cmake vs cmt
9 (17) files, 8 (20) classes, 94 (224) functions,
4.2K (15K) lines (excluding tests)
Compiler setup: 1 (3) function, 129 (850) lines,
Release locator: 1 (7) function, 208 (1350) lines.
Full option help printout: 176 (410) lines.
And the old cmt releases co-exist with new cmake releases to be used,
the new asetup should also take care of the old releases,
and make the change transparent to users/panda as much as possible.

3. Determination of Release Type
The new asetup was put into production in early February 2019, automatic determination of asetup type to use.
Parse input option and argument to determine asetup type
Old cmt asetup
python/cmtRelease/
New cmake asetup
python/cmakeRelease/
Release No < , or nighly rel_N, relN
or option --cmt
Release No ≥
or other nightly release
or option --cmake
Using the new asetup with option --cmt should behave exactly like running the old asetup (incuding cmt and cmake releases)

4. Deployment of new asetup
Ignore/warn 258 old deprecated tags and 210 deprecated long options.
Example:
% asetup --simple --nosave Athena,22.0.0,here,builds
AtlasSetup(WARNING): ignore the following deprecated options in cmake releases:
['--simple', '--nosave']
AtlasSetup(WARNING): Deprecased tag ”builds" ignored in cmake releases
Using Athena/ [cmake] with platform x86_64-slc6-gcc62-opt
at /cvmfs/atlas.cern.ch/repo/sw/software/22.0
Test area: /tmp/yesw
In addition, 2 new options "--listRemovedTags" and "--listRemovedOpts” to print out the full list of deprecated tags/options.
Also documented on the reference wiki page:

5. Tests to prepare for new asetup
Because asetup is widely used, so performed many tests and asked users to try prior to the migration to the new asetup:
StressTest (composed of >50 different tests)
and comparison with old asetup.
Tests on ALRB
Announcement sent to many mailing lists
Announcement msg printout during ALRB setup
HammerCloud tests on grid
Communication with Tier0 production and panda team
For example, added the support for nightly stamp without prefix ‘r’.
User feedbacks

6. Feature-1 of New cmake-release asetup
Keep the useful features in old asetup:
Origin shell env saving/restoration
Live grouped options plus a brief help (fit one screen)
Useful features: --simulate and --debugprint
Multiple layers of configuration:
$AtlasSetupSiteCMake,
$HOME/.asetup, $PWD/.asetup
--siteconfigfile=, --inputfile=

7. Help of New cmake-release asetup
% asetup –h Usage: asetup [options] [tags] Options: -h, --help show this help message and exit --version Print out the asetup version --printLast, --printLastSession Print out the last saved session info under PWD --listRemovedTags Print out the list of removed old asetup tags --listRemovedOpts, --listRemovedOptions Print out the list of removed old asetup options --releasebase=<dir>, --releasepath=<dir> Specify an exact path to the release bypassing the area search --helpMoreOn=<group> The group name for help printout, valid groups: [CMAKE ,RELLOC,ASETUP,DB,PLATFORM,RELNAME,ENVVAR,COMPILER,ALL], where 'All' will print all groups options Examples: asetup Athena, # set up stable release asetup Athena,master,latest. # set up the latest master nightlies asetup Athena,master,r08 # set up the master nightlies of date 08 or asetup source otherScript # source additional user script For further details, look at

8. New Feature-2 of new cmake asetup
Improved features
Indented output and with both function and caller name for option --debugprint (names are dynamically extracted)
Tag names are automatically extracted from long option names or from default configuration, less prone to error in case of adding/removing options.
Much much fast way to locate “latest” nighly release.

9. New Feature-3 of new cmake asetup
New features
New option –printLast To print out the last saved release env info.
New option --envvarsNotSet=<vars> Specify a list of envvars not to set in asetup
(The option --noLcgReleaseBase can be removed?)
New option --traceEnvvar To trace an envvar definition and change
New function "asetup source userScript” (to be explained on new slide)
To append the user script into the saved session env script, so that next time one single command "asetup" can set up both the release env and user packages env.
Define shell functions run_sys_exe and alias_sys_exe
(to be explained on new slide)
To make aliases for the sys command emacs, condor_q, condor_submit.
to make them work under sys-incompatible release env.

10. Function "asetup source userScript”
An empty-arg asetup command would restore the last saved session env, but users still need source their own package(s) env. The purpose of this new function is to help users ease this burden. asetup Athena, asetup source myPackage1/setup.sh asetup source myPackage2/setup.sh The users scripts, myPackage1/setup.sh and myPackage2/setup.sh will also be saved into file $PWD/.asetup.save. Then next time under the work directory users just need specify asetup which is equivalent to the above 3 commands.

11. Shell functions run_sys_exe & alias_sys_exe
For many releases, the env is not compatible for some system commands, such as emacs, condor commands.
Two shell functions alias_sys_exe() and run_sys_exe() are introduced to make use of
/lib64/ld-linux-x86-64.so.2 --library-path /usr/lib64
to run system commands under incompatible release env.
If the system command has already been aliased, both functions will keep the options/argument in the defined alias. For example,
% alias emacs=“emacs –nw”
% alias_sys_exe emacas
% alias emacs emacs='/lib64/ld-linux-x86-64.so.2 --library-path /usr/lib64 /bin/emacs -nw’
The command emacs has always been aliased for users after running asetup.
Advantages:
Using ld.so run system command is fast (no overhead)
Most system command can be run properly (not for child process like “git push”)

12. Up-to-date Documentation
Reference Wiki update for new asetup is up-to-date.

13. Future Plan
Short term
Solution for “git push via https” under latest master release.
Add new option –unset
to unset the current release env, restore back to
the original shell env prior to the first asetup running.
(Should we unset also defined aliases ?)
Resolve the slight dependency on previous release env for some DB-related envvars
Remove the option –noLcgReleaseBase.
Long term
Cleanup of warning msg/code for removed tags and options
Remove cmt-based asetup if cmt releases are gone