Packaging a Structural Biology Application – imod / 3dmod CVL Workshop Packaging a Structural Biology Application – imod / 3dmod James Wettenhall Nov 7th, 2012

Install without CVL Build System First Install the software on a CentOS 6.2/6.3 test machine. I used our “CVL Demo” instance. Installing from binaries is OK, as long as the binaries are relocatable CVL software must be installed in /usr/local/<package>/<version>/ Add menus, menu items and the modulefile.

imod / 3dmod running on CVL Demo instance

imod/3dmod installed in modules environment

imod/3dmod menus and menu items

Preparing imod/3dmod for packaging in the CVL Build System As root on machine where imod/3dmod is installed: cd /usr/local/ echo "This is a binary package." >> imod/4.5.3/readme.txt mv imod/4.5.3/bin/*-run . tar cvf imod-4.5.3-binary.tar imod/ gzip imod-4.5.3-binary.tar scp imod-4.5.3-binary.tar.gz cvllab1@115.146.94.0:/tmp/ mv *-run imod/4.5.3/bin/ The *-run files (used by CVL menu items) will be automatically generated by the CVL Build System. Each *-run file performs a “module load imod/4.5.3”, and then run the desired executable, e.g. /usr/local/imod/4.5.3/bin/3dmod

Using CVL Lab 1 instance to build CVL package for imod / 3dmod As root: cvlcreate imod 4.5.3 mv /tmp/imod-4.5.3-binary.tar.gz /usr/local/src/IMOD/4.5.3/ Now edit /usr/local/src/IMOD/4.5.3/config, as described here:https://confluence-vre.its.monash.edu.au/display/CVL/How+to+package+a+binary+application+and+submit+it+to+the+CVL+repository

Adding an online document link to the menu echo "google-chrome http://bio3d.colorado.edu/imod/doc/3dmodguide.html" > /usr/local/src/IMOD/4.5.3/userguide chmod a+x /usr/local/src/IMOD/4.5.3/userguide vi /usr/local/src/IMOD/4.5.3/check Add this line: cp ${SOURCE_DIR}/userguide ${INSTALL_DIR}/bin

Building the package mkdir /usr/local/tmp/imod_build cd /usr/local/tmp/imod_build/ cvlbuild Press ‘c’. Then, press ‘q’.

Building the package If running cvlbuild gives errors, check your config file very carefully for parse errors. If running cvlbuild succeeds, then run: cvlbuild –p to build an RPM. Now try installing the RPM (initially we will install locally, rather than using the CVL repository). yum install cvl-imod4_5_3-4.5.3-0.1.el6.x86_64.rpm

Testing the installation Start up a remote desktop (VNC) session to CVL Lab 1 (115.146.94.0) using the Launcher. I can see evidence of 3 mistakes. Mistake 1 I forgot to add the userguide menu item: “bin/userguide:3dmod userguide:help:false” into the MenuDesktop section of the /usr/local/src/IMOD/4.5.3/config file. NB: The MenuDesktop entry in the config file is a string, surrounded by quotes. Multiple menu items are separated by semicolons (;).

Testing the installation Mistake 2 – I forgot to add the environment variables’ initialization (e.g. setenv IMOD_DIR …) into the “EnvironmentModuleList” section of the config file. I determined this by noticing that neither of the 3dmod menu items worked. I checked the binary executables called by those menu items in /usr/share/applications/cvl-imod-4.5.3-3dmod.desktop etc. I tried running those executables from the command-line, and quickly discovered that a required environment variable (IMOD_DIR) had not been set. The required environment variables can be found in the /usr/local/Modules/modulefiles/imod/4.5.3 module initialization file on the CVL Demo instance (where I manually installed imod). The IMOD installation recommended adding an IMOD-linux.sh script to /etc/profile.d/ so that the environment variables would be available globally, but we only want them to be set when we do “module load imod”. When adding the environment variables to the “EnvironmentModuleList” section of the “config” file, remember that: The list of environment variable settings must begin with: EnvironmentModuleList=“ and end with a closing quote: “ All quotes (“) and dollar signs ($) within the EnvironmentModuleList string must be escaped using backslashes, i.e. \” and \$. All setenv lines except for the last one, should end with a semicolon (;) Don’t add any comment lines (#) within the EnvironmentModuleList string.

Testing the installation Mistake 3 – I forgot that by default, the 3dmod GUI (when run from the command-line), runs in the background – returning control to the Terminal immediately after launching. By default we run all our applications with Terminal windows for debugging, but in this case, the CVL menu system incorrectly assumes that the application has finished immediately after it is launched. The 3dmod GUI executable can accept a “-W” command-line option to workaround this problem, but we can’t add command-line arguments directly to the MenuDesktop line of the config file, so we need to create an executable “3dmod_minus_W” script in imod’s bin/ directory and rebuild the binary tarball. We will create 2 separate menu items for 3dmod – one using the “3dmod_minus_W” script which runs in a Terminal for debugging, and one using the standard “3dmod” executable, which doesn’t run in a Terminal. Choosing whether to run in a Terminal is done by specifying “true” or “false” in the MenuDesktop entry within the “config file”.

Removing the bad installation and rebuilding The CVL Build System uses the /usr/local/imod/4.5.3/ directory as part of the build/packaging process. We should remove the installed package before rebuilding. yum remove cvl-imod4_5_3 cd /usr/local/tmp/imod_build/ rm –fr * cvlbuild cvlbuild –p yum install cvl-imod4_5_3-4.5.3-0.1.el6.x86_64.rpm

It works!

Save /usr/local/src/IMOD/4.5.3/ to GitHub Using CVL-dev’s StructuralBiology repository First clone the repository from GitHub to your local computer, Then add the IMOD subdirectory to the StructuralBiology repository Then commit your changes and push them to GitHub. CVL-dev currently has one huge repository on GitHub for all StructuralBiology software. This is starting to become unmanageable – doing a “git clone” or a “git pull” from GitHub is very slow for such a large repository.

Adding package to CVL repository scp cvl-imod4_5_3-4.5.3-0.1.el6.x86_64.rpm cvlrepo@cvlrepo.massive.org.au:/home/cvlrepo/incoming Wait 10 minutes yum clean all yum list | grep cvl-imod cvl-imod4_5_3.x86_64 4.5.3-0.1 @/cvl-imod4_5_3-4.5.3-0.1.el6.x86_64 yum install cvl-imod4_5_3 Contact Jupiter Hu to request adding a package to a “yum groupinstall” group.

Time spent building this package and documenting the process. Total time taken, including documenting the steps in Powerpoint and correcting my mistakes: Approximately 3 hours (without GitHub) Hopefully much less with a bit more practice. However large (multi-gigabyte) applications will never be quick to package, even for expert users of the CVL Build System.

Acknowledgements NeCTAR Jupiter Hu Carlo Hamalainen Geoffrey Kong For funding the project. Jupiter Hu For developing the CVL Build System and for helping with troubleshooting. Carlo Hamalainen For trying out our CVL Build instructions on the CVL wiki and providing feedback. Geoffrey Kong For advice on which Structural Biology applications should be installed. Wojtek Goscinski and Paul McIntosh For steering the development of the CVL Build System.