1. SPMA & SWRep: Basic exercises HEPiX hands-on, NIKHEF 5/03 German Cancio http://cern.ch/wp4-install

2. HEPiX hands-on / Installation Task / German Cancio CERN - n° 2 Agenda u Exercise environment u Software Repository exercises u SPMA exercises

3. HEPiX hands-on / Installation Task / German Cancio CERN - n° 3 Exercise environment (I) The demo setup has been simplified compared to the real production environment. 1 server node hosting a single SWRep ( lap-163.nikhef.nl ) n N client nodes (one per alumni pair) u Only base SPMA/SWRep functionality is showed in the exercises. n The rest is left as exercise to the students ;-)  SWRep to CDB communication (for inventory) simulated with NFS mounts from lap-163.nikhef.nl to each client node u Local configuration cache (CCM) has to be updated manually n Wrapper script provided

4. HEPiX hands-on / Installation Task / German Cancio CERN - n° 4 Mapping between usernames and student nodes lap-163 TEACHER lap-183 spmuser13 lap-189 spmuser9 lap-181 spmuser10 lap-193 spmuser5 lap-152 spmuser6 lap-166 spmuser1 lap-195 spmuser2 lap-162 spmuser14 lap-180 spmuser15 lap-156 spmuser11 lap-178 spmuser12 lap-184 spmuser7 lap-174 spmuser8 lap-199 spmuser3 lap-169 spmuser4 door

5. HEPiX hands-on / Installation Task / German Cancio CERN - n° 5 Exercise environment (II) u Preparing for the exercises: n Please log in to your client node (see username/password on blackboard), and open (at least) two windows: 1. A root window (needed for SPMA execution) 2. A normal user window ( spmuserX ) (needed for ‘pan’ compilation and SWRep manipulation) u Hands-out include: n This slides n A printed copy of the SPMA and SWRep client man pages u Solutions: n http://lap-163.nikhef.nl/Solutions.txt http://lap-163.nikhef.nl/Solutions.txt

6. HEPiX hands-on / Installation Task / German Cancio CERN - n° 6 Exercise environment (III) u Checking the home directory contents: ~spmuserX/bin :  pan_wrapper.sh : wrapper for invoking pan with appropriate includes  spma_wrapper.sh : wrapper for invoking CCM, SPMA config generator and SPMA  pan.rh7 : ‘pan’ binary ~spmuserX/RPMS :  RPMS/: contains RPM demo packages, used for the exercises  SPECS/: the corresponding.spec files (not used in this demo) ~spmuser/templates :  *.tpl : the ‘pan’ template files for the SPMA configs  xml/: directory where generated XML profiles will be stored

7. HEPiX hands-on / Installation Task / German Cancio CERN - n° 7 Software Repository Exercises u Quick introduction to edg-swrep-client u Adding and removing packages to the SWRep u Querying the SWRep for information u Area (ACL) management: adding, removing, changing areas for packages

8. HEPiX hands-on / Installation Task / German Cancio CERN - n° 8 SWRep client interface u Edg-swrep-client is the SWRep client interface n /opt/edg/bin/edg-swrep-client Run edg-swrep-client --help for setup help Run edg-swrep-client –-list-commands for the list of commands u Most important commands: n Package upload and deletion: s put s pull s remove n Repository query: s query s find s listplatforms n “Area” (==ACL) query and manipulation: s chtag s addarea s rmarea

9. HEPiX hands-on / Installation Task / German Cancio CERN - n° 9 Setup verifications u SWRep to client interface /opt/edg/bin/edg-swrep-client listrights should produce something like: Areas: The SWRep ACL mechanism u Every student has a ‘private’ area (/hepix/userX), and there is a shared area (/hepix/common) u Note that each area determines who can modify which package, but the areas do not represent directories (rather hierarchical ACL tags) u Sub-areas can be created (ACL rights are cumulative) $ /home/spmuserX > /opt/edg/bin/edg-swrep-client listrights You are spmuserX, with rights to change packages with tags: /hepix/common /hepix/userX You have rights to remove packages

10. HEPiX hands-on / Installation Task / German Cancio CERN - n° 10 SWRep exercise 1 Exercise 1: Add / remove packages to/from Software Repository i. Add the package ~spmuserX/RPMS/RPMS/spma-test4-userX-0-1.i386.rpm to the SWRep, to your private “area”, for the platform i386_redhat73. use the ‘put’ command (not: pull) of edg-swrep-client for package uploads ii. Remove that package from the SWRep. n use the ‘remove’ command iii. what happens if you try to add/delete a package which is not in your private ‘area’? Hints: Use the ‘ edg-swrep-client ’ man page! Check which is your private area with the ‘ listrights ’ command

11. HEPiX hands-on / Installation Task / German Cancio CERN - n° 11 SWRep exercise 2 Exercise 2: querying the SWRep for information i. list all platforms available on the SWRep. ii. list all packages belonging to the i386_redhat73 platform. iii. find all packages matching ‘ kernel ’ on the SWRep iv. get the package information for the ‘ kernel ’ and ‘ kernel-smp ’ packages. Hints: n Read the man page… ;-)

12. HEPiX hands-on / Installation Task / German Cancio CERN - n° 12 SWRep exercise 3 Exercise 3: Area management i. move one (pick one) of your packages to the /hepix/common area. How can you verify if somebody else can delete/modify this package? ii. Add a new area called ‘test’, which is a sub-area of your private area (/hepix/userX/test ). Can you add further sub-areas to this sub- area? iii. Remove the area created in B). Does it also remove sub-areas of the created area? iv. Create a sub-area as in B), put some packages on it, and rename the whole sub-area to a different name. Hints: Use the ‘chtag’, ‘addarea’, ‘rmarea’, and ‘charea ’ commands of edg-swrep-client

13. HEPiX hands-on / Installation Task / German Cancio CERN - n° 13 SPMA exercises u Quick introduction to the PAN package templates u Adding/removing packages to the node u Local packages, and user priority switch u Handling simultaneous package versions with the SPMA

14. HEPiX hands-on / Installation Task / German Cancio CERN - n° 14 SPMA setup: package templates (I) u The Pan templates for the SPMA contain the list of packages to be installed by the SPMA. pro_software_packages_hepix.tpl and n pro_software_packages_asis.tpl u By overlaying the templates using ‘include’, we get the desired list of packages in the XML. n see next slide u ‘pan’ is invoked using a wrapper to use the right directories and includes: n ~/bin/pan_wrapper.sh

15. software_ types functions Pro_software_ packages_hepix / asis includes valuecreates repository_cern_cc_ I386_redhat73 Pro_software_hepix dummynode Pro_default_hepix Object templates (generate XML) (external object template to speed up compilation, but no XML generated) Template structure for Software packages – HEPiX simplified setup External (generated by SWRep) ‘read-only’ structural templates editable templates Object templates (XML)

16. HEPiX hands-on / Installation Task / German Cancio CERN - n° 16 SPMA setup: package templates (II) Some functions are provided for manipulating the package list in the profile, which are used in the templates: u pkg_add(“packagename”,”version-release”,”arch”); n Adds a package to the profile. u pkg_del(“packagename”,[“version-release”]); n Removes a package version (or all if none specified) from the profile. u pkg_repl(“packagename”,”new ver-rel”,”arch”[,”old ver-rel”]); n Replaces the package version ‘old’ by ‘new’ in the profile. If ‘old’ is not specified, it replaces all other versions. u It is important to understand that the ‘add’,’del’ and ‘repl’ functions do only modify the final list of desired packages. n Eg. ‘pkg_del’ does not instruct to delete any package, but removes it from the list of desired packages n Useful when modifying inherited profiles u For the exercises, we will use only the ‘pkg_add’ function.

17. HEPiX hands-on / Installation Task / German Cancio CERN - n° 17 SPMA setup: command line interface u ensure SPMA runs in ‘light’ mode (not managing all packages on the node, but only a subset) /opt/edg/bin/spma --help should include: u Most important options: n --noaction n --verbose n --userpkgs (yes/no) [run in ‘light’ or ‘full’ mode] n --userprio (yes/no)  In this tutorial, the SPMA is invoked using a wrapper ( ~/bin/spma_wrapper.sh ) which passes the options trough. The wrapper is necessary for updating the CCM cache and generating the SPMA configuration file out of the ‘pan’ XML profile. --userpkgs (default: 'yes') Allow user (== non-SPMA) installed packages

18. HEPiX hands-on / Installation Task / German Cancio CERN - n° 18 SPMA exercise 1 Exercise 1: Adding/removing packages to the node i. As spmuserX, edit the ‘ templates/pro_software_packages_hepix.tpl ’ template (with your favorite editor), and add to it the following package: n GNU.LANG_gcc-alt-2.95.2-6.i386.rpm ii. Save the template, and compile the XML profile with: ~/bin/pan_wrapper.sh ~/templates/dummynode.tpl iii. Now, use the SPMA wrapper to update the package list (downloading and installing the missing package), in dry run mode: (as root!) ~/bin/spma_wrapper --noaction iv. And now run the SPMA: (as root!) ~/bin/spma_wrapper

19. HEPiX hands-on / Installation Task / German Cancio CERN - n° 19 SPMA exercise 1. (contd) v. What happens if you re-run the SPMA again? vi. What happens if you remove the package by hand ( rpm -e GNU.LANG_gcc-alt-2.95.2 ), and run the SPMA afterwards? vii. Now comment out the line added to the template in i.) and add another line for adding the following package: n JAVA_colt-1.0b3-0.i386.rpm and re-run the SPMA. What happens? viii. Which packages would be added if the template ‘ pro_software_packages_asis.tpl ’ is include ’d into ‘pro_software_hepix.tpl’? Notes: The generated SPMA local configuration file (containing the ‘target’ package list) is kept in /var/lib/spma-target.cf The list of packages managed by the SPMA is stored in /var/lib/spma-managed-packages SPMA log file: /var/log/spma.log Note: bug triggered during presentation fixed in edg-spma-1.0.4!

20. HEPiX hands-on / Installation Task / German Cancio CERN - n° 20 SPMA exercise 2. Exercise 2: Local packages and user priority switch i. Install (by hand) the following package: GNU.LANG_gcc-alt-3.2.3 (as root) rpm -Uvh http://lap-163.nikhef.nl/swrep/i386_redhat73/GNU.LANG_gcc-alt-3.2-3.i386.rpm This package will be considered as ‘local’ or ‘managed by the user’. What happens if the SPMA is told to use a different version? Edit ‘pro_software_packages_hepix.tpl’, and add a line for gcc 2.95: ”/software/packages”=pkg_add(“GNU.LANG_gcc-alt”,”2.95.2-6”,”i386”); Recompile the profile and re-run the SPMA ~/bin/pan_wrapper.sh, then as root: ~/bin/spma_wrapper.sh The package will be downgraded since the SPMA gives by default priority to the SPMA selected packages. But this can be changed…

21. HEPiX hands-on / Installation Task / German Cancio CERN - n° 21 SPMA exercise 2 (contd.) ii. Edit the SPMA config file /etc/spma.conf (as root) for enabling priority to user packages: uncomment the line # userprio = no and change it to userprio = yes Reinstall the GNU.LANG_gcc-alt-3.2.3 package (see previous slide) rpm -Uvh http://xxxx.nikhef.nl/swrep/i386_redhat73/GNU.LANG_gcc-alt-3.2-3.i386.rpm Recompile the profile and re-run the SPMA as root: ~/bin/spma_wrapper.sh This time, the user installed package will be kept, as the user version has priority over the SPMA selected version.

22. HEPiX hands-on / Installation Task / German Cancio CERN - n° 22 SPMA exercise 3 Exercise 3: multiple versions of packages How to install both gcc 3.2 and 2.95 simultaneously? The functions for package management allow to specify an optional parameter for this: ”/software/packages”=pkg_add(“GNU.LANG_gcc-alt”,”2.95.2-6”,”i386”,”1”); ”/software/packages”=pkg_add(“GNU.LANG_gcc-alt”,”3.2-3”,”i386”,”1”); The last parameter means: ‘do not remove other versions of the package in the profile’) (See the pro_declaration_functions.tpl file for more info) i. Add the above two lines to your pro_software_packages_hepix.tpl template, regenerate the profile, and run the SPMA. What happens? (check with rpm –qa the list of installed packages) Note: for sure, having multiple versions requires that the files of the versions do not conflict!

23. HEPiX hands-on / Installation Task / German Cancio CERN - n° 23 Other SPMA exercises Exercises to the students (studying the manual page): u Using the SPMA in full mode – no user packages (careful!) u Running in “update only” mode u Using the package local cache u Study the package flags (not available in this demo setup) u Using other protocols than HTTP (not available in this demo setup) Your feedback is most welcome for further developments!!!