1. German.Cancio@cern.ch quattor NCM components introduction tutorial German Cancio CERN IT/FIO

2. Quattor NCM tutorial – German.Cancio@cern.ch - n° 2 Contents u Preparation u NCM ‘theory’ n Components – what are they, how to run them n Packaging u NCM exercises Disclaimer: This is not a full quattor tutorial, but an introduction to the NCM framework. In particular, neither CDB/pan, nor SWRep/SPMA, nor AII are covered. A basic understanding of the PAN language is recommended.

3. Quattor NCM tutorial – German.Cancio@cern.ch - n° 3 Preparation u Login onto your ‘worker node’ (see whiteboard) as root u Please download the following tar.gz file, and uncompress it in your $HOME directory (~root): n cd ~ n wget http://cern.ch/quattor/documentation/tutorials/cern-0404/ncm- tutorial.tgzhttp://cern.ch/quattor/documentation/tutorials/cern-0404/ncm- tutorial.tgz n tar xvfz ncm-tutorial.tgz u Run the setup script: n ~/ncm-tutorial/bin/tutorial_setup.sh n should print [OK]

4. Quattor NCM tutorial – German.Cancio@cern.ch - n° 4 NCM environment CCM SPMA NCM Components Cdispd NCM Registration Notification SPMA SPMA.cfg CDB nfs http ftp Mgmt API ACL’s Client Nodes SWRep Servers cache Packages (rpm, pkg) packages (RPM, PKG) PXE DHCP Mgmt API ACL’s AII server DHCP handling KS/JS PXE handling KS/JS generator Node Install CCM Node (re)install?

5. Quattor NCM tutorial – German.Cancio@cern.ch - n° 5 What are components? (1/2) u “Components” (like SUE “features” or LCFG ‘objects’) are responsible for updating local config files, and notifying services if needed u Components do only configure the system Usually, this implies regenerating and/or updating local config files (eg. /etc/sshd_config ) u Use standard system facilities (SysV scripts) for managing services n Components can notify services using SysV scripts when their configuration changes. u Components can be run n manually n via hooks (cron, boot time, etc) n via CERN-CC’s notification framework (notd) n automatically: register their interest in configuration entries or subtrees, and get invoked in case of changes (disabled at CERN-CC) u Possible to define configuration dependencies between components n Eg. configure network before sendmail n Components won’t run if a pre-dependency is unsatisfied (eg. failing prerequisite component)

6. Quattor NCM tutorial – German.Cancio@cern.ch - n° 6 What are components? (2/2) u Components are written as Perl OO class instances u Each component has to provide two methods: u Configure(): n invoked when there was a CDB configuration change or on startup n Mandatory method u Unconfigure(): n invoked when a component is to be removed n Optional method – most of the components won’t need to implement it.

7. Quattor NCM tutorial – German.Cancio@cern.ch - n° 7 How to run components?  ncm-ncd (Node Configuration Deployer): n framework and front-end for executing components (via cron, cdispd, or manually) n dependency ordering of components  cdispd (Configuration Dispatch Daemon) Monitors the config profile, and invokes registered components via ncm-ncd if there were changes n not described in this tutorial. u ncm-query n Tool for examining configuration information as cached on the node

8. Quattor NCM tutorial – German.Cancio@cern.ch - n° 8 Component (simplified) example sub Configure { my ($self,$config) = @_; # access configuration information my $arch=$config->getValue('/system/architecture’); # NVA API $self->Fail (“not supported") unless ($arch eq ‘i386’); # (re)generate and/or update local config file(s) open (myconfig,’/etc/myconfig’); … # notify affected (SysV) services if required if ($changed) { system(‘/sbin/service myservice reload’); … }

9. Quattor NCM tutorial – German.Cancio@cern.ch - n° 9 Components and CDB profile access NVA API: configuration access library u This library allows to access the node profile’s configuration (hierarchical structure) u Most popular methods: n $value=$config->getValue(‘/system/kernel/version’); n if ($config->elementExists($path)) {…} else {…} n $element=$config->getElement($path); while ($element->hasNextElement()) { my $newel=$element->getNextElement();... }

10. Quattor NCM tutorial – German.Cancio@cern.ch - n° 10 Component support libs u Reporting functions: $self->log(@array): write @array to component’s log file $self->report(@array): write @array to log and stdout. $self->verbose(@array), $self->debug(level,@array): verbose/debug output $self->warn(@array): writes a [WARN] message, increases # of warnings $self->error(@array): writes an [ERROR] message, increases # of errors Failures of reconfigurations are done using ‘ error(..) ’. Components depending on a failed component are not executed. u Advanced support libraries available (revamped from CERN’s SUE): n Configuration file manipulation n Advanced file operations n Process management n Exception management libraries See /opt/edg/lib/perl/LC for details …

11. Quattor NCM tutorial – German.Cancio@cern.ch - n° 11 Real component walktrough u ncm-state (/usr/lib/perl/NCM/Components/state.pm)/usr/lib/perl/NCM/Components/state.pm Updates a local configuration file (/etc/state.desired ) with the node’s production state (“production”, “standby”, etc.)

12. Quattor NCM tutorial – German.Cancio@cern.ch - n° 12 Packaging components: files (1/3) u Each component is packaged independently and kept in a separate CVS area. n Let’s check our example: please cd to ~/ncm-tutorial/component-cvs/ncm-state u Files to be present in a component CVS module: n README – small intro n ChangeLog – automatically maintained ChangeLog file. n LICENSE – contains license or pointer to it n MAINTAINER – one liner with email n Makefile – copied from quattor-build-tools, essentially an ‘include’ of the quattor-build-tools

13. Quattor NCM tutorial – German.Cancio@cern.ch - n° 13 Packaging components: files (2/3) n config.mk – obligatory and optional definitions s COMP – component name s DESCR – one liner with component description s VERSION – in the format.. s RELEASE – RPM release number (always 1 for time being  ) s AUTHOR – author’s email s DATE – dd/mm/yy hh:mm s Any other optional definition(s) n comp.pm.cin –component source file (a la ‘autotools’) s Source format! @TAGS@ get expanded into configuration variable values s Configuration variables set in config.mk or predefined inside quattor-build- tools (quattor-Linux.mk, quattor-SunOS.mk) n comp.pod.cin – POD doc file. Please follow conventions: s NAME s SYNOPSIS: Configure() and Unconfigure() documentation s RESOURCES: describe all used resources s DEPENDENCIES: ‘pre’ and ‘post’ dependencies s BUGS s AUTHOR s SEE ALSO n pro_declaration_component_comp.tpl.cin – PAN component template

14. Quattor NCM tutorial – German.Cancio@cern.ch - n° 14 Packaging components: files (3/3) u Extra data files (template config files, etc): Can be stored in a subdirectory called ‘DATA’ n Example: see ncm-state/DATA n The file is copied into the directory @NCM_DATA_COMP@ (typically, /usr/lib/ncm/config/name/) u Extra documentation files: Can be stored in a subdirectory called ‘DOC’ n The file is then dropped into the standard RPM package documentation directory u Extra template files: Can be stored in a subdirectory called ‘TPL’ n All files are then dropped into the standard RPM package doc directory n The pro_declaration_component… template can be stored either in the top level directory or in the ‘TPL’ directory n All templates are copied to a common directory as well (QTTR_DOCDIR/pan-templates, typically /usr/share/doc/pan-templates)

15. Quattor NCM tutorial – German.Cancio@cern.ch - n° 15 Packaging components: commands u Generating packages out of checked out sources n ‘make rpm’ – generates RPM (on RHLinux) n ‘make pkg’ – generates PKG (on Solaris) u ‘make (release|minorversion|majorversion)’ – generate new version n checks in modified files to CVS n Prompts for ChangeLog entry (one line) n Increases release / minorversion / majorversion in config.mk n Generates a new CVS tag for the component n Note: ensure your files are declared in CVS (ie. ‘cvs add’) u make clean n Removes temp files generated during package build process

16. Quattor NCM tutorial – German.Cancio@cern.ch - n° 16 Exercises

17. Quattor NCM tutorial – German.Cancio@cern.ch - n° 17 Exercise environment (I) The demo setup has been simplified compared to the real production environment. u Only base component functionality is showed in the exercises. n The rest is left as exercise to the students ! u No “real” CDB client/server flow n … but wrapper script provided for generating and updating profiles locally

18. Quattor NCM tutorial – German.Cancio@cern.ch - n° 18 Exercise environment (II) u Documentation: See http://quattor.org/documentation/tutorialshttp://quattor.org/documentation/tutorials -> CERN for this slides (and solutions) u Man pages: n ncm-query(1), ncm-ncd(1) n ncm-component(8)

19. Quattor NCM tutorial – German.Cancio@cern.ch - n° 19 Exercise environment (III) u Checking the ncm tutorial directory contents: n ~/ncm-tutorial/bin: s cdb_wrapper.sh: wrapper for running ‘pan’ and CCM s pan.rh7: ‘pan’ binary – (do not use directly) n ~/ncm-tutorial/templates: s *.tpl: the exercise ‘pan’ template files for our NCM configurations n ~/ncm-tutorial/component-cvs: s quattor-build-tools: tools for building packages s ncm-state: our example component n ~/ncm-tutorial/xml: s Temporary dir for XML profiles generated by cdb_wrapper.sh

20. Quattor NCM tutorial – German.Cancio@cern.ch - n° 20 NCM exercises First: Quick intro to the demo NCM CBD/pan template structure and contents Exercises: 1. Compile default profile 2. Modifying the profile 3. Running a component and changing its configuration 4. Extending a component (ncm-state)

21. Quattor NCM tutorial – German.Cancio@cern.ch - n° 21 CDB templates for NCM components u Please cd into ~/ncm-tutorial/templates u The templates provided serve the configuration of 3 NCM components (see next two slides) u Every time a template is updated: ‘pan’ and CCM need to be invoked using a wrapper to use the right directories and includes. ~/bin/cdb_wrapper.sh

22. Quattor NCM tutorial – German.Cancio@cern.ch - n° 22 profile_dummy pro_default_tutorial Template hierarchy for NCM config tutorial Types & structures Templates to be edited pro_declaration_component_type grubstatetestcomp pro_declaration_component_ pro_declaration_system data templates “cluster” or “service” level config values Node level config values. Inherited values can be overwritten

23. Quattor NCM tutorial – German.Cancio@cern.ch - n° 23 CDB templates for NCM: contents u $ less pro_default_tutorial.tpl u Components can have “private” configuration entries: /software/components/name/active (bool) dependencies/pre (string[]) dependencies/post(string[]) foo/... (component specific) bar/... (component specific) u Components can access configuration information anywhere in the node profile (/system/.., /software/.., /hardware/..) n Useful to share common configuration entries between components n Eg. /system/state, /system/kernel/version u All components need to declare their “private” config data types n pro_software_component_type: common definitions n pro_software_component_name: added definitions

24. Quattor NCM tutorial – German.Cancio@cern.ch - n° 24 Exercise 1 Generate default profile, run NCM i. ‘ cd ’ into ~/ncm-tutorial/templates ii. Generate the profile with../bin/cdb_wrapper.sh iii. Use ncm-query for visualizing the configured component list, and to visualize all configuration information: ncm-query –-components ncm-query –-dump / iv. Run now all active components: ncm-ncd –-configure (logfiles are generated under /var/log/ncm/..) v. Run now a specific component: eg. ncm-ncd –-configure testcomp What other component is run, and why? Note: ncm-ncd –-help and ncm-query –-help provide you with all options and default values. Try also man ncm-ncd or ncm-query.

25. Quattor NCM tutorial – German.Cancio@cern.ch - n° 25 Exercise 2 Modify default profile, run NCM i. Modify the default profile to deactivate ‘testcomp’, re-run cdb_wrapper, re-run ncm-ncd –-configure ( modify the pro_default_tutorial template ) ii. What happens if ‘grub’ is deactivated in the profile, but ‘testcomp’ is activated, and ncm-ncd is run?

26. Quattor NCM tutorial – German.Cancio@cern.ch - n° 26 Exercise 3 Modify a component’s configuration i. Read the component’s man page for ‘grub’. man ncm-grub ii. Examine which kernels are installed on your node, and modify the kernel version to be used on the next reboot What kernel is currently configured? (using ncm-query --dump ) Available kernels: cat /etc/grub.conf or ls /boot/vmlinuz* Don’t reboot the node! :-) iii. Try out what happens if you configure a non-existing kernel. iv. What happens if you run ncm-ncd –-unconfigure grub and why? v. Restore the previously configured kernel. Note: The component code is found under /usr/lib/perl/NCM/Component/grub.pm

27. Quattor NCM tutorial – German.Cancio@cern.ch - n° 27 Exercise 4 Modify the ncm-state component to add a ‘reason’ field to /etc/state.desired 1. Add a ‘reason’ field to the state NCM declaration template 2. Add the wanted value to the ‘data’ template 3. Regenerate the profile and verify with ncm-query that the new field is there 4. Add handling code to the component source to read out the new field, and to update /etc/state.desired 5. Generate the ncm-state rpm after increasing the ‘minorversion’ (not manually) 6. install the rpm (with rpm –Uvh - in the ‘real’ world this would be done via SPMA) 7. Run the component, and verify that /etc/state.desired contains the new data field 8. Did you forget to ‘activate’ the component?

28. Quattor NCM tutorial – German.Cancio@cern.ch - n° 28 http://quattor.org