on behalf of the NRC-KI Tier-1 team Experience of puppet usage for management of Tier-1 GRID cluster at NRC “Kurchatov Institute” Igor Tkachenko on behalf of the NRC-KI Tier-1 team National Research Centre "Kurchatov Institute“ Moscow, Russian Federation

Introduction Puppet - is a condition-based configuration system Simple ruby-based language of manifests and templates NRC KI Tier-1 site use puppet to manage about 200 nodes No limitation for increasing number of managed nodes

Service configuration example Service configuration - is announcement a desired state like: “I want have running ssh service with configuration file sshd_config.$hostname” class sshd { package { "openssh-server": ensure => installed } file { "/etc/ssh/sshd_config": mode => 600, notify => Service["sshd"], require => Package["openssh-server"], source => "puppet://$site_settings::master/sysconf/ssh/sshd_config.$hostname", sourceselect => 'first' } service { "sshd": enable => true, ensure => 'running', require => [ File["/etc/ssh/sshd_config"], Package["openssh-server"], ],

Security Host-based and Certificate-based authorization Puppet-CA Host-based private area [x509] path /home/t1master/x509/%H allow * Group-based private area path ~ ^/file_(metadata|content)/private/bdii-site auth yes method find, search allow /^taxer[12].t1.grid.kiae.ru$/

Daemon vs manual launch Daemon mode Manual launch All manifests stored on master-node (single point of failure) Manifests applied automatically – all errors will be applied on all servers Hard to debug Every node ask master about updates every 30 minutes Every node have own copy of all manifests You may easy debug manifests on node and it does not affected to others Fully controlled updates Easy to configure for several sources of manifests One time sync

How to use manual launch Needed full repository of puppet modules with classes declaration on each node Two ways to apply: Apply class: puppets apply –e “include some_class” Apply file Need applied file that contain include directive puppet apply “my_file.pp”

Private data Some services may require private data in manifests (passwords, for example) There are several solutions: Parsing configuration files from private area Dummy classes in public area and classes with private data in special private area. Mix it while sync process

Sync example class sync inherits site_settings { File { owner => 'root', group => 'root', mode => 0600 } file { "/etc/puppet/": source => [ "puppet://$site_settings::master/private/$site_settings::host_type/puppet", "puppet://$site_settings::master/puppet/", ], ensure => 'directory', recurse => true, purge => true, force => true, ignore => ['.svn','.*.swp'], sourceselect => 'all',

Class-based and type-based configurations Two ways to configure service Class-based Type-based Simple Unclean order of applying types (but completely defined where it needed) Allow include other puppet classes Create ruby script for configuration Useful when you have to execute some scripts for service configuration

Puppet + pdsh A lot of manual puppet runs need mechanism for parallel running Group-based pdsh with groups like puppet groups allow to apply manifests exactly where you need it Typical usage pdsh –g wn ‘puppet apply –e “include sync”’ pdsh –g wn ‘puppet apply –e “include yum::install”’

Groups definition example class site_settings::groups inherits site_settings { case $fqdn { /^io\..*/ : { $group='eos-head' } /^sdns.*/ : { $group='dcache-namespace' } /^sddb[0-9]*\..*/ : { $group='dcache-db' } /^sdpm\..*/ : { $group='dcache-poolmanager' } /^sdhs\..*/ : { $group='dcache-head' } /^sdrm\..*/ : { $group='dcache-srm' } /^sdpd[0-9]+\..*/ : { $group='dcache-dpool'} /^sespd[0-9]+\..*/ : { $group='dcache-spool-x2'} /^seipd[0-9]+\..*/ : { $group='dcache-ipool'} /^lhcbsdns\..*/ : { $group='lhcbdcache-namespace' } /^lhcbsddb[12]\..*/ : { $group='lhcbdcache-db' } /^lhcbsdpm\..*/ : { $group='lhcbdcache-poolmanager' } /^lhcbsdhs\..*/ : { $group='lhcbdcache-head' } /^lhcbsdrm\..*/ : { $group='lhcbdcache-srm' } /^lhcbseipd[0-9]+\..*/: { $group='lhcbdcache-ipool'} /^tape-sdrm.*/: { $group='dcache-tape-head'} /^sd-p-t[0-9]+\..*/: { $group='dcache-tpool'} /^sdpe[0-9]+\..*/ : { $group='eos-dpool' } /^pit\..*/ : { $group='nfs' } /^seer\..*/ : { $group='scheduler'} /^orb\..*/ : { $group='nagios' } /^n[12][0-9][0-9]\..*/: { $group='wn' } /^taxer[12]\..*/: { $group='bdii-site' } /^horn[12]\..*/: { $group='bdii-top' } /^orange\..*/: { $group='apel'} /^calc[12]\..*/: { $group='cream-ce'} /^(rnag|rnag-sb)\..*/: { $group='ngi-nagios'} /^endb[12]\..*$/ : { $group='enstore-db' } 'enh.t1.grid.kiae.ru' : { $group='enstore-head' } 'rhole.t1.grid.kiae.ru' : { $group='vobox-alice' } default: { $group='test' } }

Questions, comments?