Managing Ensembles Nilesh M. Bhide
System Access Models The Stand-alone System –Beowulf system unattached to any external network The Universally Accessible Machine –Every node is accessible from the entire Internet The Guarded Beowulf –A single front-end (“worldly node”)
Assigning Names Internal host names of the format – Reserved IP addresses – – – Dynamically assigned Address –DHCP, BOOTP
Cloning Nodes Configure one internal node and clone rest of them Advantages –Quick and easy configuration of internal nodes –Facilitates major system upgrades –Easy recovery from disk failures, accidental file system corruption
Cloning Nodes (contd.) Steps involved –Manual configuration of a single internal node Install NFS automounter –Creation of tar images for each partition omit /proc, which is not a physical disk partition tar zclf /worldly/nfsroot/partition-name.tgz.
Cloning Nodes (contd.) –Setting up a clone root partition Create a root directory for cloning on the worldly node. This should we exported via NFS This directory should contain bin, dev, etc, lib, mnt, proc, sbin, tmp subdirectories –tar -C / -c -f - dev | tar xf - –tmp and etc subdirectories should be empty –etc/fstab enrty : “none /proc proc default 0 0” Replace NFS root sbin/init script Export NFS root directory
Cloning Nodes (contd.) –Setting up BOOTP set up /etc/bootptab file.default:\ :sm= :\ :ht=ether:\ :gw= \ :rp=/export/nfsroot/: b002:ip= :ha=0080c8638a2c=.default b003:ip= :ha=0080c86359d9=.default Enable bootpd daemon on the worldly node –/etc/inetd.conf
Cloning Nodes (contd.) –Building a boot clone floppy CACR Beowulf cloning software – Building NFSROOT kernel –/usr/src/linux –mknod /dev/nfsroot b –rdev zImage /dev/nfsroot –dd if=zImage of=/dev/fd0 bs=512
Basic System Administration Booting and Shutting Down –/sbin/shutdown -h now (using prsh) –ATX power supplies /sbin/shutdown -p now The node file system –/ :The root partition, containig system configuration and log files –/boot : An optional partition for storing kernel images
Basic System Administration –/home : A partition containing all user directories –/opt : An optional partition for additional software –/usr : A partition containing all standard system software –/scratch : A partition used as scratch space for large temporary data files
Basic System Administration Account Management –NIS, NIS+ –useradd, adduser, linuxconf –Replicating /etc/passwd, /etc/group –NIS vs. Replication
Basic System Administration PRSH - Parallel Remote Shell – –e.g., prsh -- killall amok
Security Strategies CERT: System Configuration –Stand-alone, Universally accessible, Guarded IP Masquerading –NAT (Network Address Translation) –Client node set up route add default gw or /etc/sysconfig/network configuration ( GATEWAY, GATEWAYDEV )
Security Strategies (Contd.) – Worldly node set up IP Masquerade HOWTO in /usr/doc/HOWTO/mini FORWARD_IPV4=true in /etc/sysconfig/network Configure IP masquerading rules using ipfwadm –ipfwadm -F -p deny –ipfwadm -F -a masq -S /24 -D /0 –/etc/rc.d/rc.local or /etc/rc.d/init.d ipfwadm-wrapper for 2.2 Linux kernels
Security Strategies (Contd.) Restricting host access –TCP wrappers package –tcpd daemon invoked by inetd and listed in /etc/inetd.conf –/etc/hosts.deny and /etc/hosts.allow e.g., in.rshd: / – Placing worldly node behind firewall SSH:
Job Scheduling Keeps user-developed applications from interfering with each other. Currently no standard exists for job schedulers BPROC (Beowulf Distributed Process Space) –