Customizing a Cluster

What We’ll Be Doing Discuss how to add new packages to compute nodes Discuss frontend and compute node partitioning Discuss how to customize a kernel

Add A New Package

Adding a New Package to the Distribution All packages are found under ‘/home/install’ Put the new package in contrib/enterprise/3/en/os/public/ /RPMS  Where is ‘i386’, ‘x86_64’ or ‘ia64’ “Extend” an XML configuration file Rebind the distro:  “rocks-dist dist” Apply the changes:  “shoot-node compute-0-0”

Extend the “Compute” XML Configuration File To add the package named “my-app” $ cd /home/install/site-profiles/3.2.0/nodes $ cp skeleton.xml extend-compute.xml In ‘extend-compute.xml’, change: To: my-app

Extend the “Compute” XML Configuration File Rebind the distro  This copies ‘extend-compute.xml’ into /home/install/rocks-dist/…/build/nodes # cd /home/install # rocks-dist dist Reinstall a compute node # shoot-node compute-0-0

More on the Distro Rocks-dist looks for packages in:  “/home/install/ftp.rocksclusters.org” Red Hat and Rocks packages  “/home/install/contrib” Pre-built 3rd party packages  “/usr/src/redhat/RPMS” Your packages

More on the Distro Any time you add a package to the distro, you must re-run “rocks-dist dist”  Rocks-dist binds all the found packages into a Red Hat-compliant distribution

More on the Distro Rocks-dist assembles a Red-Hat compliant distribution

Your Distro - Extending Rocks You can use “rocks-dist” to build and distribute your own distribution  Merges RPMS  Resolves versions Final distribution looks just like Rocks  And, Rocks looks just like Red Hat

Custom Partitioning

Default Frontend Partitioning 6 GB for / 1 GB swap Rest of first drive is /export

Modifying Frontend Partitioning Can only change during frontend installation Note: must have ‘/export’  /export is automatically mounted by all compute nodes

Default Compute Node Partitioning 6 GB for / on first disk 1 GB for swap on first disk Remainder of first disk  Partitioned as “/state/partition1” Second disk  Partitioned as “/state/partition2” Third disk, fourth disk, etc.  Partitioned as “/state/partition3”, “/state/partition4”, etc. All non-root partitions are saved over reinstalls

Changing Size of Root and Swap on a Compute Node If just want to change size of root and swap, only need to change two variables Create the file “extend-auto-partition.xml” # cd /home/install/site-profiles/3.2.0/site-nodes/ # cp skeleton.xml extend-auto-partition.xml Above the “ ” section, add the two variables Above XML variables will create a 10 GB root partition and a 2 GB swap partition Rebind the distro (rocks-dist dist) and reinstall a compute node (shoot-node compute-0-0)

Specifying a New Partition Layout Only requirement is that ‘/’ is “big enough” Create the file “extend-auto-partition.xml” # cd /home/install/site-profiles/3.2.0/site-nodes/ # cp skeleton.xml extend-auto-partition.xml In the “ ” section, add: / --size ondisk hda swap --size ondisk hda /mydata --size 1 --grow --ondisk hda Rebind the distro (rocks-dist dist) and reinstall a compute node (shoot-node compute-0-0)

Specifying Software RAID Create the file “extend-auto-partition.xml” # cd /home/install/site-profiles/3.2.0/site-nodes/ # cp skeleton.xml extend-auto-partition.xml In the “ ” section, add: / --size ondisk hda swap --size ondisk hda raid.00 --size= ondisk hda raid.01 --size= ondisk hdb /mydata --level=1 --device=md0 raid.00 raid.01 Rebind the distro (rocks-dist dist) and reinstall a compute node (shoot-node compute-0-0)

Testing Changes Use:  “./kickstart.cgi --client=compute-0-0 > /tmp/ks.cfg” Should not see any output  That is, no error output The file “/tmp/ks.cfg” should contain your changes

Build New Kernel

Building a New Kernel from Red Hat’s Kernel Source Recommend you build on a compute node  Don’t want to pollute (or destroy!) the frontend Go the directory that contains the source # cd /usr/src/linux-2.4 Get a default ‘.config’ file # cp configs/kernel i686-smp.config.config Customize.config Make a kernel RPM # make rpm

Building a New Kernel from Red Hat’s Kernel Source Copy the new kernel RPM back into the frontend’s distribution # scp /usr/src/redhat/RPMS/i686/kernel-smp i686.rpm \ frontend-0:/home/install/contrib/enterprise/3/public/i386/RPMS/ Login back into the frontend Rebind the distro (rocks-dist dist) and reinstall a compute node (shoot-node compute-0-0)

Building a New Kernel from kernel.org’s Source On a compute node, get the Linux kernel tarball # cd /usr/src # wget Prepare the source tree # bunzip2 linux tar.bz2 # tar vfx linux tar # rm linux-2.4 # ln -s linux linux-2.4 Create a “.config” file and put it in “/usr/src/linux-2.4”

Building a New Kernel from kernel.org’s Source Build a kernel RPM # cd /usr/src/linux-2.4 # make rpm Copy the new kernel RPM back into the frontend’s distribution # scp /usr/src/redhat/RPMS/i686/ kernel-smp i686.rpm \ frontend-0:/home/install/contrib/enterprise/3/public/i386/RPMS/ Login back into the frontend Rebind the distro (rocks-dist dist) and reinstall a compute node (shoot-node compute-0-0)

Create a New RPM

Create the ‘contrib-new’ package Go to the ‘contrib’ area of Rocks  # cd /usr/src/rocks/src/contrib/ Create the ‘new’ directory:  # mkdir new Populate ‘new’ with the ‘skeleton’ files:  # cp skeleton/* new Change names: ‘skeleton’ to ‘new’  # cd new  # mv contrib-skeleton.spec.in contrib-new.spec.in  # vi Makefile Change ‘/opt/skeleton’ to ‘/opt/new’

Building the Package Execute:  # make rpm This creates a binary and source RPM:  /usr/src/redhat/RPMS/i386/contrib-new-1-1.i386.rpm  /usr/src/redhat/SRPMS/contrib-new-1-1.src.rpm

Adding the New Package to the Distribution The package contrib-new-1-1.i386.rpm is in /usr/src/redhat/RPMS/i386 To compile a new distro with the ‘contrib-new’ package:  # cd /home/install  # rocks-dist dist This creates a Red Hat compatible distro in /home/install/rocks-dist/ Look under /home/install/rocks-dist for ‘contrib- new-1-1.i386.rpm’

Assigning the New Package to a Cluster Appliance Extend the ‘compute’ appliance  # cd /home/install/site-profiles/3.2.0/nodes  # cp skeleton.xml extend-compute.xml Add the ‘contrib-new’ package  # vi extend-compute.xml  Add the line: contrib-new  To apply dynamic configuration to components of the ‘contrib-new’ package, write a program and place it between ‘ … ’ tags

Test the changes Run kickstart.cgi  # cd /home/install  #./kickstart.cgi --client=‘compute-0-0’ > /tmp/ks.cfg Look for ‘contrib-new’ in ‘%packages’ section of /tmp/ks.cfg

Reinstall Compute Nodes Use ‘shoot-node’ to reinstall compute nodes in order to apply the ‘contrib-new’ package  # shoot-node compute-0-0