COMPTUER CLUSTERING WITH LINUX-ON-CD Robert Ibershoff Computer Electronic Networking

2 OUTLINE Motivation Introduction to clustering Problem scenario Solutions (why to use openMosix) Building and testing Results and Conclusions Future possibilities

3 MOTIVATION Old computers sitting around. Still useful? Generally interesting subject.

4 INTRODUCITON: CLUSTERING Clustering is the method of linking several computers together through software to work as if they were one computer. So called ‘super-computers’ are prohibitively expensive. Blue Gene/L 

5 PROBLEM STATEMENT You are: Astronomer/biologist. Need lots of raw processing power. Usually in academic fields– meaning budget is miniscule (no supercomputer). Have only older computers of different hardware.

6 POSSIBLE SOLUTIONS Choices: Install OS and calculation software on each computer, and run them each separately (time and work intensive) Save up for years and get a small supercomputer (time and money intensive) Create a cluster from old computers

7 CLUSTER SOLUTION Windows 2003 server can create clusters. (somewhat complex to set up) Dedicated clustering software: Beowulf, IBM eServer, Sun, etc. ($$$) Linux clustering

8 LINUX CLUSTERS openMosix software turns Linux computers into a cluster Works on most versions of Linux Works with many kinds of calculation software (no special software needs to be written)

9 ClusterKnoppix Knoppix is a version of Linux that runs completely off of CD with no presence on hard drive ClusterKnoppix is Knoppix with the latest version of openMosix configured to activate automatically Utilizes existing network infrastructure (IP addresses), so a functional LAN with a DHCP server is needed.

10 ADVANTAGES Cost-Effective (CD-R media and burner) Fast cluster setup (just boot from CD) All operations happen in RAM, which is 100% erased on reboot (no viruses can survive) Addition of node to cluster is automatic upon boot

11 BUILDING THE CLUSTER Due to the unique traits of Knoppix and openMosix, the computers in a nearby lab were suitable for the cluster Drop the CD in the drive, restart (booting from CD), and you’re done. Totally non-invasive of computer: take the CD out and reboot, and computer runs exactly as it did before.

12 TESTING THE CLUSTER Used a simple looping program to load processors in clustersimple looping program Copied and pasted from website (which the instructions said to do) and compile into an executable file Screenshots

13 RESULTS Cluster expanded on its own as each node came up Loads were distributed across cluster as test program was run repeatedly

14 CONCLUSIONS ClusterKnoppix provides an extremely fast and cheap way to turn a group of computers on a LAN into a cluster. Can temporarily utilize a network for cluster computing, then leave it as it was with no difference from before

15 THE LEARNING EXPERIENCE Trying to ‘load balance’ all of my tasks proved to be trying during the semester. The internet is a priceless resource for information on more or less everything. Unrelated but interesting discoveries World Community Grid)

16 EXTENDED WORK Future career plans include astronomy, will remember the quick and cheap way to build a cluster for future reference. Gradually learning more about Linux in general, a useful and beneficial project in and of itself.