Minimalist’s Linux Cluster Changyoung Choi, Jeonghyun Kim, Seyong Kim Department of Physics Sejong University

Reported in Lattice’99

1. Usual Linux cluster Easy to build, use and maintain Off-The-Shelf everything! –Ordinary PC’s –Ordinary network components –Free OS (Linux) –Free parallel programming environment (MPI)

Room for improvement Occupies too much space Consumes too much power Not all the components in an ordinary PC is necessary for computation –Possible further reduction in cost! –Possible reduction in power consumption Supporting full bisection bandwidth is costly!

Conditions No custom-made hardwares –If it takes too long to develop -> no advantage –If there is, should be recyclable Should be efficient enough Should be do-able with small manpower

2. Construction(hardware) Node –ATX size standard mother board –Intel Pentium IV-2.4 GHz –512 Mbytes DDR SDRAM –100 Mbps ethernet card (3+1) –180W power Cage for ATX size mother board 1 front end server

CPUIntel P-4,2.4GHz\236,500(=$198 )x36 motherboardASUS P4PE\203,500(=$170 )x36 memory512MB PC2700 DDR \112,200(=$93. 5)x36 NIC3+1 realtek\53,000(=$44)x 36 powerSun ATX 180W\25,300(=$21)x 36 cable\10,000(=$8)x3 6

cabinet2000x910x750\880,000(=$ 733) cabinet accessory\365,200(=$ 304) Cost per node$534.5 Total cost$534.5x36+$1037 = $20,279 Price Includes Tax

Software WOW Linux 7.1 (equiv. RedHat 7.1) Linux Terminal Server Project (LTSP) –Remote booting (DHCP + TFTP) –Diskless node system 2-D mesh network configuration –Script developed in-house MPI programming environment –MPICH/LAM

Setup Front end wait F-end send DHCP info send kernel using TFTP service send client file system by NFS Node 1 power-up NIC listens to DHCP server IP setup complete and download kernel setup file system on memory and copy NFS mounted file system to memory

Setup(continued) Node 1 wait Node 1 send DHCP info Node 1 send kernel image Node 1 act as a pipe line between front end and node 2 for NFS Node 2 power-up NIC listens to node 1 DHCP server IP setup complete and download kernel from node 1 setup file system on memory and copy NFS mounted file system to memory

Parallel boot

3. Performance 1-D ring full QCD code –Not suitable for 2-D mesh structure 8^3 x 512 lattice

8^3 x 32 lattice tested on 1-node 1187 sec, CG = > 369 MFLOPS 8^3 x 512 lattice tested on 4-node 5522 sec, CG = > 1.24 GFLOPS 8^3 x 512 lattice tested on 8-node 3029 sec, CG = > 2.25 GFLOPS

4. Discussion recyclable –cabinet takes standard motherboard size -> replacing node is enough for a upgrade occupies smaller foot-print, consumes less power need to fine-tune need to modify the code for 1-D ring to 2-D mesh