Compaq - Indiana University Visit IU’s Compaq Parallel PC Cluster

Clusters n Becoming more widespread n Benefits: u Cost, price/performance u Familiar operating system choices u Commodity components n Issues: u Interconnect/appropriate application space u Tools (administrator and user)

n 32 Compaq ProLiant 1850R compute nodes: u Dual 400 MHz Intel PII processors u 256 MB SDRAM, 512kB L2 cache u Two 4.3 GB “hot-plug” SCSI disks u Integrated 10/100 TX Ethernet interface and Packet Engines G-NIC II Ethernet interface n 3 controls workstation (NT Primary Domain Server, NT Terminal Server, Linux development system) IU ’ s Compaq Cluster

n Fast Ethernet: HP ProCurve 4000M 10/100Base-TX Switch n Gigabit Ethernet: Packet Engines PowerRail 5200 Switch. GB interconnect still largely experimental Parallel PC Cluster Interconnect

Parallel PC Cluster

Unique features of IU ’ s Compaq cluster n Evenhanded approach to OS issues. Key questions: Which is better under what circumstances? How can we make our scientists most productive? n Combination of computer science research, (production) scientific calculation, and artistic applications n Robustness of system n Relationship with NCSA

NT and Linux n Each compute node can be run under either NT or Linux (reboot required). n Default configuration of the Parallel PC Cluster is 16 nodes Linux, 16 nodes NT n For experiments & code that scales to 64 processors, cluster run under one OS n Future need: more flexible control over OS configuration

Linux Clusters

NT Clusters

Clusters running NT & Linux

Accomplishments to date n Investigation of system management tools n NT/Linux comparisons n Gigabit Ethernet investigation n Price/performance characterization n Use of applications software for production use

OS and system software choices n NT u System monitoring & mgmt: F Big Brother F Compaq Insight Manager u Job management: LSF u Compilers: F Microsoft Visual Studio F Digital Visual Fortran F Portland Group Suite u Parallel APIs F MPI (MPI/Pro) F PVM n Linux u System monitoring: F Big Brother u Job management: PBS u Software Development F GNU C/C++/F77 F Portland Group Suite u Parallel APIs F MPICH F PVM

System monitoring - Big Brother (NT & Linux)

System monitoring & management - Compaq Insight Manager

System management with Linux

Pallas MPI Benchmarks Maximum Throughput

Pallas MPI Benchmarks Latency

NT MPI Benchmarks Maximum Throughput

NAS Parallel Benchmarks

QCD Benchmark

Compiler issues n Under NT, Digital Visual Fortran tends to perform the best n Compilers for Linux are a substantial issue. GNU compilers have poor optimization for Pentuim processors. Code that one would expect to run well can run very poorly.

Price/Performance n For applications well suited to PC clusters, Price/Performance of the Compaq Parallel PC Cluster is better than a traditional supercomputer by a factor of 3 to nearly a factor of 10 n Using QCD code developed at IU, the P/P ratios are <$100/MFLOP vs ~$300/MFLOP

Application Software n Radiance n DART n SoftImage n AutoCAD n Gaussian n 3DStudioMax (to be installed)

Scientific & artistic projects u Physics: Quantum Chromodynamics u Astrophysics: 3-D Hydrodynamics u Engineering: Computational Fluid Dynamics u UITS: cluster performance research u Television animation production (WTIU): Parallel Rendering of animations u Virtual Reality: Distributed Ray Tracing, Volume Visualization

WTIU Video

Volume Visualization

Future Projects n Computer Science: Component architectures, Distributed Objects (e.g. CAT, DCOM, ActiveX, JavaBeans) n NCSA-related activities: Globus, distributed performance testing with NCSA, superclusters n Environmental Science: Groundwater modelling

Cat Workspace and Info Browser

PC Cluster Grid Computing n SC98 iGrid demonstrations included PC clusters used n IU is positioned to participate in the development of PC superclusters u Abilene, TransPAC, vBNS, EuroLink connectivity u NCSA Alliance institution u DOE2000 participation u APAN membership

iGrid Sites vBNS ESNet NREN Teleglobe CANTAT-3 CA*Net2 TransPAC (to APAN) TANet SINGAren

Issues/areas for further investigation u Cluster management tools for NT and Linux, incl. ability to reboot single and multiple workstations with OS selection u User filespace under NT u Compilers u Network Performance, esp. Gigabit ethernet u VIA vs. Kernel I/O networking

Conclusions n IU has assembled a robust Parallel PC Cluster based on Compaq servers n Both NT and Linux have areas of strength, areas of weakness n The IU Parallel PC Cluster can be used for production work in the sciences and arts under either NT or Linux

Thanks to: n Don Berry n Dave Hart n Dan Lauer n Rick McMullen n John Naab n Anurag Shankar n Shawn Slavin n Teresa Todd

Compaq - Indiana University Visit Thank you. Any questions?

n Except where otherwise noted, the contents of this presentation are © by the Trustees of Indiana University. This content is released under the Creative Commons Attribution 3.0 Unported license ( This license includes the following terms: You are free to share – to copy, distribute and transmit the work and to remix – to adapt the work under the following conditions: attribution – you must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work). For any reuse or distribution, you must make clear to others the license terms of this work.