Lab System Environment Paul Kapinos 2014.10.07

Lab nodes: integrated to HPC Cluster OS: Scientific Linux 6.5 (RHEL6.5 compatible) Batch system: LSF 9.1  not for this lab  Storage: NetApp filer ($HOME / $WORK) no backup on $WORK Lustre ($HPCWORK) not available

Software Environment Compiler: MPI: Default: Intel 15.0 (and older) GCC 4.9 (and older) Oracle Studio, PGI MPI: Open MPI, Intel MPI No InfiniBand! 1GE only Warnings and 1/20 of usual performance Default: intel/14.0 + openmpi/1.6.5

How to login Frontends login / SCP File transfer: $ ssh [-Y] user@cluster.rz.rwth-aachen.de $ scp [[user@]host1:]file1 [...] [[user@]host2:]file2 then jump to the assigned lab node $ ssh lab5[.rz.rwth-aachen.de] cluster.rz.RWTH-Aachen.DE cluster2.rz.RWTH-Aachen.DE cluster-x.rz.RWTH-Aachen.DE GUI cluster-x2.rz.RWTH-Aachen.DE GUI cluster-linux.rz.RWTH-Aachen.DE cluster-linux-nehalem.rz.RWTH-Aachen.DE cluster-linux-xeon.rz.RWTH-Aachen.DE cluster-linux-tuning.rz.RWTH-Aachen.DE cluster-copy.rz.RWTH-Aachen.DE ‘scp’ cluster-copy2.rz.RWTH-Aachen.DE ‘scp’ cluster-x[2] only for GUI-based applications, not for compiling

Lab Node Assignment Please use your’s allocated node only or agree in advance with the node owner Node Account Institute lab1 hpclab01 EONRC lab2 hpclab02 IGPM lab3 hpclab03 GHI(AICES) lab4 hpclab04 ITV lab5 hpclab05 FZJ,PGI lab6 hpclab06 CATS lab7 hpclab07 Physik lab8 hpclab08 AIA

Lab Node Assignment Please use your’s allocated node only or agree in advance with the node owner

Lab nodes Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 64GB RAM Packages(sockets) / Cores per package / Threads per core : 2/18/2 Cores / Processors(CPUs) : 36 / 72 AVX2: 256bit register 2x Fused Multiply Add (FMA) >> double peak performance cf. previous chips 64GB RAM Stream: >100Gb/s (Triad) No InfiniBand connection MPI via 1GE network still possible Warnings and 1/20 of usual performance

Module System Many compilers, MPIs and ISV software The module system helps to manage all the packages List loaded modules / available modules $ module list $ module avail Load / unload a software $ module load <modulename> $ module unload <modulename> Exchange a module (Some modules depend on each other) $ module switch <oldmodule> <newmodule> $ module switch intel intel/15.0 Reload all modules (May fix your environment) $ module reload Find out in which category a module is: $ module apropos <modulename>

MPI No InfiniBand connection Default: Open MPI 1.6.5 MPI via 1GE network, >> warnings and 1/20 of usual performance Default: Open MPI 1.6.5 e.g. switch to Intel MPI: $ module switch openmpi intelmpi Wrapper in $MPIEXEC redirects the processes to ‘back end nodes’ by default your processes run on (random) non-Haswell node use the ‘-H’ option to start the processes on favoured node $ $MPIEXEC -H lab5,lab6 -np 12 MPI_FastTest.exe other options of the interactive wrapper $ $MPIEXEC -help | less

Documentation RWTH Compute Cluster Environment HPC Users‘s Guide (a bit outdated): http://www.rz.rwth-aachen.de/hpc/primer Online documentation (including example scripts): https://doc.itc.rwth-aachen.de/ Man-Pages for all commands available In case of errors / problems let us know: servicedesk@itc.rwth-aachen.de

Lab We provide laptops Log in to the laptops with the local „hpclab“ account (your own PC pool accounts might also work) Use X-Win32 to log in to the cluster (use “hpclab0Z” or your own account) Log in to the labZ node (use “hpclab0Z” account) Feel free to ask questions Source: D. Both, Bull GmbH