1. HOPPER CLUSTER
NEW USER ORIENTATION
February 2018

2. WELCOME & INTRODUCTIONS
HPC SYSTEM ADMINISTRATION & SUPPORT
Frank Henderson |
Bradley Morgan |
Matt Smith |
Keenan Terry |
Jeff
February 2018

3. HPC OVERVIEW Purpose & Impact CASIC Cluster (2013)
Hopper Cluster (2016)
Future Resources
Jeff
February 2018

4. Growth in Auburn Compute Power February 2018 Hopper Phase 3
CASIC
Hopper Phase 1
Hopper Phase 2
269 nodes/~175 TFLOPS
209 nodes/134 TFLOPS
117 nodes/88 TFLOPS
Jeff
74 nodes/20 TFLOPS
2013
JAN 2016
NOV 2016
OCT 2017
February 2018

5. Today’s topics Basic Command Line Tools and Techniques
Research Software: Finding, Using, Requesting, and Compiling
Job Submission
Job Scheduling
Troubleshooting
Evaluation
Jeff
February 2018
High Performance Computing at Auburn

6. Connecting to a Linux System with SSH Clients
Let’s begin
Connecting to a Linux System with SSH Clients
Windows
Mac OS
Linux*
PuTTY:
putty.org
SecureCRT:
auburn.edu/download
WinSCP (File Transfers):
Emulation*: VMWare, Cygwin
Terminal Application:
Applications > Utilities > Terminal
Third Party Options:
Prompt (iOS): panic.com/prompt
Transmit (File Transfers): panic.com/transmit
Emulation*: VMWare Fusion, VirtualBox
gnome-terminal: /usr/bin
Desktop \ DE default
Runlevel 3**:
/etc/inittab
Ctrl-Alt-F3
Recommended Distribution:
CentOS:
centos.org
Bradley
*Advanced Users
February 2018
Cluster Access & Basic Architecture

7. Let’s begin February 2018 Cluster Access & Basic Architecture
Hopper Login Node (Campus Address or VPN Required)
$ ssh
Login Node
Secure Shell Client
Scheduler
GPFS
/home
/scratch
/tools
Bradley
Do not run intense processes
on login node!
February 2018
Cluster Access & Basic Architecture

8. BASIC CONCEPTS Why Linux? A Brief History of Linux Distributions
The Shell
Kendall
February 2018
Concepts & Fundamentals

9. Why Linux? All 500 of the world’s top supercomputers use Linux
Vast majority of research software available for Linux
Linux is open source and highly customizable, adaptable
Linux is powerful and awesome
Kendall
February 2018
Concepts & Fundamentals

10. A BRIEF HISTORY OF LINUX
UNIX, Bell Labs 1969 Early Mainframes
BSD, 1977
GNU Project, 1983
1987 MINIX
1991 LINUX
Kendall
February 2018
Concepts & Fundamentals

11. Command Syntax: $ command [optional] <required>
LINUX & BASH
Command Syntax: $ command [optional] <required>
$ man <command>
$ pwd
$ ls [/path/to/file]
$ cd [/path/to/directory]
$ mkdir <directory_name>
$ cp <source_filename> <target_filename>
$ mv <source_filename> <target_filename>
Display command help
Show current directory
List directory \ folder contents
Change directory
Make directory
Copy file(s)
Move file(s) or directory
Bradley
February 2018
Basic Linux Commands

12. LINUX & BASH February 2018 Shell, Hidden Files, & Processes
$ chmod <permissions> <file>
$ chown <user:group> <file>
$ env
$ cat </path/to/filename>
$ more </path/to/filename>
$ nano
$ cd ~
$ ls –al ~
$ cat ~/.bash_profile
$ cat ~/.ssh
$ ps
$ top
Change file mode (Read \ Write \ Execute)
Change file ownership (User : Groups)
Show your Environment Settings (Modulefiles)
Display file contents
Display file contents page-by-page
Edit file(s)
~ = your home directory
The –al option displays detailed file information
The .bash_profile is executed on login
The .ssh directory contains special key files
ps will display your running processes
top will show a list of processes on the machine
Bradley
February 2018
Shell, Hidden Files, & Processes

13. Files and directories February 2018 The Linux File System Paths
Text version of folders in Windows GUI
Forward Slash (/)
/path/to/file
Home Directory
/home/<username> ~
. (dot) and .. (dot-dot)
Bradley
February 2018
The Linux File System

14. Files and directories February 2018 The Linux File System
File Ownership and Permissions
File access is controlled at three levels: Owner, Group, World
Users & Groups
The user who creates a file or directory is the owner
Owners can change the permissions of their files (chown)
Can modify group access if they are a member of the group (chown)
Read, Write, Execute (RWX)
Owner (User), Group, and World
chmod and chown
Bradley
February 2018
The Linux File System

15. FiLE permissions February 2018 The Linux File System
drwxrwxr-x 4 hpcadmin hpcadmin Oct 4 10:45 .
drwxr-xr-x 7 hpcadmin hpcadmin Oct 4 10:43 ..
drwx hpcadmin hpcadmin_lab 4096 Oct 4 10:45 dir1
drwxr-xr-x 2 hpcadmin research Oct 4 10:45 dir2
-rwx hpcadmin hpcadmin Oct 4 10:45 file1
-rwxrwx--- 1 hpcadmin hpcadmin_lab 12 Oct 4 10:45 file2
-rwxrwxrwx 1 hpcadmin hpcadmin Oct 4 10:45 file3
-rw hpcadmin hpcadmin Oct 4 10:45 file4
-rw-rw hpcadmin hpcadmin_lab 12 Oct 4 10:45 file5
-rw-rw-rw- 1 hpcadmin hpcadmin_lab 12 Oct 4 10:45 file6
-r hpcadmin hpcadmin Oct 4 10:45 file7
-r--r hpcadmin hpcadmin Oct 4 10:45 file8
-r--r--r-- 1 hpcadmin hpcadmin Oct 4 10:45 file9
Bradley
February 2018
The Linux File System

16. Files and directories
Bradley
February 2018
The Linux File System

17. Exercise: Part I February 2018 Getting Started & Basic Linux Commands
Bradley
Lab Exercise:
Part I: Getting Started & Basic Linux Commands
February 2018
Getting Started & Basic Linux Commands

18. RESEARCH SOFTWARE February 2018 Typical Workloads
OpenMP \ PTHREADS \ Multithreaded
Message Passing Interface (MPI)
GNU Parallel
Large Memory
Internal Parallelization
Bradley
February 2018
Typical Workloads

19. RESEARCH SOFTWARE February 2018 Research Software Challenges
Dependencies are highly dynamic
Requires manipulation of your shell environment
Typically requires compilation specific to the target system
Can be installed and used in home directory
Or, installed by HPC Admins:
Many popular software, compilers, and libraries already available…
Bradley
February 2018
Research Software Challenges

20. ENVIRONMENT February 2018 Shell, Hidden Files, & Processes
Traditional Method
$ echo $PATH
$ echo $LD_LIBRARY_PATH
$ export PATH=$PATH:/home/username/custom/bin
$ export LD_LIBRARY_PATH=/home/username/custom/lib:$LD_LIBRARY_PATH
Environment Module Method
$ module load <software_name>
Basic Commands
$ module avail [search_string]
$ module show <software_name>
$ module purge (must reload torque)
$ module swap <old_software> <new_software>
$ module list
Bradley
February 2018
Shell, Hidden Files, & Processes

21. COMPILING February 2018 Compiling Traditional GNU Toolchain
$ wget <
$ tar –xvzf <sw.tar.gz>
$ cd /new/path/to/sw
$ ./configure –prefix=/home/username/custom/location
$ make
$ make install
Compiling for MPI
$ module load openmpi
$ mpicc source.c –o binary
$ mpirun –np 2 ./binary
Bradley
Other Popular Methods
cmake
Binary distribution \ installer
February 2018
Compiling

22. COMPILING COMMON ISSUES February 2018 Compiling Missing Dependencies
$ gcc foo.c –o foo
./foo: error while loading shared libraries: libfoo.so: cannot open shared object file: No such file or directory
Load the corresponding module, check LD_LIBRARY_PATH to ensure library location is there
Tip: GCC Linker expects module name to be libfoo.so.x
Wrong Compiler
$ gcc mpifoo.c –o mpifoo
mpifoo.c:(.text+0x33): undefined reference to `MPI_Init‘
Bradley
In this case, there are symbols in the source code that are not recognized by the normal GCC
We have to use the OpenMPI compiler wrapper, mpicc, to compile this source code.
February 2018
Compiling

23. Exercise: Part II February 2018 Keenan Lab Exercise:
Part II: Research Software
February 2018

24. Bradley
February 2018

25. JOB SUBMISSION February 2018 Scheduler Commands Submitting Jobs
$ qsub [options] <job.sh>
$ qsub –l nodes=1:ppn=2 ./job.sh
Interactive Jobs
$ qsub –q debug –I
Monitoring Jobs
$ showq –u <userid>
$ checkjob <jobid>
$ checkjob –v –v –v <jobid>
Cancel a Job
Bradley
$ canceljob <jobid>
$ qdel <jobid>
February 2018
Scheduler Commands

26. JOB SUBMISSION February 2018 Scheduler Commands Viewing the Queues
$ qstat -q
Specify Script Parameters
$ qsub –F <arg1> [arg2] … ./job.sh
Memory Control
$ qsub –l nodes=1:ppn=20:mem=120gb
$ qsub –l nodes=1:ppn=10:pmem=10gb
Reservations
$ showres
$ whats-running-on-my-rsv.sh
$ rsub –l nodes=1:ppn=20 ./job.sh
Bradley
February 2018
Scheduler Commands

27. Exercise: Part III February 2018 Keenan Lab Exercise:
Part III: Job Submission
February 2018

28. JOB SCHEDULING What resources are available? When will my job run?
Keenan
To answer these questions, must understand the goal of the cluster.
February 2018

29. The goal
Maximize utilization of the cluster resources while guaranteeing investors the resources that they purchased.
Keenan
How to accomplish this?
Scheduler could easily just run jobs in order ( i.e., sequentially as submitted or FIFO ), but this would not accomplish the goal.
Must be able to run jobs out-of-order. Do this using 1) backfill and 2) reservations.
February 2018

30. Exercise: Part IV February 2018 Keenan Lab Exercise:
Part IV: Job Scheduling
February 2018

31. Best practices
February 2018

32. Reminders February 2018 Keep all Reminders for future use.
But use only a subset of them for any one Open House.
February 2018

33. CITATIONS & ACKNOWLEDGMENT
Please help show the importance of HPC resources in research at Auburn University by citing such in any publication or presentation that are made possible using the CASIC or Hopper Clusters.
Jeff
The Powers-That-Be are evaluating the importance of HPC at Auburn.
Acknowledging Hopper’s role in your research is the best thing that you can do to ensure that HPC is a top priority.
February 2018

34. AUBURN HPC v. ALABAMA SUPERCOMPUTER
Two distinct entities.
Please give credit to the appropriate institution.
We are NOT the same as the Alabama Supercomputer Authority.
February 2018

35. Course evaluatation http://aub.ie/hpceval February 2018
To answer these questions, must understand the goal of the cluster.
February 2018

36. HPC RESOURCES General Information: https://hpc.auburn.edu
HPC Community Blog:
Hopper User’s Guide:
Hopper HPC Mailing List:
CASIC HPC Mailing List:
Jeff
February 2018
High Performance Computing at Auburn