1. CCR Advanced Seminar: Running CPLEX Computations on the ISE Cluster
Cynthia Cornelius
Center for Computational Research
University at Buffalo, SUNY
cdc at buffalo.edu
January 2016

2. CCR Cluster - CPLEX
A cluster is a collection of individual computers connected by a network that can act as a single more powerful machine.
Each individual computer has its own CPUs, memory and disks. It runs an operating system and it connected to a network that is usually private to the cluster. An individual computer is referred to as a compute node.

3. CCR Cluster - CPLEX
The CCR cluster is collection of linux computers, private networks, a shared file system, a login computer, and job scheduler.
Resources must be requested from the job scheduler.
The user must decide what resources are required for the job.

4. ISE Cluster The josewalt partition resides in the mae cluster.
The josewalt partition has 12 compute nodes.
Each compute node has 12 CPUs (cores) and 128 GB of memory.
The CPUs are Intel Xeon 2.40GHz
The operating system is Linux CentOS 7.
The time limit for jobs is 500 hours.

5. ISE Cluster
The storage available on the cluster is the user’s home directory and the group’s projects space.
The home directory is /user/username.
The disk space quota is 5GB.
The projects space is /projects/academic/group.
Both the home and projects directories are available on all compute nodes.

6. ISE Cluster and CPLEX
You will need CPLEX installed, a module file to set the path to the CPLEX installation, a SLURM script and your data files.
Both the CPLEX installation and module file are in the group’s project space on the cluster.

7. ISE Cluster and CPLEX
The SLURM script slurm-CPLEX is located the /util/academic/slurm-scripts directory on the cluster.
Input and data files should transferred from your Windows machine to the cluster using WINSCP.

8. Accessing the ISE Cluster
Use WINSCP client on the Windows machine to transfer, edit and manage files.
Use X-Win32 or PuTTY to login to the cluster front-end machine.
Submit jobs and show job status in the login window.
See the following link for download and setup instructions:
CCR helpdesk solution: login from windows

9. Accessing the ISE Cluster
The josewalt partition resides in the mae cluster. This is not the default.
All SLURM commands reference the CCR ub- hpc cluster by default.
You have two choices for accessing the josewalt partition.
1. Always specify the cluster when issuing a SLURM command.

10. Accessing the ISE Cluster
2. Make the mae cluster your default by setting the SLURM_CONF variable.
export SLURM_CONF=/util/academic/slurm/conf /mae/slurm.conf
The advantage is that you do not have to specify the cluster for every SLURM command.
The disadvantage is that this makes using the CCR cluster more difficult. Either you must specify the ub-hpc cluster or unset the variable.

11. ISE Cluster
Use the squeue command to show the status of jobs in the josewalt partition.
squeue -M mae -p josewalt

12. ISE Cluster
Use “-u username” to show the status of only the jobs submitted by that user.
squeue -M mae -p josewalt -u username

13. ISE Cluster
Use the sinfo or snodes command to show the status and details of the compute nodes.
sinfo -M mae -p josewalt

14. ISE Cluster The snodes command to shows more details:
snodes all mae/josewalt

15. Submit a Job to the ISE Cluster
Use the sbatch command to submit a job.
sbatch slurm_script

16. Submit an Interactive Job
How to submit an interactive job to the josewalt partition,
use the fisbatch command

17. More on Interactive Jobs
Advantage: an interactive job is useful in debugging a computation.
Disadvantages:
Waiting for a login to the compute node.
A disconnection from the cluster login machine terminates the job.
An interactive job is not constrained, so it could use resources intended for another computation when sharing a compute node.

18. Cancel a Job The scancel command cancels a running or pending job.
scancel -M mae jobid

19. SLURM Script Example
The #SBATCH lines are directives to the scheduler. The directives are the resource requests, job output file, and preferences.

20. SLURM Script Example
The following lines print information to the job output file.
The job starts in the directory from which you submitted it. This is the working directory.

21. SLURM Script Example Load the module file for CPLEX.
List the loaded modules.
The ulimit command removes the size limit on the instruction stack. This helps large programs to run.

22. SLURM Script Example Run the computation using the command line.
Once all the lines of the SLURM have been executed, then the job completes and exits the compute node.