CCR Advanced Seminar: Running CPLEX Computations on the ISE Cluster Cynthia Cornelius Center for Computational Research University at Buffalo, SUNY cdc at buffalo.edu January 2016

CCR Cluster - CPLEX A cluster is a collection of individual computers connected by a network that can act as a single more powerful machine. Each individual computer has its own CPUs, memory and disks. It runs an operating system and it connected to a network that is usually private to the cluster. An individual computer is referred to as a compute node.

CCR Cluster - CPLEX The CCR cluster is collection of linux computers, private networks, a shared file system, a login computer, and job scheduler. Resources must be requested from the job scheduler. The user must decide what resources are required for the job.

ISE Cluster The josewalt partition resides in the mae cluster. The josewalt partition has 12 compute nodes. Each compute node has 12 CPUs (cores) and 128 GB of memory. The CPUs are Intel Xeon E5-2620 @ 2.40GHz The operating system is Linux CentOS 7. The time limit for jobs is 500 hours.

ISE Cluster The storage available on the cluster is the user’s home directory and the group’s projects space. The home directory is /user/username. The disk space quota is 5GB. The projects space is /projects/academic/group. Both the home and projects directories are available on all compute nodes.

ISE Cluster and CPLEX You will need CPLEX installed, a module file to set the path to the CPLEX installation, a SLURM script and your data files. Both the CPLEX installation and module file are in the group’s project space on the cluster.

ISE Cluster and CPLEX The SLURM script slurm-CPLEX is located the /util/academic/slurm-scripts directory on the cluster. Input and data files should transferred from your Windows machine to the cluster using WINSCP.

Accessing the ISE Cluster Use WINSCP client on the Windows machine to transfer, edit and manage files. Use X-Win32 or PuTTY to login to the cluster front-end machine. Submit jobs and show job status in the login window. See the following link for download and setup instructions: CCR helpdesk solution: login from windows

Accessing the ISE Cluster The josewalt partition resides in the mae cluster. This is not the default. All SLURM commands reference the CCR ub- hpc cluster by default. You have two choices for accessing the josewalt partition. 1. Always specify the cluster when issuing a SLURM command.

Accessing the ISE Cluster 2. Make the mae cluster your default by setting the SLURM_CONF variable. export SLURM_CONF=/util/academic/slurm/conf /mae/slurm.conf The advantage is that you do not have to specify the cluster for every SLURM command. The disadvantage is that this makes using the CCR cluster more difficult. Either you must specify the ub-hpc cluster or unset the variable.

ISE Cluster Use the squeue command to show the status of jobs in the josewalt partition. squeue -M mae -p josewalt

ISE Cluster Use “-u username” to show the status of only the jobs submitted by that user. squeue -M mae -p josewalt -u username

ISE Cluster Use the sinfo or snodes command to show the status and details of the compute nodes. sinfo -M mae -p josewalt

ISE Cluster The snodes command to shows more details: snodes all mae/josewalt

Submit a Job to the ISE Cluster Use the sbatch command to submit a job. sbatch slurm_script

Submit an Interactive Job How to submit an interactive job to the josewalt partition, use the fisbatch command

More on Interactive Jobs Advantage: an interactive job is useful in debugging a computation. Disadvantages: Waiting for a login to the compute node. A disconnection from the cluster login machine terminates the job. An interactive job is not constrained, so it could use resources intended for another computation when sharing a compute node.

Cancel a Job The scancel command cancels a running or pending job. scancel -M mae jobid

SLURM Script Example The #SBATCH lines are directives to the scheduler. The directives are the resource requests, job output file, and email preferences.

SLURM Script Example The following lines print information to the job output file. The job starts in the directory from which you submitted it. This is the working directory.

SLURM Script Example Load the module file for CPLEX. List the loaded modules. The ulimit command removes the size limit on the instruction stack. This helps large programs to run.

SLURM Script Example Run the computation using the command line. Once all the lines of the SLURM have been executed, then the job completes and exits the compute node.