Helix - HPC/SLURM Tutorial 14/09/2017

Tutorials 10-15 minutes presentation 20-25 minutes examples 20-30 minutes questions Open for suggested format, time, topics

High Performance Computing Cluster: A collection of nodes that can run computing jobs in parallel Nodes: Individual Servers with RAM, CPU and networking interfaces. Also known as Compute nodes. RAM: Random Access Memory CPU: Central Processing Unit. Executes the jobs Core: A CPU can have multiple cores. Can run multiple processes Threads: Processes that run at the same time Storage: Hard drive space to store data Partition: Similar to a queue Helix: Our Cluster

Helix: Our Cluster Comprises of: 8 x 32 core, 128Gb Nodes. 175 Tb of disk storage Has Groups: [Project ID, Name, Project leaders] SG0001 Systems Biology Lab at Centre for Systems Genomics Edmund Crampin SG0004 Centre for Systems Genomics Cluster - Leslie Group Allan Motyer, Ashley Farlow, Damjan Vukcevic, Stephen Leslie SG0005 Statistical genomics University of Melbourne David Balding SG0007 Centre for System Genomics Cluster – Kim-Anh Le Cao SG0009 COGENT – Bobbie Shaban SGA0001 Systems Genomics Associate Member Dr Sarah Dunstan Andrew Siebel, SGN0001 Project space for Oxford Nanopore data generated by Systems Genomics

List of partitions [bshaban@snowy-sg1 ~]$ sinfo PARTITION AVAIL TIMELIMIT NODES STATE NODELIST main* up 30-00:00:0 4 mix snowy[002-003,005,012] main* up 30-00:00:0 26 alloc snowy[001,004,006,008-011,013-031] main* up 30-00:00:0 1 idle snowy007 sysgen up 1-00:00:00 1 mix snowy035 sysgen up 1-00:00:00 5 idle snowy[032-034,040-041] sysgen-long up 30-00:00:0 5 mix snowy[036-039,043] sysgen-long up 30-00:00:0 1 idle snowy042

List of partitions. Cont.. The system genomics nodes are split into two sub-partitions (4 x 128GB + 2 x 512GB nodes each).  Half the sysgen nodes are always available for < 24 hour duration jobs.  Any job longer than 24 hours will only be eligible for a sub-partition, and the main partition.  Any user (or project) can run a maximum of 256 cores worth of jobs at a time (global default), but the maximum number of cores a single job can use on a systems genomics partition is 192 cores (4 x 32 + 2 x 32). Any larger job will only run on the main partition.

Head node – Snowy-sg1 For Job submission only! Can use srun or sbatch. Only has 34Gb of memory. Can easily be consumed by a job If head node over subscribes then no jobs can be submitted – not being monitored Submit via sbatch – job is scheduled on a compute node irrespective of head node Use srun, sbatch or sinteractive – jobs will die if head node crashes

Resources limitations per node [bshaban@snowy-sg1 ~]$ sinfo --format "%n %20E %12U %19H %6t %a %b %C %m %e" HOSTNAMES REASON USER TIMESTAMP STATE AVAIL ACTIVE_FEATURES CPUS(A/I/O/T) MEMORY FREE_MEM snowy002 none Unknown Unknown mix up (null) 31/1/0/32 125000 117569 snowy003 none Unknown Unknown mix up (null) 8/24/0/32 125000 115214 snowy005 none Unknown Unknown mix up (null) 5/27/0/32 125000 118985

Resource limitations per user [bshaban@snowy-sg1 ~]$ mylimits ----------------------------------------------------------------- The following limits apply to your account on snowy: * Max number of idle jobs in queue (see note) 5 * Default memory per core in MB 2048 * Max memory per core in MB (use --mem-per-cpu=N) 50000 * Default wall time 10 mins * Max wall time (use --time=D-hh:mm:ss) 720 hours * Default number of CPUs 1 * Max number of CPUs per job (use --ntasks==N) 560 * Max number of jobs running at one time 192

Resource limitations per user. Cont.. [bshaban@snowy-sg1 ~]$ mydisk The following is how much disk space your current projects have: Fileset Size Used Avail Use% SG0009 20480G 125G 20355G 1% SG0004 25900G 10285G 15615G 40% SG0005 10240G 4278G 5962G 42% SGN0001 9140G 132G 9008G 1%

End Start tutorial https://slurm.schedmd.com/tutorials.html https://srcc.stanford.edu/sge-slurm-conversion Workshop September 26, 2017 - 1.30-4.30pm Melbourne Bioinformatics Boardroom, 187 Grattan Street, Carlton, VIC 3053 Australia Introduction to High Performance Computing Using High Performance Computing (HPC) resources such as Melbourne Bioinformatics in an effective and efficient manner is key to modern research.  This workshop will introduce you to HPC environments and assist you to get on with your research. October 3rd as well