Unix Scripts and PBS on BioU Alexander J. Ropelewski ropelews@psc.edu

Unix Scripts Type Run Save File

A Very Simple Script #!/bin/sh # # run the ls command ls -laF File: mylsscript.sh

A Very Simple Script #!/bin/sh # # run the ls command ls -laF FIRST LINE: Tells the UNIX System the Shell (or command interpreter) to use. Can be: #!/bin/sh #!/bin/csh #!/bin/tcsh #!/usr/bin/perl Or many others… #!/bin/sh # # run the ls command ls -laF Lines beginning with # are comments File: mylsscript.sh The UNIX command(s) to execute, in this case we want the ls command

Steps to Running a Script Edit a file Place commands in file Save File Make File Executable: chmod u+x mylsscript.sh Type in the name of the script file mylsscript.sh

What can Scripts Do Scripts can assign values to variables Scripts can have control elements such as: Error Checking Conditionals (If Tests) Loops Scripts can also do simple math Scripts can run programs

Script With Conditional & Error Check #!/bin/sh cp file.out $HOME/file.out if [ $? -eq 0 ] then echo "File copied" else echo "PROBLEM" fi File: iftest.sh

Script With Conditional & Error Check Test condition: $? Is the status returned from the prior command (cp). If the prior command (cp) was successful, this will contain 0 First line tells us this is a Bourne Shell Script Copy command #!/bin/sh cp file.out $HOME/file.out if [ $? -eq 0 ] then echo "File copied" else echo "PROBLEM" fi Conditional test; format must be either : IF - THEN - FI -or- IF - THEN - ELSE - FI If condition is true write “File copied” message If condition is false write “PROBLEM” message File: iftest.sh

Script With Loop & Simple Math #!/bin/sh i="1" while [ $i -le 180 ] do echo "Iteration # $i" i=`expr $i + 1` sleep 1 done File: test1.job

Script With Loop & Simple Math First line tells us this is a Bourne Shell Script Condition: variable i is less than or equal to 180 Initialize variable i to 1 #!/bin/sh i="1" while [ $i -le 180 ] do echo "Iteration # $i" i=`expr $i + 1` sleep 1 done While Loop: Do the instructions between the do and done until the condition is satisfied Echo command is a simple print function. Assign variable i to now be equal to the old value of i plus 1. Sleep command tells system to wait 1 second before continuing File: test1.job

What Are PBS Scripts PBS scripts are just Unix shell scripts with some additional attributes (that provide resource and control information) that are executed by the PBS queuing system. PBS scripts are submitted to queues and are run when the system has resources available PBS script files will be created and run for you automatically when you use the Galaxy interface on BioU You can gain more control over your analyses by writing (and running) your own PBS scripts Sometimes you will hear PBS scripts called "jobs".

PBS Script With Loop & Simple Math #!/bin/sh #PBS -q serial #PBS -j oe #PBS -l walltime=00:10:00 #PBS -l nodes=1:ppn=1 #PBS -N PbsTest i="1" while [ $i -le 180 ] do echo "Iteration # $i" i=`expr $i + 1` sleep 1 done File: pbstest.job

PBS Script With Loop & Simple Math Parameters always begin with: #PBS Queue name: On BioU these are serial and parallel Write stderr in stdout file #!/bin/sh #PBS -q serial #PBS -j oe #PBS -l walltime=00:10:00 #PBS -l nodes=1:ppn=1 #PBS -N PbsTest i="1" while [ $i -le 180 ] do echo "Iteration # $i" i=`expr $i + 1` sleep 1 done Set the walltime limit to ten minutes Serial job, so only need one core on one node. Name the job PbsTest File: pbstest.job

Running PBS Scripts To submit a PBS script use the qsub command: % qsub pbstest.job 382.biou1.psc.edu The system will respond with the PBS identifier: 382 – The scripts job id number biou1.psc.edu - The computer id

Monitoring PBS Scripts To see if a PBS script is running, use the qstat command: % qstat Job id Name User Time Use S Queue ---------------- ---------------- ---------------- -------- - ----- 381.biou1 serial_example nigra 0 R serial 382.biou1 pbstest.job ropelews 0 Q serial

Killing PBS Scripts To kill a PBS script, first, get the PBS script identifier: % qstat Job id Name User Time Use S Queue ---------------- ---------------- ---------------- -------- - ----- 381.biou1 serial_example nigra 0 R serial 382.biou1 pbstest.job ropelews 0 Q serial Then terminate the PBS script: % qdel 382.biou1

Galaxy PBS Scripts To check on a PBS script submitted through Galaxy, use the BioU status check tool. Galaxy PBS scripts are limited to 48 hours (walltime) PBS scripts submitted through Galaxy can not be terminated by the user. PBS scripts submitted on the command line can run longer than 48 hours.