UNIX Shell-Scripting Basics

Agenda What is a shell? A shell script? Introduction to bash Running Commands Applied Shell Programming

What is a shell? % ▌

/bin/bash

What is a shell? #!/bin/bash

What is a shell? INPUTshellOUTPUTERROR

What is a shell? Any Program But there are a few popular shells…

Bourne Shells /bin/sh /bin/bash “Bourne-Again Shell” Steve Bourne

Other Common Shells C Shell ( /bin/csh ) Turbo C Shell ( /bin/tcsh ) Korn Shell ( /bin/ksh )

An aside: What do I mean by /bin ? C Shell ( /bin/csh ) Turbo C Shell ( /bin/tcsh ) Korn Shell ( /bin/ksh )

An aside: What do I mean by /bin ? /bin, /usr/bin, /usr/local/bin /sbin, /usr/sbin, /usr/local/sbin /tmp /dev /home/borwicjh

What is a Shell Script? A Text File With Instructions Executable

What is a Shell Script? % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh %./hello.sh Hello, world

What is a Shell Script? A Text File % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh %./hello.sh Hello, world

An aside: Redirection cat > /tmp/myfile cat >> /tmp/myfile cat 2> /tmp/myerr cat < /tmp/myinput cat <<INPUT Some input INPUT cat > /tmp/x 2>&1 INPUT env OUTPUTERROR 0 12

What is a Shell Script? How To Run % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh %./hello.sh Hello, world

What is a Shell Script? What To Do % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh %./hello.sh Hello, world

What is a Shell Script? Executable % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh %./hello.sh Hello, world

What is a Shell Script? Running it % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh %./hello.sh Hello, world

Finding the program: PATH %./hello.sh echo vs. /usr/bin/echo % echo $PATH /bin:/usr/bin:/usr/local/bin: /home/borwicjh/bin % which echo /usr/bin/echo

Variables and the Environment % hello.sh bash: hello.sh: Command not found % PATH=“$PATH:.” % hello.sh Hello, world

An aside: Quoting % echo ‘$USER’ $USER % echo “$USER” borwicjh % echo “\”” ” % echo “deacnet\\sct” deacnet\sct % echo ‘\”’ \”

Variables and the Environment % env […variables passed to sub-programs…] % NEW_VAR=“Yes” % echo $NEW_VAR Yes % env […PATH but not NEW_VAR…] % export NEW_VAR % env […PATH and NEW_VAR…]

Welcome to Shell Scripting!

How to Learn man  man bash  man cat  man man man –k  man –k manual Learning the Bash Shell, 2 nd Ed. “Bash Reference” Cards

Introduction to bash

Continuing Lines: \ % echo This \ Is \ A \ Very \ Long \ Command Line This Is A Very Long Command Line %

Exit Status $? 0 is True % ls /does/not/exist % echo $? 1 0

Exit Status: exit % cat > test.sh <<_TEST_ exit 3 _TEST_ % chmod +x test.sh %./test.sh % echo $? 3

Logic: test % test 1 -lt 10 % echo $? 0 % test 1 == 10 % echo $? 1

Logic: test test [ ]  [ 1 –lt 10 ] [[ ]]  [[ “this string” =~ “this” ]] (( ))  (( 1 < 10 ))

Logic: test [ -f /etc/passwd ] [ ! –f /etc/passwd ] [ -f /etc/passwd –a –f /etc/shadow ] [ -f /etc/passwd –o –f /etc/shadow ]

An aside: $(( )) for Math % echo $(( )) 3 % echo $(( 2 * 3 )) 6 % echo $(( 1 / 3 )) 0

Logic: if if something then : # “elif” a contraction of “else if”: elif something-else then : else then : fi

Logic: if if [ $USER –eq “borwicjh” ] then : # “elif” a contraction of “else if”: elif ls /etc/oratab then : else then : fi

Logic: if # see if a file exists if [ -e /etc/passwd ] then echo “/etc/passwd exists” else echo “/etc/passwd not found!” fi

Logic: for for i in do echo $i done

Logic: for for i in /* do echo “Listing $i:” ls -l $i read done

Logic: for for i in /* do echo “Listing $i:” ls -l $i read done

Logic: for for i in /* do echo “Listing $i:” ls -l $i read done

Logic: C-style for for (( expr1 ; expr2 ; expr3 )) do list done

Logic: C-style for LIMIT=10 for (( a=1 ; a<=LIMIT ; a++ )) do echo –n “$a ” done

Logic: while while something do : done

Logic: while a=0; LIMIT=10 while [ "$a" -lt "$LIMIT" ] do echo -n "$a ” a=$(( a + 1 )) done

Counters COUNTER=0 while [ -e “$FILE.COUNTER” ] do COUNTER=$(( COUNTER + 1)) done Note: race condition

Reusing Code: “Sourcing” % cat > /path/to/my/passwords <<_PW_ FTP_USER=“sct” _PW_ % echo $FTP_USER %. /path/to/my/passwords % echo $FTP_USER sct %

Variable Manipulation % FILEPATH=/path/to/my/output.lis % echo $FILEPATH /path/to/my/output.lis % echo ${FILEPATH%.lis} /path/to/my/output % echo ${FILEPATH#*/} path/to/my/output.lis % echo ${FILEPATH##*/} output.lis

It takes a long time to become a bash guru…

Running Programs

Reasons for Running Programs Check Return Code  $? Get Job Output  OUTPUT=`echo “Hello”`  OUTPUT=$(echo “Hello”) Send Output Somewhere  Redirection:  Pipes

Pipes Lots of Little Tools echo “Hello” | \ wc -c INPUT echo OUTPUTERROR 0 12 INPUT wc OUTPUTERROR 0 12 A Pipe!

Notification % echo “Message” | \ mail –s “Here’s your message” \

Dates % DATESTRING=`date +%Y%m%d` % echo $DATESTRING % man date

FTP the Hard Way ftp –n –u server.wfu.edu <<_FTP_ user username password put FILE _FTP_

FTP with wget wget \ wget –r \

FTP with curl curl –T upload-file \ -u username:password \ ftp://server.wfu.edu/dir/file

Searching: grep % grep rayra /etc/passwd % grep –r rayra /etc % grep –r RAYRA /etc % grep –ri RAYRA /etc % grep –rli rayra /etc

Searching: find % find /home/borwicjh \ -name ‘*.lis’ [all files matching *.lis] % find /home/borwicjh \ -mtime -1 –name ‘*.lis’ [*.lis, if modified within 24h] % man find

Searching: locate % locate.lis [files with.lis in path] % locate log [also finds “/var/log/messages”]

Applied Shell Programming

Make Your Life Easier TAB completion Control+R history cd - Study a UNIX Editor

pushd/popd % cd /tmp % pushd /var/log /var/log /tmp % cd.. % pwd /var % popd /tmp

Monitoring processes ps ps –ef ps –u oracle ps –C sshd man ps

“DOS” Mode Files #!/usr/bin/bash^M FTP transfer in ASCII, or dos2unix infile > outfile

sqlplus JOB=“ZZZTEST” PARAMS=“ZZZTEST_PARAMS” PARAMS_USER=“BORWICJH” sqlplus $BANNER_USER/$BANNER_PW << _EOF_ set serveroutput on set sqlprompt "" EXECUTE WF_SATURN.FZ_Get_Parameters('$JOB', '$PARAMS', '$PARAMS_USER'); _EOF_

sqlplus sqlplus \ $ARG1 $ARG2 $ARG3 if [ $? –ne 0 ] then exit 1 fi if [ -e /file/sql/should/create ] then […use SQL-created file…] fi Ask Amy Lamy!

Passing Arguments % cat > test.sh <<_TEST_ echo “Your name is \$1 \$2” _TEST_ % chmod +x test.sh %./test.sh John Borwick ignore- this Your name is John Borwick

INB Job Submission Template $1 : user ID $2 : password $3 : one-up number $4 : process name $5 : printer name % /path/to/your/script $UI $PW \ $ONE_UP $JOB $PRNT

Scheduling Jobs % crontab -l 0 0 * * * daily-midnight-job.sh 0 * * * * hourly-job.sh * * * * * every-minute.sh 0 1 * * 0 1AM-on-sunday.sh % EDITOR=vi crontab –e % man 5 crontab

Other Questions? Shells and Shell Scripts bash Running Commands bash and Banner in Practice