1. UNIX Shell-Scripting Basics

2. Agenda What is a shell? A shell script? Introduction to bash Running Commands Applied Shell Programming

3. What is a shell? % ▌

4. /bin/bash

5. What is a shell? #!/bin/bash

6. What is a shell? INPUTshellOUTPUTERROR

7. What is a shell? Any Program But there are a few popular shells…

8. Bourne Shells /bin/sh /bin/bash “Bourne-Again Shell” Steve Bourne

9. Other Common Shells C Shell ( /bin/csh ) Turbo C Shell ( /bin/tcsh ) Korn Shell ( /bin/ksh )

10. An aside: What do I mean by /bin ? C Shell ( /bin/csh ) Turbo C Shell ( /bin/tcsh ) Korn Shell ( /bin/ksh )

11. An aside: What do I mean by /bin ? /bin, /usr/bin, /usr/local/bin /sbin, /usr/sbin, /usr/local/sbin /tmp /dev /home/borwicjh

12. What is a Shell Script? A Text File With Instructions Executable

13. What is a Shell Script? % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh %./hello.sh Hello, world

14. What is a Shell Script? A Text File % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh %./hello.sh Hello, world

15. An aside: Redirection cat > /tmp/myfile cat >> /tmp/myfile cat 2> /tmp/myerr cat < /tmp/myinput cat <<INPUT Some input INPUT cat > /tmp/x 2>&1 INPUT env OUTPUTERROR 0 12

16. What is a Shell Script? How To Run % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh %./hello.sh Hello, world

17. What is a Shell Script? What To Do % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh %./hello.sh Hello, world

18. What is a Shell Script? Executable % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh %./hello.sh Hello, world

19. What is a Shell Script? Running it % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh %./hello.sh Hello, world

20. Finding the program: PATH %./hello.sh echo vs. /usr/bin/echo % echo $PATH /bin:/usr/bin:/usr/local/bin: /home/borwicjh/bin % which echo /usr/bin/echo

21. Variables and the Environment % hello.sh bash: hello.sh: Command not found % PATH=“$PATH:.” % hello.sh Hello, world

22. An aside: Quoting % echo ‘$USER’ $USER % echo “$USER” borwicjh % echo “\”” ” % echo “deacnet\\sct” deacnet\sct % echo ‘\”’ \”

23. Variables and the Environment % env […variables passed to sub-programs…] % NEW_VAR=“Yes” % echo $NEW_VAR Yes % env […PATH but not NEW_VAR…] % export NEW_VAR % env […PATH and NEW_VAR…]

24. Welcome to Shell Scripting!

25. How to Learn man  man bash  man cat  man man man –k  man –k manual Learning the Bash Shell, 2 nd Ed. “Bash Reference” Cards http://www.tldp.org/LDP/abs/html/

26. Introduction to bash

27. Continuing Lines: \ % echo This \ Is \ A \ Very \ Long \ Command Line This Is A Very Long Command Line %

28. Exit Status $? 0 is True % ls /does/not/exist % echo $? 1 0

29. Exit Status: exit % cat > test.sh <<_TEST_ exit 3 _TEST_ % chmod +x test.sh %./test.sh % echo $? 3

30. Logic: test % test 1 -lt 10 % echo $? 0 % test 1 == 10 % echo $? 1

31. Logic: test test [ ]  [ 1 –lt 10 ] [[ ]]  [[ “this string” =~ “this” ]] (( ))  (( 1 < 10 ))

32. Logic: test [ -f /etc/passwd ] [ ! –f /etc/passwd ] [ -f /etc/passwd –a –f /etc/shadow ] [ -f /etc/passwd –o –f /etc/shadow ]

33. An aside: $(( )) for Math % echo $(( 1 + 2 )) 3 % echo $(( 2 * 3 )) 6 % echo $(( 1 / 3 )) 0

34. Logic: if if something then : # “elif” a contraction of “else if”: elif something-else then : else then : fi

35. Logic: if if [ $USER –eq “borwicjh” ] then : # “elif” a contraction of “else if”: elif ls /etc/oratab then : else then : fi

36. Logic: if # see if a file exists if [ -e /etc/passwd ] then echo “/etc/passwd exists” else echo “/etc/passwd not found!” fi

37. Logic: for for i in 1 2 3 do echo $i done

38. Logic: for for i in /* do echo “Listing $i:” ls -l $i read done

39. Logic: for for i in /* do echo “Listing $i:” ls -l $i read done

40. Logic: for for i in /* do echo “Listing $i:” ls -l $i read done

41. Logic: C-style for for (( expr1 ; expr2 ; expr3 )) do list done

42. Logic: C-style for LIMIT=10 for (( a=1 ; a<=LIMIT ; a++ )) do echo –n “$a ” done

43. Logic: while while something do : done

44. Logic: while a=0; LIMIT=10 while [ "$a" -lt "$LIMIT" ] do echo -n "$a ” a=$(( a + 1 )) done

45. Counters COUNTER=0 while [ -e “$FILE.COUNTER” ] do COUNTER=$(( COUNTER + 1)) done Note: race condition

46. Reusing Code: “Sourcing” % cat > /path/to/my/passwords <<_PW_ FTP_USER=“sct” _PW_ % echo $FTP_USER %. /path/to/my/passwords % echo $FTP_USER sct %

47. Variable Manipulation % FILEPATH=/path/to/my/output.lis % echo $FILEPATH /path/to/my/output.lis % echo ${FILEPATH%.lis} /path/to/my/output % echo ${FILEPATH#*/} path/to/my/output.lis % echo ${FILEPATH##*/} output.lis

48. It takes a long time to become a bash guru…

49. Running Programs

50. Reasons for Running Programs Check Return Code  $? Get Job Output  OUTPUT=`echo “Hello”`  OUTPUT=$(echo “Hello”) Send Output Somewhere  Redirection:  Pipes

51. Pipes Lots of Little Tools echo “Hello” | \ wc -c INPUT echo OUTPUTERROR 0 12 INPUT wc OUTPUTERROR 0 12 A Pipe!

52. Email Notification % echo “Message” | \ mail –s “Here’s your message” \ borwicjh@wfu.edu

53. Dates % DATESTRING=`date +%Y%m%d` % echo $DATESTRING 20060125 % man date

54. FTP the Hard Way ftp –n –u server.wfu.edu <<_FTP_ user username password put FILE _FTP_

55. FTP with wget wget \ ftp://user:pass@server.wfu.edu/file wget –r \ ftp://user:pass@server.wfu.edu/dir/

56. FTP with curl curl –T upload-file \ -u username:password \ ftp://server.wfu.edu/dir/file

57. Searching: grep % grep rayra /etc/passwd % grep –r rayra /etc % grep –r RAYRA /etc % grep –ri RAYRA /etc % grep –rli rayra /etc

58. Searching: find % find /home/borwicjh \ -name ‘*.lis’ [all files matching *.lis] % find /home/borwicjh \ -mtime -1 –name ‘*.lis’ [*.lis, if modified within 24h] % man find

59. Searching: locate % locate.lis [files with.lis in path] % locate log [also finds “/var/log/messages”]

60. Applied Shell Programming

61. Make Your Life Easier TAB completion Control+R history cd - Study a UNIX Editor

62. pushd/popd % cd /tmp % pushd /var/log /var/log /tmp % cd.. % pwd /var % popd /tmp

63. Monitoring processes ps ps –ef ps –u oracle ps –C sshd man ps

64. “DOS” Mode Files #!/usr/bin/bash^M FTP transfer in ASCII, or dos2unix infile > outfile

65. sqlplus JOB=“ZZZTEST” PARAMS=“ZZZTEST_PARAMS” PARAMS_USER=“BORWICJH” sqlplus $BANNER_USER/$BANNER_PW << _EOF_ set serveroutput on set sqlprompt "" EXECUTE WF_SATURN.FZ_Get_Parameters('$JOB', '$PARAMS', '$PARAMS_USER'); _EOF_

66. sqlplus sqlplus $USER/$PASS @$FILE_SQL \ $ARG1 $ARG2 $ARG3 if [ $? –ne 0 ] then exit 1 fi if [ -e /file/sql/should/create ] then […use SQL-created file…] fi Ask Amy Lamy!

67. Passing Arguments % cat > test.sh <<_TEST_ echo “Your name is \$1 \$2” _TEST_ % chmod +x test.sh %./test.sh John Borwick ignore- this Your name is John Borwick

68. INB Job Submission Template $1 : user ID $2 : password $3 : one-up number $4 : process name $5 : printer name % /path/to/your/script $UI $PW \ $ONE_UP $JOB $PRNT

69. Scheduling Jobs % crontab -l 0 0 * * * daily-midnight-job.sh 0 * * * * hourly-job.sh * * * * * every-minute.sh 0 1 * * 0 1AM-on-sunday.sh % EDITOR=vi crontab –e % man 5 crontab

71. Other Questions? Shells and Shell Scripts bash Running Commands bash and Banner in Practice