1. Shell Programming

2. 222 Lecture Overview  Shell variables  Shell scripts  Control flow and Boolean operators  Shell programming tips  Shell programming examples

3. 333 Shell Variables  Unlike simpler operating systems (such as DOS), UNIX shells provide powerful programming languages  Each shell has a different language  We will only learn C Shell programming: A powerful programming language Its syntax is similar to that of the C language

4. 444 Shell Variables  Like in any programming language, the most basic elements are variables  C Shell variables are always of type string  However, they can be treated as numbers, and can be used for arithmetic operations  Arrays of variables are also supported

5. 555 Defining Variables  Since variables have no type, there is no need for a variable declaration  To define a variable, we simply assign some value to it  Assigning values to variables is done using the set command: set [variable [= value]]...

6. 666 Defining Variables  When set is used without any arguments, the values of all variables currently defined in the shell are printed  When set is used with a name but no value, the variable is created, but assigned an empty value Such variables can be used as Booleans

7. 777 Using Variables  To reference a variable in any shell command, the ' $ ' sign is used set name = John echo $name John echo $shell /bin/tcsh echo $name name: Undefined variable.

8. 888 Un-defining Variables  When a variable is no longer needed, it can be freed using the unset command:  This can also be used for setting the value of a Boolean variable to false unset variable set name unset name echo $name name: Undefined variable.

9. 999 Arrays of Variables  To define an array, a value must be assigned to each of its elements  The list of values is enclosed within parentheses – ' ( ' and ' ) '  Specific values can be accessed using square braces – ' [ ' and ' ] '

10. 10 Arrays of Variables  Unlike C and Java, arrays in the C Shell are 1-based, not 0-based If array a has 3 elements, then they can be accessed as: a[1], a[2], a[3]  To append a new element to an existing array, use the following: set a = ($a new_element)

11. 11 Arrays of Variables – Examples set colors = (red green blue) echo $colors red green blue echo $colors[2]green echo $colors[2-3]green blue set colors = ($colors yellow) echo $colors[4] yellow set shapes = ("" "" "" "") set shapes[4] = square echo $shapes[4] square

12. 12 Numeric Variables  the set command can only assign literal values to variables  To allow the right-hand-sign of an assignment to be a logical or arithmetic expression, we use the ' @ ' command: Note: the space after the ' @ ' is mandatory @ [variable = expression]...

13. 13 Expressions  An expression can contain most of the operators available in C (or in Java):  Arithmetic operators +, -, *, /, %  Relational and logical operators >, =, <=, ==, !=, &&, ||, ! The value of a logical expression is either 0 (for false) or 1 (for true)

14. 14 Expressions  The ' = ' operator can be replaced with other assignment operators: +=, -=, *=, /=, %=  The postfix increment/decrement operators (' ++ ' and ' -- ') can also be used  Since some operators use shell special characters, expressions containing them must be surrounded with parentheses

15. 15 Numeric Variables and Expressions – Examples @ count = 0 echo $count 0 @ count = 5 + 2 echo $count 7 @ result = ($count > 5) echo $result 1 @ count += 5 echo $count 12 @ count++ echo $count 13

16. 16 Numeric Variable Type  Numeric variables in the C Shell are always assumed to be integers – trying to assign fractional values will fail: @ c = 3.5 @: Badly formed number. echo $c c: Undefined variable.

17. 17 Arrays of Numeric Variables  In order to define an array of numeric values, the set command must be used  After the array is initialized, individual values can be changed using set, or using ' @ ' and an expression  Arrays can also be mixed, containing both numeric and string values

18. 18 Arrays of Numeric Variables – Example set ages = (0 0 0 0) @ ages[2] = 15 @ ages[3] = ($ages[2] + 4) echo $ages[3] 19 echo $ages 0 15 19 0 set ages[1] = teen echo $ages teen 15 19 0

19. 19 Special Forms of Variables  Number of elements in an array:  Number of characters in a regular variable:  Determine whether a variable is defined or not (1 if defined, 0 otherwise): $?variable $#array $%variable

20. 20 Special Forms of Variables – Example 0 fri 1 5 unset country echo $?country echo $days[$#days] echo $?country set days = (mon tues wed thurs fri) echo $#days 6set country = "Israel" echo $%country

21. 21 Variable Modifiers  The following modifiers can be appended to a variable, to extract only part of it ModifierAction :r Returns the variable's root (until last '.') :e Returns the variable's extension :h Returns the variable's head (path) :t Returns the variable's tail (file name)

22. 22 Variable Modifiers – Examples set phones_path = ~demo/text/phones.txt echo $phones_path /home/demo/text/phones.txt echo $phones_path:etxt echo $phones_path:r/home/demo/text/phones echo $phones_path:h/home/demo/text echo $phones_path:tphones.txt echo $phones_path:t:rphones

23. 23 Quoting Shell Variables  As we have seen, double quotes ( " ) can be used to quote some special characters  However, this does not suppress variable substitution: set my_text = ~demo/text echo "The file is in the $my_text directory." The file is in the /home/demo/text directory.

24. 24 Quoting Shell Variables  To prevent variable substitution, the text should be enclosed in single quotes ( ' ):  It is also possible to run a command, and store its output in a variable – this is called command substitution echo 'Store your name in the $user_name variable.' Store your name in the $user_name variable.

25. 25 Command Substitution  To use command substitution, the command, along with its arguments, should be enclosed in backquotes ( ` ): set satoshi_phone = `grep Satoshi phones.txt` echo $satoshi_phone NAKAMURA, Satoshi 6453 set name = Satoshi echo $name\'s phone number is: \ `grep $name phones.txt | cut -d" " -f3` Satoshi's phone number is: 6453

26. 26 Pre-defined Shell Variables  Whenever a shell is started, several variables are already defined  The values of some of these variables are constantly updated by the shell  The user can change the values of pre-defined variables to modify the behavior of the shell

27. 27 Pre-defined Shell Variables  Some pre-defined variables have values, others only act as switches (Boolean)  Shell variables that act as switches: $noclobber – if set, does not allow the user to accidentally overwrite an existing file $ignoreeof – when set, prevents accidental log-out using Ctrl-D. To leave a shell, exit or logout must be used

28. 28 Pre-defined Shell Variables  Shell variables that hold a value: $user – contains the name of the current user $home – contains the path to the home directory of the current user $path – contains the command search path $shell – contains the path to the current shell being used  Many more variables exist

29. 29 Lecture Overview  Shell variables  Shell scripts  Control flow and Boolean operators  Shell programming tips  Shell programming examples

30. 30 Shell Scripts  A shell script is a file that contains commands to be executed by the shell  Any command entered in response to a shell prompt can also be used in a script  Additionally, the shell provides control flow commands, designed specifically for use within shell scripts

31. 31 Executing a Shell Script  There are two approaches to running a shell script: Running the script within the current shell  More efficient – no shell start-up required  Variable definitions remain in effect when the script ends, and can be used in the current session Running the script in a newly-created shell  Similar to executing a binary program

32. 32 Executing a Shell Script in the Current Shell  By using the source command, a script file can be executed in the current shell:  The script is assumed to be written in the language of the current shell  If the script was written in the language of a different shell – an error may occur source script_file

33. 33 Executing a Shell Script in a New Shell  Normally, when a script is run, a new shell is created for running it  This can be done explicitly:  This is not very convenient, and still requires the user to know which shell should be used to interpret the script /bin/tcsh script_file

34. 34 Executing a Shell Script in a New Shell  The name of the shell that should be used can be embedded in the script itself  Set the first line of the script to:  If this approach is used, the script file must be made executable: #!/bin/tcsh chmod +x script_file

35. 35 Automatically Executed Shell Scripts  Several scripts are automatically executed by the C Shell at different times:.login – runs at the beginning of a session.logout – runs at the end of a session.tcshrc or.cshrc – runs every time a new shell is created  All of these files must be located in the user's home directory

36. 36 The.tcshrc File  The.tcshrc (or.cshrc ) file is run once when the user logs in, and again every time a new shell is created (for example when a shell script file is executed)  It is normally used for defining local variables and common aliases  Any C Shell command can be used in it

37. 37 A Sample.tcshrc File #!/bin/tcsh # Define aliases. alias l ls -F --color alias ll l -l alias la ll -a alias hgrep 'h | grep' alias + more set noclobber set ignoreeof set nobeep umask 077

38. 38 Command Line Arguments – $argv  The ' $argv ' variable contains the command line arguments: ' $argv[0] ' – the name of the current script ' $argv[1] ', ' $argv[2] ', … – specific command line arguments ' $argv[*] ' – all command line arguments ' $#argv ' – the number of command line arguments

39. 39 Special Variables for Use Within Shell Scripts  The following shortcuts may be used: ' $* ' instead of ' $argv[*] ' ' $1 ' instead of ' $argv[1] ', ' $2 ' instead of ' $argv[2] ', etc. ' $# ' instead of ' $#argv '  ' $< ' is used for getting input from the user: echo –n "Please enter your name: " set user_name = $<

40. 40 Using Temporary Files  The ' $$ ' variable contains the number of the current process, and can be used for generating unique file names  Somewhere in the script:  Before the script ends: ls *.c >.tmp_file.$$ /bin/rm.tmp_file.$$

41. 41 Debugging Shell Scripts  Shell scripts are run using an interpreter, so all errors are found during run-time  In order to debug a shell script, the -x option should be given (either in the first line of the script, or in the command line)  With this option set, any command is printed out just before it is executed

42. 42 Debugging Shell Scripts  Consider a script called debug_script :  First, we run it without the -x option: or #!/bin/tcsh set a = 5 @ a = 7 + 12 echo $a /bin/tcsh debug_script 19 debug_script

43. 43 Debugging Shell Scripts  Now, we run it with the -x option:  Alternatively, the -x option can be inserted directly into the first line of the script #!/bin/tcsh –x... set a = 5 @ a = 7 + 12 echo 19 19 /bin/tcsh –x debug_script

44. 44 Lecture Overview  Shell variables  Shell scripts  Control flow and Boolean operators  Shell programming tips  Shell programming examples

45. 45 Control Flow Commands  The C Shell supports the common control structures if, while and switch  It does not have a for command  Instead, arrays and lists can be traversed using the foreach command  Unlike C and Java, a ' ; ' is not required at the end of a line, and blocks are not surrounded by ' { ' and ' } '

46. 46 The if Control Structure  The C Shell if structure has two forms: A simple form, for executing a single command An if-then-else structure, for executing complex blocks of command  The format of the simple if structure: The command must be on the same line if (expression) simple-command

47. 47 The if-then-else Control Structure  The format of the if-then-else structure: if (expression) then commands else if (expression) then commands... else commands endif

48. 48 if – Example  Read command line arguments: #!/bin/tcsh if ($# == 0 || $# > 2) then echo "Usage: $0:t source [target]" exit 1 endif set source = $1 if ($# == 1) then set target = "~/backup/default" else # There are exactly two arguments. set target = $2 endif

49. 49 File Inquiry Operators  In addition to arithmetic and logical expressions, you can check the status of a file using expressions of the form: where n is a file inquiry operator  For example, is true if filename exists -n filename -e filename

50. 50 File Inquiry Operators OperatorAction -d The file is a directory -e The file exists -o The user owns the file -r The user has read access to the file -w The user has write access to the file -x The user has executable access to the file -z The file is 0 bytes long

51. 51 File Inquiry Operators – Example echo "Hello, world!" > hello if (-e hello) echo file was successfully created file was successfully created if (-d hello) echo file is a directory if (! -z hello) cat hello Hello, world! if (-rw hello) echo file has read/write permissions

52. 52 The foreach Control Structure  The foreach control structure allows convenient traversal of value lists or arrays  Its format: where argument-list is either an array variable, or a list of values or variables foreach loop-index (argument-list) commands end

53. 53 foreach – Examples #!/bin/tcsh foreach file (*.c) wc -l $file end 1593 boxes.c 371 diff_text.c 400 shape.c 370 tools.c ls boxes.c diff_text.c shape.h tools.h boxes.h shape.c tools.c

54. 54 Pattern Matching Operators  The C Shell's pattern matching capabilities can be used not only for file name generation, but also in control structures  The operator ' =~ ' tests whether the value on the left matches the pattern on the right set filename = util.c if ($filename =~ *.[ch]) echo match match

55. 55 Pattern Matching Operators  Similarly, the ' !~ ' pattern returns true if the value does not match the pattern  Reminder: pattern matching syntax is different from regular expression syntax  For example, to match First.Last : Pattern matching: [A-Z]*.[A-Z]* Regular expression: [A-Z].*\.[A-Z].*

56. 56 Pattern Matching Operators – Example #!/bin/tcsh set users = (John.Coltrane Miles.davis Mingus) foreach name ($users) if ($name !~ [A-Z]*.[A-Z]*) then echo Error: invalid user name: $name. else echo New user: $name endif end New user: John.Coltrane Error: invalid user name: Miles.davis. Error: invalid user name: Mingus.

57. 57 The while Control Structure  The while command is the generic control structure for loops, replacing C's for, while and do-while constructs  expression can use any of the operators that can be used with the if command while (expression) commands end

58. 58 while – Examples #!/bin/tcsh set more_input while ($?more_input) set input = $< if ($input == "") then unset more_input else @ plus_ten = $input + 10 echo $input + 10 = $plus_ten endif end echo Done.

59. 59 while – Examples  An example run of the previous script: 45 45 + 10 = 55 3 3 + 10 = 13 12 12 + 10 = 22 Done.

60. 60 The break Command  The break command can be used to exit a command block  Example: #!/bin/tcsh set requested = $1 foreach dir ($path) if ($dir == $requested) then echo The directory was found break endif end

61. 61 The continue Command  The continue command is used to restart an iteration of a loop at the next value  Example: #!/bin/tcsh set requested = $1 foreach dir ($path) if ($dir == /home/demo/bin) continue if ($dir == $requested) then echo The directory was found break endif end

62. 62 The switch Control Structure  The format of the switch structure: switch (test-string) case pattern: commands breaksw case pattern: commands breaksw... default: commands breaksw endsw

63. 63 switch – Examples #!/bin/tcsh set file = $2 switch ($1) case -[mM]: more $file breaksw case -[sS]: sort $file breaksw default: echo "Error: invalid option" breaksw endsw

64. 64 Lecture Overview  Shell variables  Shell scripts  Control flow and Boolean operators  Shell programming tips  Shell programming examples

65. 65 Comments  Comments in the C Shell are marked with ' # ' Reminder: the comment in the first line is not just for documentation, it has a special meaning #!/bin/tcsh # Some sample comments echo hello echo goodbye # This is an in-line comment echo "#This is not a comment" hello goodbye #This is not a comment

66. 66 Script Return Values  Like any program, a shell script can return a numeric value, that signifies its exit status  Values are retuned using the exit command  A non-zero return value signifies that an error has occurred, zero means OK  If no exit command is encountered, the default return value (0) is returned

67. 67 Calling Other Scripts  A script can be called from within another script, by simply using its name as a command  To get the return value of the called script, use the $status variable (short version: $? )  To save the output of the called script in a variable, use backquotes ( ` )

68. 68 Calling Other Scripts  Given a shell script called my_script : set output = `my_script` echo $status 7 cat my_script #!/bin/tcsh echo hello world exit 7 echo $output hello world

69. 69 Quoting Shell Variables  Consider the following sequence:  What happened? set output = "hello world" if ($output != goodbye) echo no if: Expression Syntax. set output = hello if ($output != goodbye) echo no no

70. 70 Quoting Shell Variables  The problem: the variable was expanded into two different strings  To prevent this problem, we surround the variable with double quotes: set output = "hello world" if ("$output" != goodbye) echo no no

71. 71 Appending Strings to Variables  Consider the following problem:  Solution: echo ${prefix}_src tmp_src set prefix = tmp echo $prefix_src prefix_src: Undefined variable.

72. 72 Lecture Overview  Shell variables  Shell scripts  Control flow and Boolean operators  Shell programming tips  Shell programming examples

73. 73 Script Example – underline  Given a script called underline :  The result of running the script: #!/bin/tcsh set one = $1 two = $2 set message = "Comparing users $one and ${two}:" echo $message repeat $%message echo -n = # Print underscore. echo underline bibi zipi Comparing users bibi and zipi: ==============================

74. 74 Getting User Information  Several UNIX commands allow us to get information about other users currently logged-on to the system: who – Shows who is logged on w – Shows who is logged on, and also what they are doing finger – Provides various information about users currently logged-on

75. 75 Shell Script Examples  Let us write some shell scripts for keeping in touch with our colleagues: ison – Checks whether a given user is currently logged-on to the system isidle – Checks if a user is active or idle waitfor – Runs in the background, and issues a message when the given user logs-on

76. 76 Script Example – ison #!/bin/tcsh if ($# < 1) then echo Usage: $0:t user exit 1 endif set user = $1 set finger_user = `finger | grep $user` if ("$finger_user" == "") then echo not logged on else echo logged on endif

77. 77 Script Example – isidle #!/bin/tcsh if ($# < 1) then echo Usage: $0:t user_name exit 1 endif set name = $1 if (`ison $name` == "not logged on") then echo not logged on exit 2 end...

78. 78 Script Example – isidle... set idle_id = `finger | grep Idle | \ sed 's/\(.*\)Idle.*/\1/' | wc -c` @ idle_end = $idle_id + 3 set idle = `finger | grep $name | \ cut -c $idle_id-$idle_end | \ tr -d " " | sort | head -1` if ($idle == "") then echo active else echo idle endif

79. 79 Script Example – waitfor #!/bin/tcsh if ($# < 1) then echo Usage: $0:t user exit 1 endif set user = $1 while (1) set idle = `isidle $1` if ("$idle" == "active") then echo "User $user is now active!" exit 0 endif sleep 10 end

80. 80 Example – Reversing Input #!/bin/tcsh set num = $#argv while ($num > 0) echo -n "$argv[$num] " @ num-- end echo ""

81. 81 Example – File Duplication #!/bin/tcsh if ($#argv != 3) then echo Usage: $0:t source target number exit 1 endif set in = $1 out = $2 num = $3 if (! -r $in || -e $out || ! -w.) then echo Error: problem with read/write permissions. exit 2 endif set i = 0 while ($i < $num) cat $in >>! $out @ i++ end