Shell Programming

The UNIX Shell The UNIX shell listens to what you type and executes commands at your request. ls, lpr, mv, rm, telnet, netscape, ... Command Library Printers Files Memory User command: lpr file UNIX Kernel UNIX Shell result or status results (on screen)

Popular Shells sh Bourne shell (the original shell) csh C-shell (pronounced as “sea shell”) tcsh Like csh with more functions bash “Bourne again” shell ksh Korn shell zsh Z-shell

More Utilities (2) echo Display command line input to screen $ echo Hi, I am Bill, the President of the US! Hi, I am Bill, the President of the US! date Print the date and time $ date Wed Feb 3 12:13:07 HKT 1999

More Utilities (3) head Display first few lines of file $ head -2 letter3 Ms. Lewinski: It is getting late. Please order some pizza and stop tail Display last few lines of file $ tail -2 letter3 Thanks! Bill grep Find a pattern in a file $ grep ”some pizza” letter3

More Utilities (4) sort Sort the lines in lexical order $ sort letter3 Bill It is getting late. Please order some pizza and stop Ms. Lewinski: Thanks! by my office. We'll tidy up a few more things before calling it a night. $ sort -r letter3

Input/Output Redirection (2) Using the “>” character after a command to redirect output to a named file: $ sort names > names.sort $ uniq names.sort Bill Clinton Bill Gates Monica Lewinski This will create a file test: $ cat > test type line 1 type line 2 <ctrl-d> $ cat test names sort names.sort uniq display

Input/Output Redirection (3) Using the “>>” character after a command to redirect output to APPEND to a named file: Typing to the end of a file $ cat >> test type line 3 type line 4 <ctrl-d> $ cat test type line 1 type line 2 Append file1 to file2 $ cat file1 >> file2

Input/Output Redirection (4) Using the “<” character after a command to redirect input from a named file: $ uniq < names.sort This is the same as: $ uniq names.sort Using input and output redirection together: $ sort < names > names.sort names.sort uniq display names sort names.sort

Pipes (1) The standard output of a program can be “piped” into the standard input of another program: $ sort names | uniq Bill Clinton Bill Gates Monica Lewinski names sort uniq display

Pipes (2) Several pipes can be connected: $ sort names | uniq | grep "Bill" Bill Clinton Bill Gates Pipes and I/O redirection can be used together: $ sort -r names | uniq >names.rev $ cat names.rev Monica Lewinski

Directory Shorthands “.” is the directory itself “..” is the parent directory In most shells “~” means your home directory) ~user means user’s home directory, so: $ more ~jbond/.plan looks at the file .plan in /home/jbond, which is jbond’s home directory.

Special Directories “/” (pronounced “slash” and also called “the “root”) is the ancestor of all files in the file system /bin and /usr/bin contain UNIX utilities (e.g., cat) /dev contains special files which describe “devices” such as terminals and printers /etc has administrative programs like password files /tmp is for temporary files; periodically deleted Every directory has at least two entries: “.” is the directory itself, and “..” is the directory’s parent

Security and Access Permissions (1) There are three types of users: The owner of the file (user) The group of the file (group) Anyone else (other) There are three types of permission (independent of each other): Read permission Write permission Execute permission

Security and Access Permissions (2) Use ls -l to see file permissions -rw-r--r-- 1 jbond cs 154 Feb 4 15:00 letter3 There are four sets of items in the permissions: -rw-r--r-- The type is: “-” regular files, “d” directories , “l” symbolic links. The next nine characters indicate if the file is readable, writable, or executable for the file owner, the file group, or other users, respectively. Byte size Name #links Group Last modification Permissions User type user group other

Changing Permissions (1) The chmod command is used to modify permissions. chmod can only be used by the owner of a file/dir (or the administrator root). The arguments are: chmod [ugoa] [+-=] [rwxdd] [file/dir] In other words: Optionally, one of the characters: u (user/owner), g (group), o (other), or a (all). Optionally, one of the characters: + (add permission), - (remove permission), or = (set permission). Any combination of the characters r (read), w (write), or x (execute).

Changing Permissions (2) Another way to change permission is to use numbers representing the permissions. The arguments are: chmod nungno [file/dir] Code table (3 bits) --- 0 r-- 4 --x 1 r-x 5 -w- 2 rw- 6 -wx 3 rwx 7 nu - user’s permission code ng - group’s permission code no - other’s permission code -rwxrwxrwx

Permission Example To let everybody read or write the file letter1 $ chmod a+rw letter1 (chmod 666 letter1) $ ls -l letter1 -rw-rw-rw- 1 jbond cs 154 Feb 4 15:00 letter1 To allow user to execute file letter1 $ chmod u+x letter1 (chmod 766 letter1) -rwxrw-rw- 1 jbond cs 154 Feb 4 15:00 letter1* To not let “other” to read or write file letter1 $ chmod o-rw letter1 (chmod 760 letter1) -rwxrw---- 1 jbond cs 154 Feb 4 15:00 letter1* To let “group” only read the file letter1 $ chmod g=r letter1 (chmod 740 letter1) -rwxr----- 1 jbond cs 154 Feb 4 15:00 letter1*

Alias The C Shell has the alias command, which allows you to create command shortcuts. $ alias dir "ls -F" $ alias rm "rm -i" $ alias + "chmod u+x *" $ alias - "chmod u-x *" $ alias 111 "cd ~horner/111" $ pwd /bin $ 111 /homes/horner/111 If you put the alias commands in your .cshrc file, you can use them every time you login.

Combining Commands (2) Commands can be grouped together using parentheses There are two main reasons to group commands: To create a “single command” out of a group of commands (especially useful before a pipe): $ (cat letter1; head -2 names) | sort >list To run a set of commands in their own subshell (especially when trying to limit the effect of a cd command): $ (cd secret; ls | wc -l); ls | wc -l 3 25 This line has the effect of counting the files in secret, and then counting the files in the current directory.

A shell can be used in one of two ways: Shells A shell can be used in one of two ways: A command interpreter, used interactively A programming language, to write shell scripts (your own custom commands)

Shell Scripts A shell script is just a file containing shell commands, but with a few extras: The first line of a shell script should be a comment of the following form: #!/bin/sh for a Bourne shell script. Bourne shell scripts are the most common, since C Shell scripts have buggy features. A shell script must be readable and executable. chmod +rx scriptname As with any command, a shell script has to be “in your path” to be executed. If “.” is not in your PATH, you must specify “./scriptname” instead of just “scriptname”

Shell Script Example Here is a “hello world” shell script: $ ls -l -rwxr-xr-x 1 horner 48 Feb 19 11:50 hello* $ cat hello #!/bin/sh # comment lines start with the # character echo "Hello world" $ hello Hello world $ The echo command functions like a print command in shell scripts.

Shell Variables (1) The user variable name can be any sequence of letters, digits, and the underscore character, but the first character must be a letter. To assign a value to a variable: number=25 name="Bill Clinton" There cannot be any space before or after the “=“ Internally, all values are stored as strings.

Shell Variables (2) To use a variable, precede the name with a “$”: $ cat test1 #!/bin/sh number=25 name="Bill Clinton" echo "$number $name" echo '$number $name' $ test1 25 Bill Clinton $number $name $ Note the difference of single and double quotes in this example.

User Input (1) Use the read command to get and store input from the user. $ cat test2 #!/bin/sh echo "Enter name: " read name echo "How many friends do you have? " read number echo "$name has $number friends!” $ test2 Enter name: Bill Clinton How many friends do you have? too many Bill Clinton has too many friends!

User Input (2) read reads one line of input from the keyboard and assigns it to one or more user-supplied variables. $ cat test3 #!/bin/sh echo "Enter name and how many friends:" read name number echo "$name has $number friends!" $ test3 Enter name and how many friends: Bill Clinton 63 Bill has Clinton 63 friends! BillC 63 BillC has 63 friends! Bill Bill has friends! Leftover input words are all assigned to the last variable.

Special Symbol : $ (1) Use a backslash before $ if you really want to print the dollar sign: $ cat test4 #!/bin/sh echo "Enter amount: " read cost echo "The total is: \$$cost" $ test4 Enter amount: 18.50 The total is $18.50

Special Symbol : $ (2) You can also use single quotes for printing dollar signs. Single quotes turn off the special meaning of all enclosed dollar signs: $ cat test5 #!/bin/sh echo "Enter amount: " read cost echo 'The total is: $' "$cost" $ test5 Enter amount: 18.50 The total is $ 18.50

Simple calculation using expr (1) Shell programming is not good at numerical computation, it is good at text processing. However, the expr command allows simple integer calculations. Here is an interactive Bourne shell example: $ i=1 $ expr $i + 1 2 To assign the result of an expr command to another shell variable, surround it with backquotes: $ i=`expr $i + 1` $ echo "$i"

Simple calculation using expr (2) The * character normally means “all the files in the current directory”, so you need a “\” to use it for multiplication: $ i=2 $ i=`expr $i \* 3` $ echo $i 6 expr also allows you to group expressions, but the “(“ and “)” characters also need to be preceded by backslashes: $ echo `expr 5 + \( $i \* 3 \)` 11

Simple calculation using expr (3) - Example $ cat test6 #!/bin/sh echo "Enter height of rectangle: " read height echo "Enter width of rectangle: " read width area=`expr $height \* $width` echo "The area of the rectangle is $area" $ test6 Enter height of rectangle: 10 Enter width of rectangle: 5 The area of the ractangle is 50 10.1 5.1 expr: non-numeric argument Does not work for floats!

Backquotes – Command Substitution A command or pipeline surrounded by backquotes causes the shell to: Run the command/pipeline Substitute the output of the command/pipeline for everything inside the quotes You can use backquotes anywhere: $ whoami clinton $ cat test7 #!/bin/sh user=`whoami` numusers=`who | wc -l` echo "Hi $user! There are $numusers users logged on." $ test7 Hi clinton! There are 6 users logged on.

Backquote Example (1) $ cat big #!/bin/sh dir=`pwd` big=`ls -l | sort +4 | tail -1 | cut -c55-70` size=`ls -l | sort +4 | tail -1 | cut -c33-40` echo "The biggest file in $dir is $big." echo "$big is $size bytes." $ big The biggest file in /homes/horner/111 is letter1. letter1 is 155 bytes. sort +4 sorts in increasing order based on the fourth field (fields start from zero, and are separated by whitespace). cut -cx-y cuts characters from position x to position y (the first position is 1). 1 2 3 4 5 6 123456789012345678901234567890123456789012345678901234567890123 -r--r--r-- 1 horner cs 155 Feb 12 16:00 letter1 field4 field0

Backquote Example (2) $ wc -w letter1 7 letter1 $ cat big1 #!/bin/sh dir=`pwd` big=`ls -l | sort +4 | tail -1 | cut -c55-70` nline=`wc -l $big | cut -c6-8` nword=`wc -w $big | cut -c6-8 ` nchar=`wc -c $big | cut -c6-8 ` echo "The biggest file in $dir is $big." echo "$big has $nline lines, $nword words, $nchar characters." $ big1 The biggest file in /homes/horner/111 is letter1. letter1 has 7 lines, 29 words, 155 characters. wc -w counts the number of words in the file (“words” are separated by whitespace). wc -c counts the number of characters in the file.

The shell allows several control flow statements: if while for