1 © 2012 John Urrutia. All rights reserved. Chapter 8 The Bourne Again Shell Halleluiah!

2 © 2012 John Urrutia. All rights reserved. Topics Background Creating a Simple Shell Script Command Separation and Grouping Redirecting Standard Error Job Control Directory Stack Manipulation

3 © 2012 John Urrutia. All rights reserved. Background The original Bourne shell was developed by Steve Bourne of AT&T Bell Laboratories.  Many shell scripts have been written to help manage a UNIX system.  The bash has been written to mimic the Bourne shell  Bourne and bash use sh for invocation

4 © 2012 John Urrutia. All rights reserved. Background bash is POSIX compliant Efforts are underway to make it fully POSIX compliant. bash can more closely comply to POSIX with the –posix option.

5 © 2012 John Urrutia. All rights reserved. Topics Background Creating a Simple Shell Script Command Separation and Grouping Redirecting Standard Error Job Control Directory Stack Manipulation

6 © 2012 John Urrutia. All rights reserved. Shell script is not monetary Set of command stored in a file.  Used to support operational functions by combining many command into one group  Provides flow control commands which can alter the order of command execution.

7 © 2012 John Urrutia. All rights reserved. Script Execution Enter the filename on the command line  Must have execute permission  Must be in the PATH

8 © 2012 John Urrutia. All rights reserved. Topics Background Creating a Simple Shell Script Command Separation and Grouping Redirecting Standard Error Job Control Directory Stack Manipulation

9 © 2012 John Urrutia. All rights reserved. Command Separation Newline (nl)X’0D0A’  ends command and initiates execution Semicolon (;)  just separates commands Backslash (\)X’5C0D0A’  at end of line and before you type return Allows command to be continued

10 © 2012 John Urrutia. All rights reserved. Command Separation ( cont. ) Ampersand (&)  execute task in the background Pipe ( | )  pipe

11 © 2012 John Urrutia. All rights reserved. Command Grouping Parenthesis used to group commands  causes Shell to create a subshell  additional processes are created as required when the subshell runs the commands within the parenthesis  (ls ; date; w) ; more  (ls ; date; w) | more

12 © 2012 John Urrutia. All rights reserved. Topics Background Creating a Simple Shell Script Command Separation and Grouping Redirecting Standard Error Job Control Directory Stack Manipulation

13 © 2012 John Urrutia. All rights reserved. Streams Revisited Three streams  standard in < or 0<  standard out > or 1>  standard error 2>

14 © 2012 John Urrutia. All rights reserved. Streams standard I/O cat x y  if x exists and y does not, contents of x and error message due to y are sent to terminal  both standard out and standard error default to the terminal

15 © 2012 John Urrutia. All rights reserved. Streams Continued cat x y 2>error.log  standard error is sent to a file to separate it from the expected results of the command cat x y 2>>newfile 1>>newfile  standard out is redirected to newfile

16 © 2012 John Urrutia. All rights reserved. Here boy << The Here Document  Allows in-stream data to feed a script.  Must start with << and a data delimiter character  Data delimiter character on line by itself - terminates

17 © 2012 John Urrutia. All rights reserved. Topics Background Creating a Simple Shell Script Command Separation and Grouping Redirecting Standard Error Job Control Directory Stack Manipulation

18 © 2012 John Urrutia. All rights reserved. Job Control Ampersand &  tells the Operating system to run the job in the background  User will still be able to interact with the shell  Pure Bourne shell has limited ability. Can not deal with a specific job it has put into background after initial creation. C shell much better.

19 © 2012 John Urrutia. All rights reserved. Job Control (continued) First two jobs in background, c in foreground  a & b & c Entire sequence put into background  a | b | c & All three jobs executed in background  a & b & c & jobs – builtin function displays the jobs running in the background

20 © 2012 John Urrutia. All rights reserved. Job Control (continued) …]$ xman& [1] 1246 …]$ date& [2] 1247 …]$ Tue Sep 11 6:17 PDT 2001 [2]+ Donedate …]$ find /usr –name ace –print > out & [2] 1269 …]$ jobs [1]- Runningxman & [2]+ Runningfind /usr –name ace …

21 © 2012 John Urrutia. All rights reserved. Job Control (continued) …]$ (sleep 5;cat>mytext)& [1] 1343 …]$ date Tue Sep 11 6:30 PDT 2001 [1]+ Stopped (tty input) (sleep 5;cat>mytext) …]$ fg (sleep 5;cat>mytext) Remember to let the cat out!

22 © 2012 John Urrutia. All rights reserved. Topics Background Creating a Simple Shell Script Command Separation and Grouping Redirecting Standard Error Job Control Directory Stack Manipulation

23 © 2012 John Urrutia. All rights reserved. Directory Stack Manipulation You can store a list of frequently used directories in a stack Push-down (LIFO)  The three stack commands  dirs  pushd  popd

24 © 2012 John Urrutia. All rights reserved. Directory Stack Manipulation dirs – displays all the directories in the stack  When stack is empty displays the Working Directory (~ is your home directory)

25 © 2012 John Urrutia. All rights reserved. Directory Stack Manipulation pushd someDirectoryName–  Change working directory  “pushes” directory onto the stack  Display the directory stack

26 © 2012 John Urrutia. All rights reserved. Directory Stack Manipulation pushd –  “swaps” top of stack with next element  Change working directory to top of stack  Display the directory stack

27 © 2012 John Urrutia. All rights reserved. Directory Stack Manipulation pushd +2 –  “swaps” top of stack with +2 element  Change working directory to top of stack  Display the directory stack

28 © 2012 John Urrutia. All rights reserved. Directory Stack Manipulation popd –  “pops” removes top entry from stack  Change working directory to top of stack

29 © 2012 John Urrutia. All rights reserved. Directory Stack Manipulation popd +2 –  Removes the 3 rd entry from stack  DOES NOT CHANGE Working Directory

30 © 2012 John Urrutia. All rights reserved. Topics Processes Parameters and Variables History Alias Command-line Expansion

31 © 2012 John Urrutia. All rights reserved. Processes and Subshells A process is the execution of a command  login to LINUX  execution of a LINUX utility  execution of a shell script creates a new process  script commands each start a new process Process structure is hierarchical  Parent processes spawn or fork children

32 © 2012 John Urrutia. All rights reserved. PID’s … Process ID’s  Sequentially Assigned by the system when a process is started ps  Displays all processes for your userid

33 © 2012 John Urrutia. All rights reserved. ps –Al All Please Displays a long list of all processes including those not attached to a terminal. Command preceded by – was initiated by the init process

34 © 2012 John Urrutia. All rights reserved. Process status All processes have a status that can change to:  D – Sleeping Do not interrupt (Can’t)  N – Reduced priority  R – Available for execution (running)  S – Sleeping (Waiting)  T – Stopped or being traced  Z – Zombie waiting for child to terminate

35 © 2012 John Urrutia. All rights reserved. Process Flow User logs in: shell process is created User issues command, enters return  Shell creates a subshell  child process is forked or spawned  unless the command is built into the bourne shell process

36 © 2012 John Urrutia. All rights reserved. Process flow (cont.)  Subshell is a clone of the parent shell  Subshell tries to exec the command  If it’s a program, the program runs  If it’s a shell script, exec fails and subshell interprets commands.  If it’s neither command fails

37 © 2012 John Urrutia. All rights reserved. Process Flow Parent Shell sleeps until child shell finishes  (unless job was executed in background) Variables that are used in a parent can be sent to a child, but the reverse is not true.

38 © 2012 John Urrutia. All rights reserved. Process Flow Shell Scripts need to have execute permission. You just type the file name as you would a command. Alternative (new subshell): sh file Alternative (current shell): file

39 © 2012 John Urrutia. All rights reserved. Starting bash When bash is called, various startup files are run to issue commands and define environmental variables Which startup file(s) begin depends upon how bash is called Use these startup files to make the shell work for you

40 © 2012 John Urrutia. All rights reserved. Login shells Login shells are called with the --login option We don’t usually do this – it’s done for us Will first run /etc/profile, which contains global default settings

41 © 2012 John Urrutia. All rights reserved. Login shells Next, it will attempt to run ~/.bash_profile ~/.bash_login ~./profile

42 © 2012 John Urrutia. All rights reserved. Login shells, con’t Commands in those three files can override the defaults in /etc/profile Once one of those files are executed, control is passed to the user When the user logs out, bash runs ~/.bash_logout  Usually clears temporary information

43 © 2012 John Urrutia. All rights reserved. Interactive nonlogin shells Shells that you spawn yourself by typing bash Runs ~/.bashrc  This file is usually called by ~/.bash_profile for login shells  Often this file will also run /etc/bashrc, which again contains system defaults

44 © 2012 John Urrutia. All rights reserved. Noninteractive shells These are the shells used to run scripts These shells do not run any of the aforementioned startup files They do however inherit the calling shell’s environmental variables marked for export

45 © 2012 John Urrutia. All rights reserved. Noninteractive shells So basically anything you set for the login shell is set for the noninteractive shell

46 © 2012 John Urrutia. All rights reserved. Working with Startup Files In the end, these startup files are just shell scripts Obey the same rules and conventions that scripts must use for the particular shell you’re using Most important files are probably.bashrc and.bash_profile

47 © 2012 John Urrutia. All rights reserved. Startup Files, con’t Simplify – have.bash_profile call.bashrc Just edit the startup files in your favorite editor When done, you can apply changes to your current shell using either. or source Otherwise, logout and login again

48 © 2012 John Urrutia. All rights reserved. Creating a Shell Script Use a text editor like vi First line should start with #! Followed by the absolute pathname of the shell that is to interpret the script. (default is C shell)  #!/bin/sh Lines which start with a # are comments  (except the special line mentioned above)

49 © 2012 John Urrutia. All rights reserved. Dot s de way to Execute it The exec command  Executes scripts or programs  Runs under the same PID  Provides access to the original environment variables  Terminates current process.

50 © 2012 John Urrutia. All rights reserved. Dot s de way to Exec it The dot command  Executes only scripts  Runs under the same PID  Provides access to the current environment variables  Returns to next command in script

51 © 2012 John Urrutia. All rights reserved. Topics Processes Parameters and Variables History Alias Command-line Expansion

52 © 2012 John Urrutia. All rights reserved. Parameters & Variables Parameters  Any user accessible variable  Positional on the command line Two types of Variables  Keyword Shell variables  User Shell variables

53 © 2012 John Urrutia. All rights reserved. Keyword Shell Variables HOME (contains login directory) PATH (Used by shell to locate commands you type in)  /usr/bin:/usr/sbin:/class/n01/bin: MAIL (contains name of central post office file for your mail) PS1, PS2 (primary and secondary prompts)

54 © 2012 John Urrutia. All rights reserved. Keyword Variables (continued) CDPATH  like PATH, except used by cd command TZ  timezone IFS  Internal field separator. Blanks, tabs and newline are defaults

55 © 2012 John Urrutia. All rights reserved. User Created Variables Create a variable by giving a name of your choice and an optional value  name=charlie  NO blanks around the equal sign!! Remove variable  unset name Keep variable but remove value  name=

56 © 2012 John Urrutia. All rights reserved. Readonly Shell Variables Two types: user created variable that has been declared to be readonly  readonly name  keeps later statements from changing the value Special Shell variables  Positional Variables  Miscellanous variables

57 © 2012 John Urrutia. All rights reserved. Positional Variables $1 through $9  Keep the first nine arguments entered after the name of the shell script …]$ myscrpt aardvark dog cat  $1 will contain the word aardvark  $2 will contain the word dog  $3 will contain the word cat

58 © 2012 John Urrutia. All rights reserved. Miscellaneous Variables $* contains all arguments (not just the first one) similar to $*, except that it internally quotes each argument. $# total number of arguments $$ process id number (pid) of current process

59 © 2012 John Urrutia. All rights reserved. Shift command Promotes values of each positional variable to the left. Contents of $1 go to ‘bit bucket’ Contents of $2 go to $1 Contents of $3 go to $2 etc (etcetera, not etci)

60 © 2012 John Urrutia. All rights reserved. Set `em up - Shift `em out set  Populates the $[1-9] variables shift  Moves each $ variable 1 position to the left.

61 © 2012 John Urrutia. All rights reserved. d1]$ date Thu Apr 20 11:28:38 PDT 2000 d1]$ set `date` d1]$ echo $1 Thu d1]$ shift d1]$ echo $1 Apr The Shift ing sands of time

62 © 2012 John Urrutia. All rights reserved. = or Unset – That’s the ? =  Creates and/or populates any user variable unset  Removes a user variable

63 © 2012 John Urrutia. All rights reserved. export - it to the World User variables are local to the current process export  Gives child processes copies of user variables

64 © 2012 John Urrutia. All rights reserved. I do declare declare (typeset) – sets attributes for user variables  f – identify as a function name  i – integer uses binary storage  x – marks for export  r – Read Only …]$ declare –ix export_var=6

65 © 2012 John Urrutia. All rights reserved. What variables? declare and set  Display all variables and current values declare [-f -i -x -r]  Display all variables and current values containing one or more of the specified attributes

66 © 2012 John Urrutia. All rights reserved. Read ‘em and weep read var1 var2 var3 …  Takes standard input and populates one or more user variables  IFS (internal field separator) delimited  Default is space tab newline

67 © 2012 John Urrutia. All rights reserved. Dots impossible! How do I change shell variables permanently?  Create or modify the.profile file the next time you login the changes will be there.  To do it now execute the. Command …]$.profile

68 © 2012 John Urrutia. All rights reserved. Command substitution Replaces a command with the output of a command  Similar to pipe but does not create a file Two syntaxes  ` command ` – Old Syntax  $( command ) – New Syntax

69 © 2012 John Urrutia. All rights reserved. Where’s the exit ? exit number  Allows you to set a condition or return code from the process  This value is referenced by $?

70 © 2012 John Urrutia. All rights reserved. Topics Processes Parameters and Variables History Alias Command-line Expansion

71 © 2012 John Urrutia. All rights reserved. History The history mechanism was adopted from the C shell  It maintains a list of line commands  Each entry is called an event  Each event can be re-called and re- executed via a shorthand command by using the event number.

72 © 2012 John Urrutia. All rights reserved. A Historical Re-Execution The built-in command fc (fix command)  Allows viewing of previous commands  Allows correction of a previous command And re-execution of the culprit fc –l [ first last ]  Lists all commands that meet the criteria  first and last can be either event # or a string

73 © 2012 John Urrutia. All rights reserved. A Historical Re-Execution fc –e editor [ first last ]  Edits all commands that meet the criteria with the editor specified.  FCEDIT varaible will set the default editor if one is not specified As soon as you exit the editor everything in the buffer gets executed!

74 © 2012 John Urrutia. All rights reserved. A Historical Re-Execution fc –s event# [ oldstring=newstring ]  Re-Executes the specified event# without entering editor mode  If present a string substitution occurs … ]$ fc -s old.data.file=new.data.file

75 © 2012 John Urrutia. All rights reserved. A Historical Re-Execution Event Number Execution  !! – Re-Executes the previous event  !44 – Re-Executes event 44  !-4 – Re-Executes 4 th previous event  !$ Identifies the last token of the previous command line

76 © 2012 John Urrutia. All rights reserved. A Historical Re-Execution Event Text Execution  !cat – Re-Executes the previous event beginning with “cat”  !?cat? – Re-Executes the previous event containing the string “cat”  !$ Identifies the last token of the previous command line

77 © 2012 John Urrutia. All rights reserved. A Historical Re-Execution Event Text substitution with event modifiers  !!:s/car/cat – Re-Executes the previous event after substituting “cat” for “car”  ^car^cat – Shorthand for the above

78 © 2012 John Urrutia. All rights reserved. A Historical Re-Execution Other Event Modifiers  P – No execution just print  h – head removes last pathname element  e – removes all but the filename extension  r – remove the filename extension  t – tail removes all but last pathname element  [g]s/old/new/ – Substitute old with the new

79 © 2012 John Urrutia. All rights reserved. Topics Processes Parameters and Variables History Alias Command-line Expansion

80 © 2012 John Urrutia. All rights reserved. Alias This built-in was borrowed from the C shell. Allows substitution of a string for a command.  alias [ name [= command value ] ] Will not work inside a script. or any other name would work

81 © 2012 John Urrutia. All rights reserved. or any other name would work Alias or any other name would work To ‘ (quote) or “ (double quote)  ‘ – expands shell variables at execution  “ – expands shell variables at definition Let’s analyze the following: …]$ alias p1=“echo my prompt is $PS1” …]$ alias p2=‘echo my prompt is $PS1’ …]$ PS1=“Hello?” Hello?

82 © 2012 John Urrutia. All rights reserved. or any other name would work Alias or any other name would work Hello? p1 My prompt is \W]\$ Hello? P2 My prompt is Hello?

83 © 2012 John Urrutia. All rights reserved. Topics Processes Parameters and Variables History Alias Command-line Expansion

84 © 2012 John Urrutia. All rights reserved. The Expand-O-Matic Shell Command-Line expansion  Before execution  Shell parses the command line into tokens  Tokens are processed to expand the command line  The entire command line is then executed

85 © 2012 John Urrutia. All rights reserved. The Expand-O-Matic Shell Brace expansion { }  Used to specify filenames when pathname expansion is not required or string substitution is used  Consists of:  Preamble  Postamble …]$ echo b{a,e,i,o,u}d bad bed bid bod bud

86 © 2012 John Urrutia. All rights reserved. The Expand-O-Matic Shell Tilde expansion ~  Parses the token for a / or a Space  If parsed token is not null  Test the value for login name and use if valid  If not valid no substitution  If token is null  Substitute value of HOME for the ~  ~- previous working directory  ~+ current working directory

87 © 2012 John Urrutia. All rights reserved. The Expand-O-Matic Shell Parameter expansion  When a dollar sign ($) is followed by a number  The positional parameter from the command line is substituted

88 © 2012 John Urrutia. All rights reserved. The Expand-O-Matic Shell Variable expansion  Special case – when a dollar sign ($) is followed by a variable name. The shell substitutes the variables value.  General case – ${variable}  The braces insulate the variable from what is around it.  Without insulation substitution may not occur

89 © 2012 John Urrutia. All rights reserved. The Expand-O-Matic Shell …]$ BORK=borken …]$ FORKOLA=$BORKforkola …]$ echo $FORKOLA …]$ FORKOLA=${BORK}forkola …]$ echo $FORKOLA borkenforkola

90 © 2012 John Urrutia. All rights reserved. The Expand-O-Matic Shell Command Substitution  $( command ) or $’ command ’  Replaces token with the standard output of the command …]$ echo $(cat animals) dog cat aardvark Dog mouse cat elephant zebra

91 © 2012 John Urrutia. All rights reserved. The Expand-O-Matic Shell Arithmetic Expansion  $[ expression ]  Evaluates expression and replaces token with the value  Treats all variables as integers  Converts strings to integers  Uses the same syntax as the C language  Operators + - * / % =

92 © 2012 John Urrutia. All rights reserved. The Expand-O-Matic Shell Arithmetic Expansion  let – Built-in  Allows arithmetic evaluation without expansion  Sets the exit code based on the last expression evaluated  1 – if the value is zero  0 – for all other values let a=5+3 b=7-6 echo $a $b 8 1

93 © 2012 John Urrutia. All rights reserved. The Expand-O-Matic Shell Word Splitting  IFS – Infernal Field Separator  Default is one of space tab or newline  Adds additional characters as field separators.  Only works on fields that have some form of expansion  Caution – Changing this will affect the way the shell operates

94 © 2012 John Urrutia. All rights reserved. The Expand-O-Matic Shell …]$ a=w:x:y:z …]$ cat $a cat: w:x:y:z: No such file or directory …]$ IFS=“:” …]$ cat $a cat: w: No such file or directory cat: x: No such file or directory cat: y: No such file or directory cat: z: No such file or directory

95 © 2012 John Urrutia. All rights reserved. The Expand-O-Matic Shell Pathname Expansion  Uses the wildcard tokens  * - ? - [ - ]  Globbing  Ambiguous File reference

96 © 2012 John Urrutia. All rights reserved. The Expand-O-Matic Shell The Order of Expansion 1.Brace – { } 2.Tilde – ~ 3.Parameter 4.Variable 5.Command substitution 6.Arithmetic 7.Word splitting 8.Pathname