Shell Programming Winter Quarter 2005 Rolando V. Raqueño

Unix Power Tools Reference Chapter 35 Shell Programming for the Uninitiated Chapter 36 Shell Programming for the Initiated Winter Quarter 2005 Rolando V. Raqueño

UNIX Tip of the Day Directory maneuvering commands pushd, popd, and, dirs % cd /usr/tmp % pwd /usr/tmp % pushd ~rvrpci /cis/staff/rvrpci Winter Quarter 2005 Rolando V. Raqueño

UNIX Tip of the Day % dirs /cis/staff/rvrpci /usr/tmp % pushd % pwd /usr/tmp /cis/staff/rvrpci % pushd /usr/local/bin Winter Quarter 2005 Rolando V. Raqueño

UNIX Tip of the Day % dirs /usr/local/bin /usr/tmp /cis/staff/rvrpci % pwd /usr/local/bin % pushd /usr/tmp /usr/local/bin Winter Quarter 2005 Rolando V. Raqueño

UNIX Tip of the Day % dirs /usr/tmp /usr/local/bin /cis/staff/rvrpci % pwd /usr/tmp % pushd +2 Winter Quarter 2005 Rolando V. Raqueño

UNIX Tip of the Day % dirs /cis/staff/rvrpci /usr/tmp /usr/local/bin % popd /usr/tmp /usr/local/bin Winter Quarter 2005 Rolando V. Raqueño

IMPORTANT UNIX Concepts Environment and Shell Variables These allow you to customize your UNIX environment They are different in terms of their SCOPE SCOPE determines the visibility of a variable Winter Quarter 2005 Rolando V. Raqueño

Other IMPORTANT UNIX Concepts Environment Variable Examples are TERM and DISPLAY Set a particular variable to a value by using the setenv command You can print the value of a particular variable or all the environment variable using the printenv command Winter Quarter 2005 Rolando V. Raqueño

Shell Variables Shell variables are similar to Environment variables except they have a limited scope, i.e., they exist only in the shell which they are defined. Environment variables on the other hand, exist in all its children shells Winter Quarter 2005 Rolando V. Raqueño

Environment & Shell Variables Why is this important? UNIX uses Environment and Shell Variables control a number of processes Customizes your working environment Variables used for UNIX Scripts They are typically defined and initialized in your .login and .cshrc files Winter Quarter 2005 Rolando V. Raqueño

Useful Shell Variables filec #Allows file completion path #List of command directories history #Number of commands to remember Winter Quarter 2005 Rolando V. Raqueño

What is shell programming? automate a set of UNIX commands. Just like any programming language “wrappers” black box a customized collection of UNIX commands. Example of shell programs .login .cshrc Shell programming is a way to automate a set of UNIX commands. It behaves in much the same manner as other programming languages and allows you to process repetitive tasks (batch files) Shell programming also allows you to create “wrappers” which essentially treat as a black box a customized collection of UNIX commands. Winter Quarter 2005 Rolando V. Raqueño

.login file set path=($HOME/bin /usr/local/bin \ /usr/ucb /usr/sbin /bin /usr/bin \ /usr/bin/X11 .) stty dec new tset -I -Q set mail=/usr/spool/mail/$USER set editmode = emacs umask 077 biff n date Winter Quarter 2005 Rolando V. Raqueño

.cshrc file if ($?prompt) then set notify set history = 100 set savehist = 100 alias pd pushd alias pop popd alias vt100 "set term = vt100" endif Winter Quarter 2005 Rolando V. Raqueño

When these files are executed? .cshrc is automatically executed when you start a new shell .login only gets executed once when you first login Can be re-executed by giving the source command % source .cshrc Winter Quarter 2005 Rolando V. Raqueño

Other useful .login and .cshrc entries set filec set cdpath=(~ ~rvrpci/pub ~/mythesis) Other common entries set path=( $path /usr/local/bin) set path=(/usr/local/bin $path) Winter Quarter 2005 Rolando V. Raqueño

User defined shell program Determine name of command Determine input, output, and option arguments Determine UNIX commands to execute Establish error trapping Make shell program executable Winter Quarter 2005 Rolando V. Raqueño

A simple shell program dd command to swap bytes % dd if=input.dat of=output.dat bs=2 conv=swab Very difficult to remember Very little utility to non-UNIX geeks (normal people) Unfortunately, this is very difficult to remember and you want other people to be able to use this command without being UNIX experts Winter Quarter 2005 Rolando V. Raqueño

We would rather see... % swap_bytes input.dat output.dat Winter Quarter 2005 Rolando V. Raqueño

Special Shell Variables Set % swap_bytes input.dat output.dat $0 $1 $2 command $argv[1] $argv[2] Winter Quarter 2005 Rolando V. Raqueño

Another Special Shell Variables % swap_bytes input.dat output.dat $#argv Indicates how many arguments are present In this case, 2 Winter Quarter 2005 Rolando V. Raqueño

shell program swap_bytes #!/bin/csh -f dd if=$1 of=$2 bs=2 conv=swab Winter Quarter 2005 Rolando V. Raqueño

Making swap_bytes shell script executable % ls -l swap_bytes -rw------- ... swap_bytes % chmod u+x swap_bytes -rwx------ ... swap_bytes Winter Quarter 2005 Rolando V. Raqueño

To run swap_bytes swap_bytes becomes just another unix command! % swap_bytes input.dat output.dat Winter Quarter 2005 Rolando V. Raqueño

Limitation of swap_bytes No error trapping Should give usage when typing command % swap_bytes usage: swap_bytes input_file output_file Winter Quarter 2005 Rolando V. Raqueño

Improvement to swap_bytes #!/bin/csh -f if ( $#argv != 2 ) then echo "usage: $0 input_file output_file" exit 1 endif dd if=$1 of=$2 bs=2 conv=swab Winter Quarter 2005 Rolando V. Raqueño

Commad exit status By convention exit 0 Indicates successful command completion exit 1 (or non-zero) Indicates some error condition Winter Quarter 2005 Rolando V. Raqueño

Informational message from swap_bytes UNIX style informational message % swap_bytes usage: swap_bytes input_file output_file Winter Quarter 2005 Rolando V. Raqueño

Interactive swap_bytes If you want a “friendlier” shell program Have it query the user for the inputs Another special shell variable can be used $< Winter Quarter 2005 Rolando V. Raqueño

Interactive swap_bytes #!/bin/csh -f if ( $#argv != 2 ) then echo -n "Please enter the input file> " set input=$< echo -n "Please enter the output file> " set output=$< endif dd if=$input of=$output bs=2 conv=swab Winter Quarter 2005 Rolando V. Raqueño

Interactive swap_bytes example User simply types the command % swap_bytes Please enter the input file> input.dat Please enter the output file> output.dat Winter Quarter 2005 Rolando V. Raqueño

UNIX Quotes Winter Quarter 2005 Rolando V. Raqueño

A note about quotes in UNIX % set a=ls % echo a % echo $a % set b=“$a” % echo $b % set b=‘$a’ % set b=`$a` % echo $b Winter Quarter 2005 Rolando V. Raqueño

A note about shell variables Shell variables can also double up as arrays Using the previous example, % echo $b % echo $b[1] % echo $#b % echo $b[$#b] Winter Quarter 2005 Rolando V. Raqueño

A more complex shell program In pbmplus utilities, rawtopgm conversion exists pgmtoraw conversion does not A version of pgmtoraw in a programming language like C Time consuming Program will likely be used infrequently Solution: shell program If you study the pbmplus utilities, you will notice that there is a rawtopgm conversion, but there is no pgmtoraw. Now you can write a version of rawtopgm in a programming language like C, but chances are the time it will take to get a working program will be long. It is possible to write one using shell programming. Winter Quarter 2005 Rolando V. Raqueño

pgmtoraw shell script design Define input and output files Figure out dimensions of input image Determine number of bytes for input image Determine number of bytes for header Need to strip out the header bytes Write out headerless image data Winter Quarter 2005 Rolando V. Raqueño

Define input and output files pgmtoraw #!/bin/csh -f set command_name=$0 set number_args=$#argv if( $number_args != 1 ) then echo “usage:$command_name input_file_name” exit 1 endif . Winter Quarter 2005 Rolando V. Raqueño

Dimensions of input image ( pnmfile) % more test_data.pgm P2 3 3 255 1 2 3 4 5 6 7 8 9 % pnmfile test_data.pgm test_data.pgm: PGM plain, 3 by 3 maxval 255 Winter Quarter 2005 Rolando V. Raqueño

pgmtoraw (continued) set input_file=$1 set pnm_info = `pnmfile $input_file` set image_type = $pnm_info[2] set data_type = $pnm_info[3] set width = $pnm_info[4] set height = $pnm_info[6] set maxval = $pnm_info[8] set pixel_bytes = 1 @ image_bytes = $width * $height Winter Quarter 2005 Rolando V. Raqueño

pgmtoraw (continued) set file_info=`wc -c $input_file` set bytes_in_file = $file_info[1] @ header = $bytes_in_file - $image_bytes dd if=$input_file bs=$pixel_bytes skip=$header Winter Quarter 2005 Rolando V. Raqueño

Resulting pgmtoraw utility Uses existing routines to obtain information pnmfile wc dd Functional tool written in 20 command lines Winter Quarter 2005 Rolando V. Raqueño

Current Limitations of Resulting pgmtoraw utility No check between “ASCII” vs. “RAW” pgm if( data_type == ‘plain,’) ... No provisions for multibyte per pixel case Use pnmfile results to check for above cases endian case needs to be addressed for multibyte case Above conditions can be addressed by suite of UNIX commands Winter Quarter 2005 Rolando V. Raqueño

Shell Scripts Wrappers and IDL Another utility formerly missing in pbmplus jpegtopnm or pnmtojpeg IDL has jpeg read/write capability Create a “wrapper” that makes an idl program pbmplus-like Winter Quarter 2005 Rolando V. Raqueño

pnmtojpeg.pro pro pnmtojpeg, input_file, output_file read_ppm, input_file, image write_jpeg, output_file, image end Winter Quarter 2005 Rolando V. Raqueño

Usage of pnmtojpeg.pro in IDL IDL> pnmtojpeg,‘image.pnm’,’image.jpg’ Winter Quarter 2005 Rolando V. Raqueño

Usage of pnmtojpeg.pro in IDL IDL> pnmtojpeg,‘image.pnm’,’image.jpg’ For IDL to automatically find pnmtojpeg.pro It must be in the current working directory Directory containing pnmtojpeg.pro must be defined in the ENVIRONMENT VARIABLE IDL_PATH Winter Quarter 2005 Rolando V. Raqueño

IDL_PATH environment variable setenv IDL_DIR /cis/common/rsi/idl_5 setenv IDL_PATH \+$IDL_DIR/lib:\+$IDL_DIR/examples: \+/dirs/common/rsi/idl_5:\+/dirs/common/lib/idl:\+~/lib/idl Winter Quarter 2005 Rolando V. Raqueño

pnmtojpeg.csh #!/bin/csh -f echo pnmtojpeg “,” “’”$1”’” “,” “’”$2”’” | idl Winter Quarter 2005 Rolando V. Raqueño

Usage of pnmtojpeg.csh % pnmtojpeg.csh image.pnm image.jpg Winter Quarter 2005 Rolando V. Raqueño

Limitation of pnmtojpeg.csh Does not conform to pbmplus piping, i.e., % tifftopnm file.tif | pnmscale 2.0 > new_file.pnm No error trapping Winter Quarter 2005 Rolando V. Raqueño

Usage cases of pnmtojpeg (no command line arguments) % tifftopnm file.tif | pnmscale 2.0 | pnmtojpeg > new_file.jpg Winter Quarter 2005 Rolando V. Raqueño

Usage cases of pnmtojpeg (1 command line argument) % pnmtojpeg image.pnm > image.jpg Winter Quarter 2005 Rolando V. Raqueño

Usage cases of pnmtojpeg (2 command line arguments) % pnmtojpeg image.pnm image.jpg Winter Quarter 2005 Rolando V. Raqueño

Yet another wrapper pnmtojpeg #!/bin/csh -f # If user interrupts process, jump to stop onintr stop # $0 is the command name # $#argv is the number of arguments # $$ is the process id Winter Quarter 2005 Rolando V. Raqueño

Code for no argument case if($#argv == 0) then set input_file = /usr/tmp/$0_input_$$ set output_file = /usr/tmp/$0_output_$$ cat > $input_file pnmtojpeg.csh $input_file $output_file cat $output_file . Winter Quarter 2005 Rolando V. Raqueño

Code for 1 argument case else if($#argv ==1) then set input_file = $1 set output_file = /usr/tmp/$0_output_$$ pnmtojpeg.csh $input_file $output_file cat $output_file . Winter Quarter 2005 Rolando V. Raqueño

Code for 2 argument case else set input_file = $1 set output_file = $2 pnmtojpeg.csh $input_file $output_file endif #clean up when finished stop: rm -f /usr/tmp/$0_input_$$ rm -f /usr/tmp/$0_output_$$ Winter Quarter 2005 Rolando V. Raqueño

pnmtojpeg summary Produced a “new” pbmplus utility Used UNIX shell scripting Argument handling Scratch space /usr/tmp Process id handling Clean up Integrated IDL program and commands 21 lines of UNIX commands Winter Quarter 2005 Rolando V. Raqueño

Summary The “dot” files Basics of Shell Scripting Special Shell Variables Seamless integration of UNIX to other utilities (IDL) Winter Quarter 2005 Rolando V. Raqueño

Other Shell Constructs to keep in mind foreach while case Winter Quarter 2005 Rolando V. Raqueño