1 Winter Quarter 2003Rolando V. Raqueño Shell Programming

2 Winter Quarter 2003Rolando V. Raqueño Unix Power Tools Reference Chapter 35 –Shell Programming for the Uninitiated Chapter 36 –Shell Programming for the Initiated

3 Winter Quarter 2003Rolando V. Raqueño UNIX Tip of the Day Directory maneuvering commands pushd, popd, and, dirs % cd /usr/tmp % pwd /usr/tmp % pushd ~rvrpci % pwd /cis/staff/rvrpci

4 Winter Quarter 2003Rolando V. Raqueño UNIX Tip of the Day % dirs /cis/staff/rvrpci /usr/tmp % pushd % pwd /usr/tmp % dirs /usr/tmp /cis/staff/rvrpci % pushd /usr/local/bin

5 Winter Quarter 2003Rolando V. Raqueño UNIX Tip of the Day % dirs /usr/local/bin /usr/tmp /cis/staff/rvrpci % pwd /usr/local/bin % pushd % dirs /usr/tmp /usr/local/bin /cis/staff/rvrpci

6 Winter Quarter 2003Rolando V. Raqueño UNIX Tip of the Day % dirs /usr/tmp /usr/local/bin /cis/staff/rvrpci % pwd /usr/tmp % pushd +2 % pwd /cis/staff/rvrpci

7 Winter Quarter 2003Rolando V. Raqueño UNIX Tip of the Day % dirs /cis/staff/rvrpci /usr/tmp /usr/local/bin % popd % dirs /usr/tmp /usr/local/bin

8 Winter Quarter 2003Rolando V. Raqueño IMPORTANT UNIX Concepts Environment and Shell Variables –These allow you to customize your UNIX environment –They are different in terms of their SCOPE SCOPE determines the visibility of a variable

9 Winter Quarter 2003Rolando V. Raqueño Other IMPORTANT UNIX Concepts Environment Variable –Examples are TERM and DISPLAY –Set a particular variable to a value by using the setenv command –You can print the value of a particular variable or all the environment variable using the printenv command

10 Winter Quarter 2003Rolando V. Raqueño % Environment Variables % Examples –To set environment variables % setenv TERM vt100 % setenv DOG Goofy –print out the terminal type % printenv TERM vt100 –print out all environment variables % printenv

11 Winter Quarter 2003Rolando V. Raqueño Shell Variables Shell variables are similar to Environment variables except they have a limited scope, i.e., they exist only in the shell which they are defined. Environment variables on the other hand, exist in all its children shells To illustrate this concept, let us look at the following example

12 Winter Quarter 2003Rolando V. Raqueño Environment vs. Shell Variables % set prompt = "Parent Shell > " Parent Shell > setenv DOG Goofy Parent Shell > set mouse=Mickey Parent Shell > printenv DOG Goofy Parent Shell > echo $mouse Mickey Parent Shell > xterm & (YOU SHOULD NOW HAVE A NEW xterm WINDOW) THIS IS KNOWN AS “SPAWNING A NEW (OR CHILD) PROCESS”

13 Winter Quarter 2003Rolando V. Raqueño Environment vs. Shell Variables (IN THE NEW xterm WINDOW, DO THE FOLLOWING) % set prompt = "Child Shell > " Child Shell > printenv DOG Goofy Child Shell > echo $mouse mouse: Undefined variable.

14 Winter Quarter 2003Rolando V. Raqueño Environment vs. Shell Variables Child Shell > setenv DOG Pluto Child Shell > set mouse=Minnie Child Shell > printenv DOG Pluto Child Shell > echo $mouse Minnie Child Shell > exit (THE xterm WINDOW SHOULD NOW GO AWAY - THIS PROCESS HAS NOW BEEN KILLED)

15 Winter Quarter 2003Rolando V. Raqueño Environment vs. Shell Variables Parent Shell > Parent Shell > printenv DOG Goofy Parent Shell > echo $mouse Mickey Parent Shell >

16 Winter Quarter 2003Rolando V. Raqueño Environment & Shell Variables Why is this important? –UNIX uses Environment and Shell Variables control a number of processes – Customizes your working environment –Variables used for UNIX Scripts They are typically defined and initialized in your.login and.cshrc files

17 Winter Quarter 2003Rolando V. Raqueño Useful Shell Variables filec #Allows file completion path #List of command directories cdpath #List of candidate directories to cd into history #Number of commands to remember

18 Winter Quarter 2003Rolando V. Raqueño What is shell programming? Shell programming –automate a set of UNIX commands. –Just like any programming language –“wrappers” black box a customized collection of UNIX commands. –Example of shell programs.login.cshrc

19 Winter Quarter 2003Rolando V. Raqueño.login file set path=($HOME/bin /usr/local/bin \ /usr/ucb /usr/sbin /bin /usr/bin \ /usr/bin/X11.) stty dec new tset -I -Q set mail=/usr/spool/mail/$USER set editmode = emacs umask 077 biff n date

20 Winter Quarter 2003Rolando V. Raqueño.cshrc file if ($?prompt) then set notify set history = 100 set savehist = 100 alias pd pushd alias pop popd alias vt100 "set term = vt100" endif

21 Winter Quarter 2003Rolando V. Raqueño When these files are executed?.cshrc –is automatically executed when you start a new shell.login –only gets executed once when you first login Can be re-executed by giving the source command % source.cshrc

22 Winter Quarter 2003Rolando V. Raqueño Other useful.login and.cshrc entries set filec set cdpath=(~ ~rvrpci/pub ~/mythesis) Other common entries set path=( $path /usr/local/bin) set path=(/usr/local/bin $path)

23 Winter Quarter 2003Rolando V. Raqueño User defined shell program Determine name of command Determine input, output, and option arguments Determine UNIX commands to execute Establish error trapping Make shell program executable

24 Winter Quarter 2003Rolando V. Raqueño A simple shell program dd command to swap bytes % dd if=input.dat of=output.dat bs=2 conv=swab Very difficult to remember Very little utility to non-UNIX geeks (normal people)

25 Winter Quarter 2003Rolando V. Raqueño We would rather see... % swap_bytes input.dat output.dat

26 Winter Quarter 2003Rolando V. Raqueño Special Shell Variables Set % swap_bytes input.dat output.dat $0$1$2 $argv[1]$argv[2] command

27 Winter Quarter 2003Rolando V. Raqueño Another Special Shell Variables % swap_bytes input.dat output.dat $#argv Indicates how many arguments are present In this case, 2

28 Winter Quarter 2003Rolando V. Raqueño shell program swap_bytes #!/bin/csh -f dd if=$1 of=$2 bs=2 conv=swab

29 Winter Quarter 2003Rolando V. Raqueño Making swap_bytes shell script executable % ls -l swap_bytes -rw swap_bytes % chmod u+x swap_bytes % ls -l swap_bytes -rwx swap_bytes

30 Winter Quarter 2003Rolando V. Raqueño To run swap_bytes swap_bytes becomes just another unix command! % swap_bytes input.dat output.dat

31 Winter Quarter 2003Rolando V. Raqueño Limitation of swap_bytes No error trapping Should give usage when typing command % swap_bytes usage: swap_bytes input_file output_file

32 Winter Quarter 2003Rolando V. Raqueño Improvement to swap_bytes #!/bin/csh -f if ( $#argv != 2 ) then echo "usage: $0 input_file output_file" exit 1 endif dd if=$1 of=$2 bs=2 conv=swab

33 Winter Quarter 2003Rolando V. Raqueño Commad exit status By convention exit 0 Indicates successful command completion exit 1 (or non-zero) Indicates some error condition

34 Winter Quarter 2003Rolando V. Raqueño Informational message from swap_bytes UNIX style informational message % swap_bytes usage: swap_bytes input_file output_file

35 Winter Quarter 2003Rolando V. Raqueño Interactive swap_bytes If you want a “friendlier” shell program –Have it query the user for the inputs Another special shell variable can be used $<

36 Winter Quarter 2003Rolando V. Raqueño Interactive swap_bytes #!/bin/csh -f if ( $#argv != 2 ) then echo -n "Please enter the input file> " set input=$< echo -n "Please enter the output file> " set output=$< endif dd if=$input of=$output bs=2 conv=swab

37 Winter Quarter 2003Rolando V. Raqueño Interactive swap_bytes example User simply types the command % swap_bytes Please enter the input file> input.dat Please enter the output file> output.dat

38 Winter Quarter 2003Rolando V. Raqueño UNIX Quotes

39 Winter Quarter 2003Rolando V. Raqueño A note about quotes in UNIX % set a=ls % echo a % echo $a % set b=“$a” % echo $b % set b=‘$a’ % echo $b % set b=`$a` % echo $b

40 Winter Quarter 2003Rolando V. Raqueño A note about shell variables Shell variables can also double up as arrays Using the previous example, % echo $b % echo $b[1] % echo $#b % echo $b[$#b]

41 Winter Quarter 2003Rolando V. Raqueño A more complex shell program In pbmplus utilities, rawtopgm conversion exists pgmtoraw conversion does not A version of pgmtoraw in a programming language like C –Time consuming –Program will likely be used infrequently Solution: shell program

42 Winter Quarter 2003Rolando V. Raqueño pgmtoraw shell script design Define input and output files Figure out dimensions of input image Determine number of bytes for input image Determine number of bytes for header Need to strip out the header bytes Write out headerless image data

43 Winter Quarter 2003Rolando V. Raqueño Define input and output files pgmtoraw #!/bin/csh -f set command_name=$0 set number_args=$#argv if( $number_args != 1 ) then echo “usage:$command_name input_file_name” exit 1 endif...

44 Winter Quarter 2003Rolando V. Raqueño Dimensions of input image ( pnmfile ) % more test_data.pgm P % pnmfile test_data.pgm test_data.pgm: PGM plain, 3 by 3 maxval 255

45 Winter Quarter 2003Rolando V. Raqueño pgmtoraw (continued) set input_file=$1 set pnm_info = `pnmfile $input_file` set image_type = $pnm_info[2] set data_type = $pnm_info[3] set width = $pnm_info[4] set height = $pnm_info[6] set maxval = $pnm_info[8] set pixel_bytes = image_bytes = $width * $height

46 Winter Quarter 2003Rolando V. Raqueño pgmtoraw (continued) set file_info=`wc -c $input_file` set bytes_in_file = header = $bytes_in_file - $image_bytes dd if=$input_file bs=$pixel_bytes skip=$header

47 Winter Quarter 2003Rolando V. Raqueño Resulting pgmtoraw utility Uses existing routines to obtain information pnmfile wc dd Functional tool written in 20 command lines

48 Winter Quarter 2003Rolando V. Raqueño Current Limitations of Resulting pgmtoraw utility No check between “ASCII” vs. “RAW” pgm if( data_type == ‘plain,’)... No provisions for multibyte per pixel case –Use pnmfile results to check for above cases –endian case needs to be addressed for multibyte case Above conditions can be addressed by suite of UNIX commands

49 Winter Quarter 2003Rolando V. Raqueño Shell Scripts Wrappers and IDL Another utility formerly missing in pbmplus jpegtopnm or pnmtojpeg IDL has jpeg read/write capability –Create a “wrapper” that makes an idl program pbmplus -like

50 Winter Quarter 2003Rolando V. Raqueño pnmtojpeg.pro pro pnmtojpeg, input_file, output_file read_ppm, input_file, image write_jpeg, output_file, image end

51 Winter Quarter 2003Rolando V. Raqueño Usage of pnmtojpeg.pro in IDL IDL> pnmtojpeg,‘image.pnm’,’image.jpg’

52 Winter Quarter 2003Rolando V. Raqueño Usage of pnmtojpeg.pro in IDL IDL> pnmtojpeg,‘image.pnm’,’image.jpg’ For IDL to automatically find pnmtojpeg.pro –It must be in the current working directory –Directory containing pnmtojpeg.pro must be defined in the ENVIRONMENT VARIABLE IDL_PATH

53 Winter Quarter 2003Rolando V. Raqueño IDL_PATH environment variable setenv IDL_DIR /cis/common/rsi/idl_5 setenv IDL_PATH \+$IDL_DIR/lib:\+$IDL_DIR/examples: \+/dirs/common/rsi/idl_5:\+/dirs/common/lib/id l:\+~/lib/idl

54 Winter Quarter 2003Rolando V. Raqueño pnmtojpeg.csh #!/bin/csh -f echo pnmtojpeg “,” “’”$1”’” “,” “’”$2”’” | idl

55 Winter Quarter 2003Rolando V. Raqueño Usage of pnmtojpeg.csh % pnmtojpeg.csh image.pnm image.jpg

56 Winter Quarter 2003Rolando V. Raqueño Limitation of pnmtojpeg.csh Does not conform to pbmplus piping, i.e., % tifftopnm file.tif | pnmscale 2.0 > new_file.pnm No error trapping

57 Winter Quarter 2003Rolando V. Raqueño Usage cases of pnmtojpeg (no command line arguments) % tifftopnm file.tif | pnmscale 2.0 | pnmtojpeg > new_file.jpg

58 Winter Quarter 2003Rolando V. Raqueño Usage cases of pnmtojpeg (1 command line argument) % pnmtojpeg image.pnm > image.jpg

59 Winter Quarter 2003Rolando V. Raqueño Usage cases of pnmtojpeg (2 command line arguments) % pnmtojpeg image.pnm image.jpg

60 Winter Quarter 2003Rolando V. Raqueño Yet another wrapper pnmtojpeg #!/bin/csh -f # If user interrupts process, jump to stop onintr stop # $0 is the command name # $#argv is the number of arguments # $$ is the process id

61 Winter Quarter 2003Rolando V. Raqueño Code for no argument case if($#argv == 0) then set input_file = /usr/tmp/$0_input_$$ set output_file = /usr/tmp/$0_output_$$ cat > $input_file pnmtojpeg.csh $input_file $output_file cat $output_file.

62 Winter Quarter 2003Rolando V. Raqueño Code for 1 argument case else if($#argv ==1) then set input_file = $1 set output_file = /usr/tmp/$0_output_$$ pnmtojpeg.csh $input_file $output_file cat $output_file.

63 Winter Quarter 2003Rolando V. Raqueño Code for 2 argument case else set input_file = $1 set output_file = $2 pnmtojpeg.csh $input_file $output_file endif #clean up when finished stop: rm -f /usr/tmp/$0_input_$$ rm -f /usr/tmp/$0_output_$$

64 Winter Quarter 2003Rolando V. Raqueño pnmtojpeg summary Produced a “new” pbmplus utility Used UNIX shell scripting –Argument handling –Scratch space /usr/tmp –Process id handling –Clean up Integrated IDL program and commands 21 lines of UNIX commands

65 Winter Quarter 2003Rolando V. Raqueño Summary The “dot” files Basics of Shell Scripting Special Shell Variables Seamless integration of UNIX to other utilities (IDL)

66 Winter Quarter 2003Rolando V. Raqueño Other Shell Constructs to keep in mind foreach while case