Rolando V. RaqueñoWednesday, June 10, 2015 Statistics Function Implementation Traditional Programming Approach

Rolando V. RaqueñoWednesday, June 10, 2015 minimum.pro function minimum, x n = n_elements(x) answer = x(0) for i=1L, n-1 do begin if( answer gt x(i) ) then $ answer = x(i) endfor return, answer end

Rolando V. RaqueñoWednesday, June 10, 2015 sum.pro function sum, x n = n_elements( x ) answer = 0 for i = 0, n-1 do begin answer = answer + x(i) endfor return, answer end

Rolando V. RaqueñoWednesday, June 10, 2015 Overflow Problems What if the array is an integer array? Sums will have a maximum limit based on the number of bits specified by the integer type.

Rolando V. RaqueñoWednesday, June 10, 2015 Corrected sum.pro function sum, x n = n_elements( x ) answer = 0.0D for i = 0L, n-1 do begin answer = answer + x(i) endfor return, answer end

Rolando V. RaqueñoWednesday, June 10, 2015 mean.pro function mean, x n = n_elements(x) answer = sum(x) / n return, answer end

Rolando V. RaqueñoWednesday, June 10, 2015 Integer Problems IDL> b=9/5 IDL> print,b 1

Rolando V. RaqueñoWednesday, June 10, 2015 Integer Problems IDL> b=9/double(5) IDL> print,b

Rolando V. RaqueñoWednesday, June 10, 2015 Corrected mean.pro function mean, x n = n_elements(x) answer = sum(x) / double(n) return, answer end

Rolando V. RaqueñoWednesday, June 10, 2015 sum_squares.pro function sum_squares, x squares = x^2.0 answer= sum( squares ) return, answer end

Rolando V. RaqueñoWednesday, June 10, 2015 variance.pro function variance, x n = double(n_elements(x)) answer=(sum_squares(x)-(sum(x)^2.0/n))/ (n-1) return, answer end

Rolando V. RaqueñoWednesday, June 10, 2015 sd.pro function sd, x answer = sqrt(variance(x)) return, answer end

Rolando V. RaqueñoWednesday, June 10, 2015 Shell Programming

Rolando V. RaqueñoWednesday, June 10, 2015 UNIX Tip of the Day Directory maneuvering commands pushd, popd, and, dirs % cd /usr/tmp % pwd /usr/tmp % pushd ~rvrpci % pwd /cis/staff/rvrpci

Rolando V. RaqueñoWednesday, June 10, 2015 UNIX Tip of the Day % dirs /cis/staff/rvrpci /usr/tmp % pushd % pwd /usr/tmp % dirs /usr/tmp /cis/staff/rvrpci % pushd /usr/local/bin

Rolando V. RaqueñoWednesday, June 10, 2015 UNIX Tip of the Day % dirs /usr/local/bin /usr/tmp /cis/staff/rvrpci % pwd /usr/local/bin % pushd % dirs /usr/tmp /usr/local/bin /cis/staff/rvrpci

Rolando V. RaqueñoWednesday, June 10, 2015 UNIX Tip of the Day % dirs /usr/tmp /usr/local/bin /cis/staff/rvrpci % pwd /usr/tmp % pushd +2 % pwd /cis/staff/rvrpci

Rolando V. RaqueñoWednesday, June 10, 2015 UNIX Tip of the Day % dirs /cis/staff/rvrpci /usr/tmp /usr/local/bin % popd % dirs /usr/tmp /usr/local/bin

Rolando V. RaqueñoWednesday, June 10, 2015 IMPORTANT UNIX Concepts Environment and Shell Variables –These allow you to customize your UNIX environment –They are different in terms of their SCOPE SCOPE determines the visibility of a variable

Rolando V. RaqueñoWednesday, June 10, 2015 Other IMPORTANT UNIX Concepts Environment Variable –Examples are TERM and DISPLAY –Set a particular variable to a value by using the setenv command –You can print the value of a particular variable or all the environment variable using the printenv command

Rolando V. RaqueñoWednesday, June 10, 2015 % Environment Variables % Examples –To set environment variables % setenv TERM vt100 % setenv DOG Goofy –print out the terminal type % printenv TERM vt100 –print out all environment variables % printenv

Rolando V. RaqueñoWednesday, June 10, 2015 Shell Variables Shell variables are similar to Environment variables except they have a limited scope, i.e., they exist only in the shell which they are defined. Environment variables on the other hand, exist in all its children shells To illustrate this concept, let us look at the following example

Rolando V. RaqueñoWednesday, June 10, 2015 Environment vs. Shell Variables % set prompt = "Parent Shell > " Parent Shell > setenv DOG Goofy Parent Shell > set mouse=Mickey Parent Shell > printenv DOG Goofy Parent Shell > echo $mouse Mickey Parent Shell > xterm & (YOU SHOULD NOW HAVE A NEW xterm WINDOW) THIS IS KNOWN AS “SPAWNING A NEW (OR CHILD) PROCESS”

Rolando V. RaqueñoWednesday, June 10, 2015 Environment vs. Shell Variables (IN THE NEW xterm WINDOW, DO THE FOLLOWING) % set prompt = "Child Shell > " Child Shell > printenv DOG Goofy Child Shell > echo $mouse mouse: Undefined variable.

Rolando V. RaqueñoWednesday, June 10, 2015 Environment vs. Shell Variables Child Shell > setenv DOG Pluto Child Shell > set mouse=Minnie Child Shell > printenv DOG Pluto Child Shell > echo $mouse Minnie Child Shell > exit (THE xterm WINDOW SHOULD NOW GO AWAY - THIS PROCESS HAS NOW BEEN KILLED)

Rolando V. RaqueñoWednesday, June 10, 2015 Environment vs. Shell Variables Parent Shell > Parent Shell > printenv DOG Goofy Parent Shell > echo $mouse Mickey Parent Shell >

Rolando V. RaqueñoWednesday, June 10, 2015 Environment & Shell Variables Why is this important? –UNIX uses Environment and Shell Variables control a number of processes – Customizes your working environment –Variables used for UNIX Scripts They are typically defined and initialized in your.login and.cshrc files

Rolando V. RaqueñoWednesday, June 10, 2015 Useful Shell Variables filec #Allows file completion path #List of command directories cdpath #List of candidate directories to cd into history #Number of commands to remember

Rolando V. RaqueñoWednesday, June 10, 2015 What is shell programming? Shell programming –automate a set of UNIX commands. –Just like any programming language –“wrappers” black box a customized collection of UNIX commands. –Example of shell programs.login.cshrc

Rolando V. RaqueñoWednesday, June 10, 2015.login file set path=($HOME/bin /usr/local/bin \ /usr/ucb /usr/sbin /bin /usr/bin \ /usr/bin/X11.) stty dec new tset -I -Q set mail=/usr/spool/mail/$USER set editmode = emacs umask 077 biff n date

Rolando V. RaqueñoWednesday, June 10, 2015.cshrc file if ($?prompt) then set notify set history = 100 set savehist = 100 alias pd pushd alias pop popd alias vt100 "set term = vt100" endif

Rolando V. RaqueñoWednesday, June 10, 2015 When these files are executed?.cshrc –is automatically executed when you start a new shell.login –only gets executed once when you first login Can be re-executed by giving the source command % source.cshrc

Rolando V. RaqueñoWednesday, June 10, 2015 Other useful.login and.cshrc entries set filec set cdpath=(~ ~rvrpci/pub ~/mythesis) Other common entries set path=( $path /usr/local/bin) set path=(/usr/local/bin $path)

Rolando V. RaqueñoWednesday, June 10, 2015 User defined shell program Determine name of command Determine input, output, and option arguments Determine UNIX commands to execute Establish error trapping Make shell program executable

Rolando V. RaqueñoWednesday, June 10, 2015 A simple shell program dd command to swap bytes % dd if=input.dat of=output.dat bs=2 conv=swab Very difficult to remember Very little utility to non-UNIX geeks (normal people)

Rolando V. RaqueñoWednesday, June 10, 2015 We would rather see... % swap_bytes input.dat output.dat

Rolando V. RaqueñoWednesday, June 10, 2015 Special Shell Variables Set % swap_bytes input.dat output.dat $0$1$2 $argv[1]$argv[2] command

Rolando V. RaqueñoWednesday, June 10, 2015 Another Special Shell Variables % swap_bytes input.dat output.dat $#argv Indicates how many arguments are present In this case, 2

Rolando V. RaqueñoWednesday, June 10, 2015 shell program swap_bytes #!/bin/csh -f dd if=$1 of=$2 bs=2 conv=swab

Rolando V. RaqueñoWednesday, June 10, 2015 Making swap_bytes shell script executable % ls -l swap_bytes -rw swap_bytes % chmod u+x swap_bytes % ls -l swap_bytes -rwx swap_bytes

Rolando V. RaqueñoWednesday, June 10, 2015 To run swap_bytes swap_bytes becomes just another unix command! % swap_bytes input.dat output.dat

Rolando V. RaqueñoWednesday, June 10, 2015 Limitation of swap_bytes No error trapping Should give usage when typing command % swap_bytes usage: swap_bytes input_file output_file

Rolando V. RaqueñoWednesday, June 10, 2015 Improvement to swap_bytes #!/bin/csh -f if ( $#argv != 2 ) then echo "usage: $0 input_file output_file" exit 1 endif dd if=$1 of=$2 bs=2 conv=swab

Rolando V. RaqueñoWednesday, June 10, 2015 Commad exit status By convention exit 0 Indicates successful command completion exit 1 (or non-zero) Indicates some error condition

Rolando V. RaqueñoWednesday, June 10, 2015 Informational message from swap_bytes UNIX style informational message % swap_bytes usage: swap_bytes input_file output_file

Rolando V. RaqueñoWednesday, June 10, 2015 Interactive swap_bytes If you want a “friendlier” shell program –Have it query the user for the inputs Another special shell variable can be used $<

Rolando V. RaqueñoWednesday, June 10, 2015 Interactive swap_bytes #!/bin/csh -f if ( $#argv != 2 ) then echo -n "Please enter the input file> " set input=$< echo -n "Please enter the output file> " set output=$< endif dd if=$input of=$output bs=2 conv=swab

Rolando V. RaqueñoWednesday, June 10, 2015 Interactive swap_bytes example User simply types the command % swap_bytes Please enter the input file> input.dat Please enter the output file> output.dat

Rolando V. RaqueñoWednesday, June 10, 2015 UNIX Quotes

Rolando V. RaqueñoWednesday, June 10, 2015 A note about quotes in UNIX % set a=ls % echo a % echo $a % set b=“$a” % echo $b % set b=‘$a’ % echo $b % set b=`$a` % echo $b

Rolando V. RaqueñoWednesday, June 10, 2015 A note about shell variables Shell variables can also double up as arrays Using the previous example, % echo $b % echo $b[1] % echo $#b % echo $b[$#b]

Rolando V. RaqueñoWednesday, June 10, 2015 A more complex shell program In pbmplus utilities, rawtopgm conversion exists pgmtoraw conversion does not A version of pgmtoraw in a programming language like C –Time consuming –Program will likely be used infrequently Solution: shell program

Rolando V. RaqueñoWednesday, June 10, 2015 pgmtoraw shell script design Define input and output files Figure out dimensions of input image Determine number of bytes for input image Determine number of bytes for header Need to strip out the header bytes Write out headerless image data

Rolando V. RaqueñoWednesday, June 10, 2015 Define input and output files pgmtoraw #!/bin/csh -f set command_name=$0 set number_args=$#argv if( $number_args != 1 ) then echo “usage:$command_name input_file_name” exit 1 endif...

Rolando V. RaqueñoWednesday, June 10, 2015 Dimensions of input image ( pnmfile ) % more test_data.pgm P % pnmfile test_data.pgm test_data.pgm: PGM plain, 3 by 3 maxval 255

Rolando V. RaqueñoWednesday, June 10, 2015 pgmtoraw (continued) set input_file=$1 set pnm_info = `pnmfile $input_file` set image_type = $pnm_info[2] set data_type = $pnm_info[3] set width = $pnm_info[4] set height = $pnm_info[6] set maxval = $pnm_info[8] set pixel_bytes = image_bytes = $width * $height

Rolando V. RaqueñoWednesday, June 10, 2015 pgmtoraw (continued) set file_info=`wc -c $input_file` set bytes_in_file = header = $bytes_in_file - $image_bytes dd if=$input_file bs=$pixel_bytes skip=$header

Rolando V. RaqueñoWednesday, June 10, 2015 Resulting pgmtoraw utility Uses existing routines to obtain information pnmfile wc dd Functional tool written in 20 command lines

Rolando V. RaqueñoWednesday, June 10, 2015 Current Limitations of Resulting pgmtoraw utility No check between “ASCII” vs. “RAW” pgm if( data_type == ‘plain,’)... No provisions for multibyte per pixel case –Use pnmfile results to check for above cases –endian case needs to be addressed for multibyte case Above conditions can be addressed by suite of UNIX commands

Rolando V. RaqueñoWednesday, June 10, 2015 Shell Scripts Wrappers and IDL Another utility formerly missing in pbmplus jpegtopnm or pnmtojpeg IDL has jpeg read/write capability –Create a “wrapper” that makes an idl program pbmplus -like

Rolando V. RaqueñoWednesday, June 10, 2015 pnmtojpeg.pro pro pnmtojpeg, input_file, output_file read_ppm, input_file, image write_jpeg, output_file, image end

Rolando V. RaqueñoWednesday, June 10, 2015 Usage of pnmtojpeg.pro in IDL IDL> pnmtojpeg,‘image.pnm’,’image.jpg’

Rolando V. RaqueñoWednesday, June 10, 2015 Usage of pnmtojpeg.pro in IDL IDL> pnmtojpeg,‘image.pnm’,’image.jpg’ For IDL to automatically find pnmtojpeg.pro –It must be in the current working directory –Directory containing pnmtojpeg.pro must be defined in the ENVIRONMENT VARIABLE IDL_PATH

Rolando V. RaqueñoWednesday, June 10, 2015 IDL_PATH environment variable setenv IDL_DIR /cis/common/rsi/idl_5 setenv IDL_PATH \+$IDL_DIR/lib:\+$IDL_DIR/examples: \+/dirs/common/rsi/idl_5:\+/dirs/common/lib/id l:\+~/lib/idl

Rolando V. RaqueñoWednesday, June 10, 2015 pnmtojpeg.csh #!/bin/csh -f echo pnmtojpeg “,” “’”$1”’” “,” “’”$2”’” | idl

Rolando V. RaqueñoWednesday, June 10, 2015 Usage of pnmtojpeg.csh % pnmtojpeg.csh image.pnm image.jpg

Rolando V. RaqueñoWednesday, June 10, 2015 Limitation of pnmtojpeg.csh Does not conform to pbmplus piping, i.e., % tifftopnm file.tif | pnmscale 2.0 > new_file.pnm No error trapping

Rolando V. RaqueñoWednesday, June 10, 2015 Usage cases of pnmtojpeg (no command line arguments) % tifftopnm file.tif | pnmscale 2.0 | pnmtojpeg > new_file.jpg

Rolando V. RaqueñoWednesday, June 10, 2015 Usage cases of pnmtojpeg (1 command line argument) % pnmtojpeg image.pnm > image.jpg

Rolando V. RaqueñoWednesday, June 10, 2015 Usage cases of pnmtojpeg (2 command line arguments) % pnmtojpeg image.pnm image.jpg

Rolando V. RaqueñoWednesday, June 10, 2015 Yet another wrapper pnmtojpeg #!/bin/csh -f # If user interrupts process, jump to stop onintr stop # $0 is the command name # $#argv is the number of arguments # $$ is the process id

Rolando V. RaqueñoWednesday, June 10, 2015 Code for no argument case if($#argv == 0) then set input_file = /usr/tmp/$0_input_$$ set output_file = /usr/tmp/$0_output_$$ cat > $input_file pnmtojpeg.csh $input_file $output_file cat $output_file.

Rolando V. RaqueñoWednesday, June 10, 2015 Code for 1 argument case else if($#argv ==1) then set input_file = $1 set output_file = /usr/tmp/$0_output_$$ pnmtojpeg.csh $input_file $output_file cat $output_file.

Rolando V. RaqueñoWednesday, June 10, 2015 Code for 2 argument case else set input_file = $1 set output_file = $2 pnmtojpeg.csh $input_file $output_file endif #clean up when finished stop: rm -f /usr/tmp/$0_input_$$ rm -f /usr/tmp/$0_output_$$

Rolando V. RaqueñoWednesday, June 10, 2015 pnmtojpeg summary Produced a “new” pbmplus utility Used UNIX shell scripting –Argument handling –Scratch space /usr/tmp –Process id handling –Clean up Integrated IDL program and commands 21 lines of UNIX commands

Rolando V. RaqueñoWednesday, June 10, 2015 Summary The “dot” files Basics of Shell Scripting Special Shell Variables Seamless integration of UNIX to other utilities (IDL)

Rolando V. RaqueñoWednesday, June 10, 2015 Other Shell Constructs to keep in mind foreach while case