Introduction to Advanced UNIX March Kevin Keay

Outline Overview Overview Resources Resources The basic form of a typical C-shell script The basic form of a typical C-shell script Introduction to C-Shell Programming Introduction to C-Shell Programming Introduction to C-Shell Programming Introduction to C-Shell Programming Advanced UNIX materials: The following are PDF documents that may be used for reference but won’t be discussed in detail during this short course: Advanced UNIX materials: The following are PDF documents that may be used for reference but won’t be discussed in detail during this short course: Part 1 : Scripts Scripts Part 2 : A summary of useful software A summary of useful softwareA summary of useful software Part 3 : Reanalysis products Reanalysis productsReanalysis products Lab session (informal) 2-3 PM UNIX Lab Lab session (informal) 2-3 PM UNIX Lab See: See:

Overview This is a practical course that is designed to give you the basic skills to write UNIX C-shell scripts This is a practical course that is designed to give you the basic skills to write UNIX C-shell scripts You can only truly understand the concepts by putting them into practice! You can only truly understand the concepts by putting them into practice!

Overview (2) We will look at the basic form of a C-shell script We will look at the basic form of a C-shell script There will be a quick tour through Introduction to C-Shell Programming There will be a quick tour through Introduction to C-Shell Programming There are some extra advanced UNIX materials (Parts 1-3) that you may need to reference. Some aspects will be looked at during the lab session but they are intended for personal reference There are some extra advanced UNIX materials (Parts 1-3) that you may need to reference. Some aspects will be looked at during the lab session but they are intended for personal reference Part 1 covers C-shell scripts and some additional concepts Part 1 covers C-shell scripts and some additional concepts Part 2 is a summary of some useful software Part 2 is a summary of some useful software Part 3 focuses on downloading and decoding reanalysis data (NetCDF and GRIB) Part 3 focuses on downloading and decoding reanalysis data (NetCDF and GRIB) Finally, there is a short Lab session Finally, there is a short Lab session

Resources A very useful and concise reference book covering UNIX and the C shell is: A very useful and concise reference book covering UNIX and the C shell is: UNIX in a Nutshell, O’Reilly and Associates (Engineering Library: GILL). A useful online resource from the University of Surrey, UNIX Tutorial for Beginners, is available at: A useful online resource from the University of Surrey, UNIX Tutorial for Beginners, is available at: For PDF files of the handouts see: For PDF files of the handouts see: All UNIX commands should have a manual (man) page on the machine that you are using e.g. man awk. However sometimes the pages are not installed or are in an unexpected location. It may be more convenient to use a search engine like Google to find information on a command. All UNIX commands should have a manual (man) page on the machine that you are using e.g. man awk. However sometimes the pages are not installed or are in an unexpected location. It may be more convenient to use a search engine like Google to find information on a command.

The basic form of a typical C-shell script The first line of a C-shell script (text file) is: The first line of a C-shell script (text file) is: #!/bin/csh –f The –f option tells the script not to source (read) the user’s.cshrc file; this is faster and also makes the script more portable. All other lines starting with # are comments All other lines starting with # are comments Commands may be continued onto subsequent lines with \ Commands may be continued onto subsequent lines with \ Multiple commands can be placed on a single line with ; Multiple commands can be placed on a single line with ; Spaces around operators and commands are required but there are no other restrictions Spaces around operators and commands are required but there are no other restrictions

The basic form of a typical C-shell script (2) For neatness, end the script with exit (not essential) For neatness, end the script with exit (not essential) Shell variables start with a $ (this is only omitted with using set - see below) Shell variables start with a $ (this is only omitted with using set - see below) The shell variable $#argv contains the number of arguments (items) typed on the command line. The items are referred to as shell variables $1, $2, $3, …, $#argv The shell variable $#argv contains the number of arguments (items) typed on the command line. The items are referred to as shell variables $1, $2, $3, …, $#argv

The basic form of a typical C-shell script (3) For example: For example: myscript jja yes would give: $#argv = 3 $1 = jja $2 = $3= yes

The basic form of a typical C-shell script (4) Shell variables are either string (text) or numeric Shell variables are either string (text) or numeric A string variable is set (defined) by: A string variable is set (defined) by: set var = value e.g. set x = Fred Note: Variables and values are case-sensitive i.e. x is not the same as X Variable names can be several characters e.g. set Case2 = 3 Variable names can be several characters e.g. set Case2 = 3 To print (display) the value of a variable: To print (display) the value of a variable: echo $var e.g. echo $x Fred

The basic form of a typical C-shell script (5) A numeric variable is set (defined) by: A numeric variable is set (defined) var = integer k = -7 Simple calculations may be performed Simple calculations may be performed j = ($k + 1)[j would be – = k = ($k + 1) would change k from -7 to -6 Could also use C k j = (2 * $k - 5) [k= -7 would give j= -19] Note: Put spaces around operators like *, + etc. Floating point operations are not normally used in scripts but can be done with the command bc Floating point operations are not normally used in scripts but can be done with the command bc e.g. echo " " | bc would print 8.4 on the screen set y = `echo " " | bc` would save the answer in variable y

The basic form of a typical C-shell script (6) A shell variable may be given the output of a command (or user program) as in the bc example. In general: A shell variable may be given the output of a command (or user program) as in the bc example. In general: set var = `command` e.g. set ff = `ls *.dat` Remember: ls is used to list files. If the files are 5.dat 12.dat 13.dat then echo $ff would display: 5.dat 12.dat 13.dat 5.dat 12.dat 13.dat

The basic form of a typical C-shell script (7) To extract the individual elements of variable ff we can use $#ff to find the number of items (similar to $#argv) To extract the individual elements of variable ff we can use $#ff to find the number of items (similar to $#argv) e.g. echo $#ff would display 3 echo $ff[1] would display 5.dat echo $ff[2] would display 12.dat echo $ff[3] would display 13.dat

The basic form of a typical C-shell script (8) A while loop is used to set up an iterative or repetitive procedure: A while loop is used to set up an iterative or repetitive procedure: while (condition is true) commands end end k = 1 # Initialise k while ($k <= 3) # Process loop while k <= 3 echo $k # Display k echo $k # Display k = ($k + 1) # Increment k by k = ($k + 1) # Increment k by 1end

The basic form of a typical C-shell script (9) A foreach loop is used to set up an iterative or repetitive procedure involving files: A foreach loop is used to set up an iterative or repetitive procedure involving files: foreach var (files) commands end end e.g.foreach f ([A-Z]*) # f is a file starting with a capital letter echo $f # Print filename echo $f # Print filename mv $f $f.cap # Rename file to have extension.cap mv $f $f.cap # Rename file to have extension.capend

The basic form of a typical C-shell script (10) An if-else-endif structure is used to control script branching: An if-else-endif structure is used to control script branching: if (condition 1 is true) commands else if (condition 2 is true) commandselsecommands endif endif Note: use ‘else if` not elseif Note: use ‘else if` not elseif

The basic form of a typical C-shell script (11) e.g. e.g. if ($x == 6 || $s == sea) then do something else if ($x > 10) then do something else else go here if neither of the above conditions is true endif Simpler or more complex forms are possible Simpler or more complex forms are possible e.g. if ($y == 2) ls –l *.dat

The basic form of a typical C-shell script (12) The while or if (condition) uses C notation for logical operations: The while or if (condition) uses C notation for logical operations: ||or && and == equal >= greater than or equal <= less than or equal != not equal != not equal >greater than <less than

The basic form of a typical C-shell script (13) An example An example #!/bin/csh –f if ($#argv != 1) then echo “Usage: myscript name” exit # If incorrect usage end the script else # If only 1 argument, branch here set n = ($1) # Put n equal to $1 endif echo “Your name is “$n # You could do other things here exit

The basic form of a typical C-shell script (14) Another example Another example #!/bin/csh –f set ff = `ls *.dat` # ff contains filenames nf = $#ff # nf equals the no. of files (items in j = 1 # Set counter j to 1 while ($j <= $nf) # Loop while j <= nf set f = $ff[$j] # Set f to be the jth file set f = $ff[$j] # Set f to be the jth file ls –l $f # List the details of f ls –l $f # List the details of f # We could use the file f as an argument to a command or program myprog $f myprog = ($j + 1) # Increment counter j by = ($j + 1) # Increment counter j by 1 end # Go here when j > nf echo “We are finished!” # A message to the screen exit

Introduction to C-shell programming Refer to the PDF guide: Refer to the PDF guide: /UNIX_Course/Intro_to_C- Shell_Programming_2008.pdf /UNIX_Course/Intro_to_C- Shell_Programming_2008.pdf

Advanced UNIX materials The following are PDF documents that may be used for reference but won’t be discussed in detail during this short course: The following are PDF documents that may be used for reference but won’t be discussed in detail during this short course: Part 1 : Scripts Scripts Part 2 : A summary of useful software A summary of useful softwareA summary of useful software Part 3 : Reanalysis products Reanalysis productsReanalysis products

Lab session Lab session (informal) 2-3 PM UNIX Lab Lab session (informal) 2-3 PM UNIX Lab See: _Course/Advanced_UNIX_Lab_Session_2010.pdf See: _Course/Advanced_UNIX_Lab_Session_2010.pdf _Course/Advanced_UNIX_Lab_Session_2010.pdf _Course/Advanced_UNIX_Lab_Session_2010.pdf