1 P51UST: Unix and Software Tools Unix and Software Tools (P51UST) Awk Programming (2) Ruibin Bai (Room AB326) Division of Computer Science The University of Nottingham Ningbo, China

This Lecture Awk commands Loops and conditionals Arrays Functions 2 P51UST: Unix and Software Tools

Awk Commands Types of commands –Assignments of variables or arrays –Input/output –String operations –Control-flow commands –User-defined functions (commands) 3 P51UST: Unix and Software Tools

Assignment User defined awk variables are initialised to either zero or the empty string, depending on how they are used. Assign variables with an =, E.g. –FS = “:” –var = count+2 –var = max-min Assignment syntax is less strict –Can have space before and/or after equal to sign ‘=’ 4 P51UST: Unix and Software Tools

Input/output (1) print function –print [argument] [destination] –Used to print one or more variables, fields, or strings to standard out. –If arguments separated by space, output will be concatenated, if separated by commas, output will be separated by OFS (output field separator). –Strings are enclosed by qutation marks. –The output can be redirected to files Print $1, $2, $4 > “myfile”

Input/output (2) Formatted printing: printf Syntax printf ([format [, values]]) Very similar syntax to printf in sh. printf ("*%-10s*%-5d*%+5d\n","hello",10,10) Output: *hello *10 * +10

Input/output (3) getline - Read a string from keyboard or from a file getline [variable string] [<input file] or command | getline [variable string] Example: target file is not specified { printf(“Please enter two values > ”) getline printf(“$1 = %s\t$2 = %s\n”, $1, $2) }

getline Reading from keyboard getline var-name <“-” or getline var-name <“/dev/stdin” or getline var-name <“/dev/tty” Example BEGIN{ printf (“Please enter the name I should search for: > “) getline name < “/dev/tty” } $1 == name || $2==name || $3 == name { printf("$1 = %s\t$2 = %s\t$3 = %s\n", $1, $2, $3) }

A Note About getline getline is a function and does return a value, BUT if you put brackets after it, e.g.: –getline() –You will get an error! Examples –getline newValue < “myFile” BEGIN {printf “Enter a name:>“ getline < “-” print } 9 P51UST: Unix and Software Tools In this example, the user is prompted to enter their name. This is assigned to $0 and the print statement outputs the value of $0 by default Here, the input record is assigned to the variable “newValue”

Control-Flow Commands (1) Conditionals if (condition) { statement1 statement2 … } else { statement3 statement4 … } Conditional operators <less than <=less than or equal to ==equal to > greater than >= greater than or equal != not equal to ~ /re/contains the regular expression re.

Control-Flow Commands (2) For loops for (x= start; x<=maximum; x++){ command(s) } Or for (element in array) { command(s) }

for loop example BEGIN{ for (x=1; x<=10; x++) print x }

Control-flow Commands (3) While loops BEGIN { while (name==“”){ printf(“Give me a name please >”) getline name <“/dev/stdin” } $1==name || $2==name { printf(“here are the data you requested: \n\n”) printf(“\t%s\n\n”,$0) }

Control-flow Commands (4) break –Used to exit from a loop continue –Skip the current body of a loop to the next loop next –The next statement forces awk to immediately stop processing the current record and go on to the next record. nextfile –Skip the remainder of an input file and go on to the next input file

Arrays in Awk awk has arrays with elements subscripted with numbers or strings (associative arrays) Assign arrays in one of two ways: –Name them in an assignment statement myArray[i]=n++ –Use the split() function (to be discussed shortly) n=split(input, words, " ")

Array in Awk (2) Under awk, it's customary to start array indices at 1, rather than 0. myarray[1]="jim“ myarray[2]=456 Array elements can be subscripted with number or string. BEGIN { my_array[1] = "pear" my_array[2] = "tree"; my_array["David"] = "Cassidy"; } 16 P51UST: Unix and Software Tools

Reading Elements in an Array Using a for loop: Using the operator in: …use this to see if an element exists. It does so by testing to see if its index exists (nawk) for (item in array) print array[item] if (index in array) …

An Array Example BEGIN { my_array[1] = "Partridge" my_array[2] = "pear" my_array[3] = "tree" my_array[13] = "Cassidy" print "Print array element using item-in-array for loop:" for (i in my_array) print i "=" my_array[i] print "\nPrint array element using c-style for loop:" min=1; max=13 for (i=min; i<= max; i++) { if (i in my_array) print i "=" my_array[i] } 18 P51UST: Unix and Software Tools

An Array Example BEGIN { my_array[1] = "Partridge" my_array[2] = "pear" my_array[3] = "tree" my_array[13] = "Cassidy" print "Print array element using item-in-array for loop:" for (i in my_array) print i "=" my_array[i] print "\nPrint array element using c-style for loop:" min=1; max=13 for (i=min; i<= max; i++) { if (i in my_array) print i "=" my_array[i] } 19 P51UST: Unix and Software Tools Test whether a value has ever stored for a index value A value can be stored at any index Elements are not printed in order here

Copying an Array The awk language does not support assignment of arrays. Thus, to copy an array, you must copy the individual values from one array to the next. BEGIN { arr_len = split( "Mary lamb freezer", my_array ); for (word in my_array) { copy_array[word] = my_array[word] } for (word in copy_array) { print copy_array[word] } 20 P51UST: Unix and Software Tools

Delete an Array Element Syntax Delete array_name[key] Example BEGIN { my_array["purple"] = "Partridge"; my_array["mountain"] = "pear"; my_array["majesties"] = "tree"; my_array["fruited"] = "Cassidy"; mykey = "fruited"; delete my_array["mountain"]; delete my_array[mykey]; for (i in my_array) { print i "=" my_array[i]; } } 21 P51UST: Unix and Software Tools

String Functions (1) length ([argument]) –Return the length of the argument index (string, target) –Return the location or byte posion of the first byte of the target string within the whole string. substr (string, start [, length]) –Return a substring of the whole string, starting at start split (string, array [, separator]) –Splits the string into many words and stores into array.

String Functions (2) Assume the following target file (/etc/passwd) 1.Username 2. password, 3.User ID (UID), 4.Group ID (GID), 5.User ID Info, 6.Home directory, 7.Command/shell

String Functions (3) Print each of users’ login and first name using index and substr functions BEGIN{ print "Here are the user ID\'s and first names from /etc/passwd" FS=":" } { blank = index($5," ") first = substr($5, 1, blank-1) printf("User ID = %-15s \t first name = %-25s\n", $1, first) }

String Functions (4) Using function split to print each of users’ login, first name and last name. BEGIN{ FS=":" } { howmany= split($5, names, " ") printf("User ID = %-15s firstname = %-15s lastname = %-15s\n", $1, names[1],names[howmany]) }

The system() Function The system() function allows a programmer to execute a command whilst within an awk script. system(“cmd”) The awk script waits for the command to finish before continuing execution. The output of the command is NOT available for processing from within awk. The system() function returns an exit status which can be tested by the awk script.

An Example Using system() BEGIN { if (system(“mkdir UST”) == 0) { if (system(“cd UST”) !=0) print “change directory – failed” } else print “make directory - failed” } 27 P51UST: Unix and Software Tools This example tries to create a new directory called UST. If successful, the code tries to change directory to UST. If not, an error is printed.

An Example Using system() 28 P51UST: Unix and Software Tools $ awk -f create.awk $ ls UST $ awk -f create.awk mkdir: UST: File exists make directory - failed Here, the script (called create.awk) is run and is successful. “ls UST” doesn’t return anything because UST is empty. Here, the script is run for a second time and so the mkdir command fails because UST already exists. The first error is given by the mkdir command, the second error is given by the awk script

User-Defined Functions You can define your own functions in awk, in much the same way as you define a function in C or Java –Thus code that is to be repeated can be grouped together inside a function –Allows code reuse! –NOTE: when calling a function you have defined yourself, no space is allowed between the function name and the opening bracket.

An Example using a Function and an Array # capitalise the first letter of each word in a string function capitalise(input) { result= "" n=split(input, words, " ") for (i=1; i <=n; i++) { w = words[i] w = toupper(substr(w, 1, 1)) substr(w, 2) if (i > 1) result = result " " result = result w } return result } # this is the main program { print capitalise($0) } 30 P51UST: Unix and Software Tools

Break-down of Example # capitalise the first letter of each word in a string function capitalise(input) { … Variable to be used in function - input contains whatever the caller called the function with

Break-down of Example (2) … result= "" n=split(input, words, " ") … Take the input and split it up into the array “words” - divide the input wherever there is a space n is the result returned by the split command and contains the number of elements in the array “words” # Set result to be an empty string

Break-down of Example (3) … for (i=1; i <=n; i++) { w = words[i] w = toupper(substr(w, 1, 1)) substr(w, 2) if (i > 1) result = result " " result = result w } return result } … Assign element to w For each element of array from 1 to the number of elements… Tag a space on to the end of the result string Tag the next word on to the end of the result string Take the substring which starts at the first character and has a length of 1 and capitalise using toupper() Take remainder of string starting at 2nd character and append it to capitalised character

Break-down of Example (4) … # this is the main program { print capitalise($0) } This is a comment in awk Call the capitalise function with the entire input record. Print the result.

Output from Example Given the input file: …our Capitalise function will output: In theory there is no difference between theory and practice, but in practice there is In Theory There Is No Difference Between Theory And Practice, But In Practice There Is