1. Introduction to Unix – CS 21 Lecture 12

2. Lecture Overview A few more bash programming tricks The here document Trapping signals in bash cut and tr sed awk

3. The Here Document Redirects stdin to a specific set of text located inside the same file << COMMAND << MARKER Data MARKER

4. Example of Here

5. Why Is This Useful? Allows you to keep all relevant information in one file Example: Database you want to search Don’t need to clutter up your directory with unnecessary temporary files

6. Trapping Signals Catching a signal is also called trapping a signal You can tell bash programs what to do when they receive different signals Analogy: When a postcard arrives, what do I do?

7. The ‘trap’ Command Usage: trap ‘COMMAND’ Signals Example: trap ‘cat errorMsg’ 4 6 In order to prevent you from running a program forever, signal number 9 cannot be trapped

8. Example Of trap

9. Two Helper Filters cut Break individual lines apart tr Change characters into different characters

10. The ‘cut’ Command More precise control over individual lines Will cut out certain words from each individual line so they can be processed Usage: cut [FLAGS] FILE

11. Flags -d Delimiter -f Field number Example cut –d: -f3 myFile

12. Example Of Cut

13. The ‘tr’ Command Translate Change on a one to one basis characters from one thing to another Usage: tr ‘Set1’ ‘Set2’ Example: tr ‘abc’ ‘ABC’ < myFile

14. Example Of tr

15. Two More Powerful Tools sed Stream Editor awk Alfred Aho, Peter Weinberger, and Brian Kernighan

16. The ‘sed’ Command/Language Filter Like grep, sort, or uniq, it takes input and performs some operation on it to filter the output Usage: sed ‘Address Command’ Address specifies where to make changes Command specifies what change to make Example: sed ‘4d’ textFile

17. Address Specification Addresses could be line numbers or regular expressions No address – each line One address – only that line Two comma separated addresses – All lines in between ! – All other lines

18. Commands Available To sed a\ Append text c\ Replace text i\ Insert text before d Delete lines s Make substitutions

19. Examples

20. More Examples

21. Substitution Example Same syntax as vi

22. When Would You Want To Use sed? sed works on streams, so it is perfect to be placed in the middle of a pipe in order to change the output from one format to another Example: If a program always prints out 4 lines of junk for every good line, sed can be used to weed out the junk

23. Example

24. awk Answers the question: What do I do if I want to search for a pattern and actually use it? Combination of grep and commands Searches for some pattern or condition and performs some command on it Complete programming language Looks a lot like C syntactically Variables are declared bash style

25. Pattern And Command awk in its most basic form simply executes a command on all lines that match (or adhere to) a certain pattern Usage: awk ‘Pattern { Command }’ FILE Just like sed, if there is no pattern, then every line will be matched

26. Example

27. Different Ways To Run Awk awk ‘Pattern { Command }’ awk –f awkFile inputFile Since awk itself can be a complex language, you can store all the commands in a file and run it with the –f flag

28. Important awk Concepts Record Every line of an input file is a record The current record can be referenced with $0 Field Every word in a record is called a field Each field is numbered and can be referred to $1 is the first record, $2 is the second, etc.

29. Special Predefined awk Variables RS The character that acts as a record separator Default is the end of a line FS The character that acts as a field separator Default is whitespace (space, tab, etc) Can be redefined with the –F flag

30. Example

31. Other awk Variables NF = number of fields in the current record NR = Total number of records seen so far OFS = Output field separator ORS = Output record separator

32. BEGIN And END Blocks Two special patterns that can be matched BEGIN Commands are executed before any records are looked at END Commands are executed after all records are processed

33. Example

34. Awk Patterns /regular expression/ -> same as egrep Relational expression >, =, <=, == Pattern && pattern Pattern || pattern Pattern1 ? Pattern2 : pattern3 If Pattern1 is True, then Pattern2, else pattern 3 (pattern) ! Pattern Pattern1, pattern2

35. Example Patterns

36. Awk Actions Enclosed in { } () Grouping $ Field reference ++ -- Increment, decrement ^ Exponentiation + - ! Plus, minus, not * / % Multiplication, division, and modulus

37. Control Flow Statements Inside of commands, you can have control flow if while for

38. If Syntax if (condition) { Statements } else { Statements }

39. While Syntax while (Condition) { Statements }

40. For Syntax for (Declaration ; Condition ; Increment ) { Statements } for ( j=0; j < 5; j++) { print “hello world” }

41. When Would You Want To Use awk? Whenever you want to search for some pattern and perform some action Example: I want to go through and calculate the average score on the Midterm

42. Example

43. Another Example Adding 12 points to everyone’s midterm score

44. Putting Them Together – awk and sed

45. awk Versus bash $ arguments Always enclose everything to awk in single quotes so they don’t get interpreted $1 to awk means something completely different than $1 to bash $1 in awk means first field $1 in bash means first command line argument

46. Next Time Looking at some of the string and mathematical awk functionality find Putting everything together The complete bash programming necessities Quiz 2 – Next Tuesday