1. I/O Redirection & Regular Expressions CS 2204 Class meeting 4 *Notes by Doug Bowman and other members of the CS faculty at Virginia Tech. Copyright 2001-2003.

2. (C) Doug Bowman, Virginia Tech, 20012 Redirecting stdout Instead of writing to the terminal, you can tell a program to print its output to another file using the > operator >> operator is used to append to a file Examples: man ls > ls_help.txt Echo $PWD > current_directory cat file1 >> file2

3. (C) Doug Bowman, Virginia Tech, 20013 Redirecting stdin Instead of reading from the terminal, you can tell a program to read from another file using the < operator Examples: Mail user@domain.com < message interactive_program < command_list

4. (C) Doug Bowman, Virginia Tech, 20014 Pipes and filters Pipe: a way to send the output of one command to the input of another Filter: a program that takes input and transforms it in some way wc - gives a count of words/lines/chars grep - searches for lines with a given string more sort - sorts lines alphabetically or numerically

5. (C) Doug Bowman, Virginia Tech, 20015 Examples of filtering ls -la | more cat file | wc man ksh | grep “history” ls -l | grep “gifford” | wc who | sort > current_users

6. (C) Doug Bowman, Virginia Tech, 20016 What is a regular expression (RE)? A pattern Defines a set of strings of characters Any string in the set is said to be “matched” by the RE (the RE matches the string)

7. (C) Doug Bowman, Virginia Tech, 20017 Why REs? Pattern matching is a useful tool in many real- world situations: Search for a file on a filesystem Find and replace strings in a file Extract particular data elements from a database REs are an important part of formal languages - one of the basic CS theory disciplines

8. (C) Doug Bowman, Virginia Tech, 20018 UNIX programs that use REs grep (search within files) egrep ( grep but with extended RE’s) vi/emacs (text editors) ex (line editor) sed (stream editor) awk (pattern scanning language) perl (scripting language)

9. (C) Doug Bowman, Virginia Tech, 20019 Characters vs. metacharacters In patterns, characters can be any character except a newline Metacharacters are special characters that have a special meaning To use metacharacters as regular characters in a pattern, quote them with the ‘\’ character

10. (C) Doug Bowman, Virginia Tech, 200110 Basic vs. Extended RE’s In basic regular expressions the metacharacters ?, +, {, |, (, and ) have no special meaning ( grep). To make them special, instead use the backslashed versions \?, \+, \{, \|, \(, and \) For extended regular expressions these are interpreted as special Grep –E  egrep

11. (C) Doug Bowman, Virginia Tech, 200111 Using egrep egrep pattern filename(s) To be safe, put quotes around your pattern Examples: egrep “abc” file.txt (print lines containing “abc”) egrep -i “abc” file.txt (same, but ignore case) egrep -v “abc” file.txt (print lines not containing “abc”) egrep -n “abc” file.txt (include line numbers)

12. (C) Doug Bowman, Virginia Tech, 200112 Metacharacters 1 Period (.): matches any single character “a.c” matches abc, adc, a&c, a;c, … “u..x” matches unix, uvax, u3(x, … Asterisk (*) matches zero or more occurrences of the previous RE Not the same as wildcards in the shell! “ab*c” matches ac, abc, abbc, abbbc, … “.*” matches any string

13. (C) Doug Bowman, Virginia Tech, 200113 Metacharacters 2 Plus (+): matches one or more occurrences of the preceding RE “ab+c” matches abc, abbc, but not ac Question mark (?): matches zero or one occurrences of the preceding RE “ab?c” matches ac, abc but not abbc Logical or (|): matches RE before or RE after bar “abc|def” matches abc or def

14. (C) Doug Bowman, Virginia Tech, 200114 Metacharacters 3 Caret (^): means beginning of line “^D” matches all strings starting with D Dollar sign ($) means end of line “d$” matches all strings ending with d Backslash (\): used to quote other metacharacters “file\.txt” matches file.txt but not fileatxt

15. (C) Doug Bowman, Virginia Tech, 200115 Metacharacters 4 Square brackets ([ ]) indicate a set/range of characters Any characters in the set will match ^ before the set means “not” - between characters indicates a range Examples: “[fF]un” matches fun, Fun “b[aeiou]g” matches bag, beg, big, bog, bug “[A-Z].*” matches any string beginning with a captial letter “[^abc].*” matches any string not starting with a, b, or c

16. (C) Doug Bowman, Virginia Tech, 200116 Metacharacters 5 Parentheses ( ): used to group REs when using other metacharacters “a(bc)*” matches a, abc, abcbc, abcbcbc, … “(foot|base)ball” matches football, baseball Braces ({ }): specify the number of repetitions of an RE “[a-z]{3}” matches three lowercase letters “m.{2,4}” matches strings starting with m between three and five letters

17. (C) Doug Bowman, Virginia Tech, 200117 What do these mean? egrep “^B.*s$” file egrep “ [0-9]{3} ” file egrep “num(ber)? [0-9]+” file egrep “word” file | wc -l egrep “[A-Z].*\?” file What about if they were grep?

18. (C) Doug Bowman, Virginia Tech, 200118 Practice Construct egrep commands that find in file : Lines beginning with a word of at least 10 characters Lines containing a student ID number in standard 3- part form Number of lines with 2 consecutive capitalized words Number of lines not ending in an alphabetic character Lines containing a word beginning with a vowel at the end of a sentence