1. I/O Redirection and Regular Expressions February 9 th, 2004 Class Meeting 4

2. Redirecting stdout Instead of displaying output to the terminal, you can redirect the output to another file using the > operator >> operator is used to append output to an existing file Examples: man ls > ls_help.txt echo $PWD > current_directory cat file1 >> file2

3. Redirecting stdin Instead of reading from a terminal, you can tell a program to read from another file using the < operator Examples: mail user@domain.com < messageuser@domain.com interactive_program < command_list

4. Pipes and Filters Pipe – a way to send the output of one command to the input of another Filter – a program that takes input and transforms it in some way wc – gives a count of words/lines/characters grep – searches a line with a given string (regular expression) more sort – sorts lines alphabetically or numerically

5. Examples of Filtering ls -la | more cat file | wc man ksh | grep “history” ps aux | grep “user1” | wc -l who | sort > current_users

6. What is a Regular Expression? A regular expression (RE) is a pattern of metacharacters that define a set of strings Each of these strings is said to match the regular expression Pattern matching is useful in many real-world situations: searching for a file on the filesystem finding and replacing text in a file extracting data elements from a database

7. Unix programs that use REs grep (search within files) egrep ( grep with extended REs) vi / emacs (text editors) sed (stream editor) awk (pattern scanning language) perl (scripting language)

8. Basic vs. Extended REs In basic regular expressions the metacharacters ?, +, { }, ( ), and | have no special meaning ( grep ) To give them special meaning, use the escaped versions: \?, \+, \{ \}, \( \), and \| When using extended regular expressions, these metacharacters have special meaning grep -E = egrep

9. Using egrep egrep pattern filename(s) To be safe, put quotation marks around your pattern Examples: egrep “abc” textfile egrep -i “abc” textfile egrep -v “abc” textfile egrep -n “abc” textfile

10. Metacharacters Period (.): matches any single character “a.c” matches abc, adc, a&c, a;c, … “u..x” matches unix, uvax, u3(x, … Asterisk (*): matches zero or more occurences of the previous RE not the same as wildcards in the shell “ab*c” matches ac, abc, abbc, abbbc, … “.*” matches any string

11. Metacharacters (cont) Plus (+): matches one or more occurences of the preceding RE “ab+c” matches abc, abbc, abbc, but not ac Question Mark (?): matches zero or one occurences of the preceding RE “ab?c” matches ac or abc, but not abbc Logical Or (|): matches RE before | or RE after | “abc|def” matches abc or def

12. Metacharacters (cont) Caret (^): beginning of line “^D” matches all strings starting with D Dollar Sign ($): end of line “d$” matches all strings ending with d Backslash (\): escapes other metacharacters “file\.txt” matches file.txt, but not fileatxt

13. Metacharacters (cont) Square Brackets ([ ]): indicates a set/range of characters any characters in the set will match ^ before the set negates the set - indicates range Examples: “[fF]un” matches fun or Fun “b[aeiou]g” matches bag, beg, big, bog, bug “[A-Z].*” matches any string beginning with a capital letter “[^abc].*” matches any string not containing an a, b, or c

14. Metacharacters (cont) Parentheses ( ): used to group REs when using other metacharacters “a(bc)*” matches a, abc, abcbc, abcbcbc, … “(foot|base)ball” matches football or baseball Braces ({ }): specify the number of repetitions of an RE “[a-z]{3}” matches three lowercase letter “m.{2,4}” matches strings with m followed by between 2 and 4 characters

15. What do these mean? egrep “^B.*s$” file egrep “ [0-9]{3} “ file egrep “num(ber)? [0-9]+” file egrep “word” file | wc -l egrep “[A-Z].*\?” file What if grep was used instead? Remember, RE matches largest string --color option illustrates the largest match