Agenda Regular Expressions (Appendix A in Text) –Definition / Purpose –Commands that Use Regular Expressions –Using Regular Expressions –Using the Replacement String –sed UNIX utility

Regular Expressions Definition: –A “Regular Expression” is a set of one or more strings of characters. –Regular Expressions are used to help search and perform operations of strings that are contained inside files. Many languages including PERL use regular expressions

Regular Expressions Purpose: –Regular expressions can be used with Linux / UNIX commands such as grep, awk, vi and sed –Regular expressions make it easier for user to search, change or process data in files from the issuing a command at the shell prompt or while working within vi editor

Linux (Unix) Utilities that Use Regular Expressions grepSearch for patterns. viSearch and replace patterns within the vi editor in last line mode sedEdit files matching patterns from shell prompt (without using editor) awkSearch for patterns and process matched patterns (discussed later in course)

Simple Regular Expression A Simple Regular Expression is a regular expression that defines a string of characters A regular expression such as a character is used to display all regular expressions in the file that “match”

Complex Regular Expression A Complex Regular Expression uses special characters with letters and numbers in order to provide an ambiguous regular expression match. This is similar to an ambiguous file reference, except we are matching strings contained inside a file, and rules for wildcard searches differ.

Complex Regular Expressions (Wildcard Symbols). matches any character (similar to ?) *matches one or more occurrences of that character ^matches beginning of line (must be at beginning of regular expression) $matches end of line (must follow ending character to match) [ ]similar to wildcard, except if ^ is first character then matches characters not inside brackets

Complex Regular Expressions (Quoting Special Characters) \inhibits meaning of special characters \<matches beginning of a word \>matches end of a word \ matches only if pattern is a full word, delimited by spaces, punctuation, beginning or end of line

Longest Possible Match –A regular expression using special characters always matches the longest possible string starting as far toward the beginning of the line as possible.

Using Regular Expressions with grep Grep uses regular expressions to match patterns of strings contained in files. Format: grep “regular expression” file –Note that grep command places regular expression in double quotes as opposed to using a delimiters such as /

Using Regular Expressions with vi Regular expressions can be used with the vi editor to make it easier to search for complex patterns that are contained in files. The commands to search and replace regular expressions are typed in “last-line mode”

Searching & Replacing Patterns :s/pattern1/pattern2/ - substitutes pattern2 for the first occurrence of pattern1 :1,$ s/pattern1/pattern2/ - substitutes pattern2 for the first occurrence of pattern1 in each line of the file ($ means last line,. means current line) :.,.+10 s/pattern1/pattern2/ - substitutes 11 lines starting with current line :s/pattern1/ *** & ***/ - & is the value of the string matched by pattern1, and is the only character with special meaning in the second pattern

sed Utility Used to modify standard output from a file to be stored into another file. Considered to be a streaming editor without having to “physically edit” file (like vi) Format: sed [options] [script-file] [file-list] options: -f read and make editing changes from script file -n do not print lines from file-list as standard output

Streaming Editor - sed Processing Steps using sed: –sed reads one line from the input file –sed reads first command from script file (or command line or function) and if matched, takes specified action –sed reads next command line from script or function and if matched, takes specified action –Process is repeated until all commands in script file are executed

Streaming Editor - sed Note: –sed command does not make changes to the input file, instead it is used to modify standard output that can be redirected to other files or piped to other UNIX commands. –For examples involving editing techniques with sed command, refer to the samples section of this week’s notes (week 12)

sed - Examples sed ‘/abc/ p’ file –Prints standard output of file along with standard output of pattern abc match (can use “-n” option to display pattern match only) sed ‘5,15 d’ file –delete lines 5 to 15 of file from standard output sed ‘/abc/ d’ file –Does not write-out lines that are matched from standard output.

sed - Examples sed 's/pattern1/pattern2/' file1 –substitute pattern1 with pattern 2 (every occurrence) sed '10 q' file1 –displays first 10 lines then quit sed '/pattern/ q' file1 –displays to first matching line

sed - Examples sed '/pattern/ d' file1 –same as grep -v 'pattern' file1 sed '/pattern1/s/pattern2/pattern3/g' file1 –substitute only on lines matching pattern 1 sed '/pattern1/s/ /pattern3/g' file1 –If pattern2 omitted, it is defaulted to same as pattern 1

sed - Examples sed -n '10,20 s/pattern1/pattern2/ p' file1 –-n suppresses default write to output, p will print matched lines

Additional sed Commands a \ text Append text, which has each embedded newline pre­ceeded by a backslash. i \ text Insert text, which has each embedded newline pre­ceeded by a backslash. q Immediately quit the sed script without processing any more input, except that if auto-print is not diabled the current pattern space will be printed. r filename Append text read from filename.

Additional sed Commands r filename Append text read from filename. w filename Write text to filename.