Lecture 9  sed

sed  sed is a stream-oriented editor the input (file/std input) flows through the program sed and is directed the standard output the input (file/std input) flows through the program sed and is directed the standard output Used primarily for non interactive operations Used primarily for non interactive operations  sed [-n] –f script_file file or sed [-n] `command` file sed executes the given command or script file that contain commands on each line of the input (file) sed executes the given command or script file that contain commands on each line of the input (file) -n: turn off default printing -n: turn off default printing

sed Commands  p: print line -n prevents lines from being printed twice -n prevents lines from being printed twice  d: delete line  s/old/new/: substitute old with new  s/old/new/g: substitute all occurrences of old with new  !: negates a command  Full list of commands can be found on page 129

sed Examples  sed p file.txt  sed –n p file.txt  sed d file.txt  sed \!d file.txt  p and d seem a bit worthless, don’t they? They purpose will become more clear when we discuss addresses.

sed: Substitution  The strongest feature of sed  Syntax is s/expression/string/flag expression is a regular expression expression is a regular expression string is a string string is a string  sed ‘s/|/:/’ data.txt substitute the character ‘|’ with the character ‘:’ substitute the character ‘|’ with the character ‘:’  sed ‘s/|/:/g’ data.txt

Some Useful Substitution Flags  g: global (replace all matches on the line).  p: print the line if a successful match sed ‘s/old/new/g’ file.txt sed ‘s/old/new/g’ file.txt sed ‘s/old/new/gp’ file.txt sed ‘s/old/new/gp’ file.txt sed –n ‘s/old/new/gp’ file.txt sed –n ‘s/old/new/gp’ file.txt

Regular Expressions for sed  The usual suspects ^, $,., *, [ ], [^ ], \( \), \ ^, $,., *, [ ], [^ ], \( \), \  A new operator &: the string which matches the expression &: the string which matches the expression can be used in the substitution stringcan be used in the substitution string s/hello/**&**/g replaces all occurrences of hello with **hello**s/hello/**&**/g replaces all occurrences of hello with **hello**

sed Addressing  So far, we have been applying sed commands to every line makes p and d not very useful makes p and d not very useful  With addressing, we can apply commands to some, but not all lines  sed can use 0 addresses (all lines) 0 addresses (all lines) 1 address (a single line) 1 address (a single line) 2 addresses (a range of lines) 2 addresses (a range of lines)  Address can be line numbers of context (defined by regular expressions)

Line Number Addressing Examples %sed –n ‘3,4p’ foo.txt Since sed prints each line anyway, if we only want lines 3 & 4 (instead of all lines with lines 3 & 4 duplicated) we use the –n %sed –n ‘$p’ foo.txt For each line, if that line is the last line, print %sed –n ‘3,$p’ foo.txt For each line, if that line is the third through last line, print

Context Addressing Examples  Use patterns/regular expressions rather than explicitly specifying line numbers %sed –n ‘/^From: /p’ $HOME/mbox retrieve all the sender lines from the mailbox file, i.e., for each line, if that line starts with ‘From’, print it. Note that the / / mark the beginning and end of the pattern to match retrieve all the sender lines from the mailbox file, i.e., for each line, if that line starts with ‘From’, print it. Note that the / / mark the beginning and end of the pattern to match %ls –l | sed –n ‘/^.....w/p’ For each line, if the sixth character is a W, print For each line, if the sixth character is a W, print

Context Ranges  sed ‘/hello/,/there/d’ file.txt delete all lines that occur between a line that matches hello and a line that matches there. The hello and there lines are also removed. delete all lines that occur between a line that matches hello and a line that matches there. The hello and there lines are also removed.  Multiple contexts are possible two contexts specified by a single range two contexts specified by a single range

sed Addressing  Using a ! after the address means all lines which do not match the address sed ‘1\!d’ test.txt sed ‘1\!d’ test.txt

Example file northwestNWCharles Main westernWESharon Gray southwestSWLewis Dalsass southernSOSuan Chin southeastSEPatricia Heme easternEATB Savage northeastNEAM Main Jr northNOMargot Webber centralCTAnn Stephens sed ‘/north/p’ filesed –n ‘s/west/north/g’ file sed ‘3,$d’ filesed ‘s/\(Mar\)got/\1ianne/p’ file sed ‘s/west/north/g’ filesed ‘/west/,/east/s/$/**VACA**/’ file sed '/north/a\ hello,word' datafile hello,word' datafile

sed: Using files  Tedious to type in commands at the prompt, especially if commands are repetitive  Can put commands in a file and sed can use them  sed –f cmds.sed data.txt file with commands

sed scripts  Series of commands can be put in a file and use the ‘-f’ option.  Can also create an sed script: s/vi/emacs/g /[Ww]indows/d p

Another Example  sed script to remove all HTML tags from a file: s/ ]*>//g p

Reading Assignment  Chapter 5