Unix Talk #2 (sed)

You have learned… Regular expressions, grep, & egrep grep & egrep are tools used to search for text in a file AWK -- powerful What about SED?

Things in common between awk and sed They are invoked using similar syntax Stream-oriented (hence stream editor) Use regular expressions for pattern matching Allow the user to specify instructions in a script The command reads a stream of input from a file and processes it using text

Common syntax Command [options] script filename Script You tell the program what to do, which is called instructions Each instruction has two parts Pattern Procedure (actions) (Sound familiar?)

SED A tool usually designed for a short line substitution, deletion and print Allows for automation of editing process similar to those you might find in vi or ex (or ed) Non-destructive Reads in a stream of data from a file, performs a set of actions, & outputs the results

SED SED reads the data one line at a time, make a copy of the input line & places it in a buffer called the pattern space Modifies that copy in the pattern space Outputs the copy to standard output NOTE: 1.The pattern space holds the line of text currently being processed 2. You don’t make changes to the original file 3. If you want to capture this output, what do you do?

SED Syntax Options Sed syntax Must give sed instruction or scriptfile only prints matches f scriptfile run commands in scriptfile e allows multiple instructions on a single line Sed syntax Sed [option] ‘instruction’ file(s) Sed –f scriptfile file(s) Must give sed instruction or scriptfile Can use file redirection to create a new file

Sed options -e Only needed when you supply more than one instruction on the command line sed –e ‘script’ –e ‘script’ file NOTE: even if you have more than one instruction on the command line, there is still other ways to accomplish it without using -e

Print Command cat /home/fac/pub/fruit_prices sed –n 'p' fruit_prices What does it do? Try with out the –n. What happens?

Print Command Can specify zero, one or two addresses to print, the address can be a line number or the pattern for matching sed –n '1p' fruit_prices The line counter does not reset for multiple input files sed –n '$p' fruit_prices #prints last line sed –n '6,8p' #prints lines 8-10 sed –n '/^$/p' fruit_prices sed –n '1, /^$/p' fruit_prices # range sed –n '/App*/p' fruit_prices sed –n '/^[0-1]/p' fruit_prices sed –n '/[^7-9]$/p' fruit_prices What happens if you remove the caret?

Print Command sed –n '/\$1\./p' fruit_prices sed –n '/1./p' fruit_prices Need to use –n when printing, otherwise you get multiple copies of lines

Read data into a variable Create a script: read –p “Enter fruit name: ” fruitName sed –n “/$fruitName/p” fruit_prices Always surround you patterns with “” or ‘’ to prevent problems

Delete Command sed '/^A/d' fruit_prices Cat the file after you have run the command. Is the line gone?

Delete Command sed ‘1d’ fruit_prices sed ‘$d’ fruit_prices sed‘1,/^$/d’fruit_prices.txt > newfile

Substitute To change one pattern to another Syntax Flags s/pattern/replacement/flags Flags n: A number (1 to 512) indicating that a replacement should be made for only the nth occurrence of the pattern g: Make changes globally on all occurrences in the pattern space. Normally only the first occurrence is replaces.

Substitute If you like to change the original file, you must do sed ‘s/Fruit/Fruit_list/’ fruit_prices sed ‘s/a/A/g’ fruit_prices Try the previous command without g If you like to change the original file, you must do copy and redirect to update original file cp fruit_prices fruit_prices.old sed ‘s/Fruit/Fruit_list/’ fruit_prices.old>fruit_prices

Substitute Reuse the matched string with ‘&’ Sed ‘s/[0-9]\.[0-9][0-9]*/\$&/’ filename

sed – the Stream Editor.1 sed is an editor for editing data on the fly as part of a pipeline Usage: sed -e 'command' -e 'command' -e 'command' ... Reads stdin and applies the commands to each line in the order they are on the command line Prints each line of stdin to stdout after the commands have been applied Most commands are s/ . . . / . . . / commands Can use regexps in the 1st part Can use parentheses, back references in the 1st part Can use & and \1 \2 \3 . . . in the second part Can append ‘g’ to the s/ . . . / . . . / command to change all occurrences on a line

sed – the Stream Editor.2 Examples: Print out all usernames from /etc/passwd sed -e 's/^\([^:]*\):.*$/\1/' </etc/passwd Print out ONLY the hidden files in the working directory # delete lines that do NOT begin with a period ls -a | sed -e '/^[^.]/d‘ OR # print ONLY lines that DO begin with a period # NOTE: -n option suppresses printing unless # indicated via a p command ls -a | sed -n -e '/^\./p' Print out file names followed by the time of modification # NOTE: -r option enables extended regexps WITHOUT # the need to escape parentheses ls -l | sed -r -e 's/^([^ ]+ +){5}//' \ -e 's/^(.*) (.*)/\2 -> \1/' sed -r -e 's/^([^ ]+ +){5}//‘ #Remove first Five fields.

Append a \ text Appends text to the line following the one matched sed ‘/^F/a\ #here is a list of the fruit’ fruit_prices

Insert i \ text Inserts text prior to the line that is matched sed ‘/^Pi/i\ Orange: $1.99’ fruit_prices.old>fruit_prices

Scripts When? Series of sed commands Syntax sed –f scriptname datafilename

Scripts Cat scriptTest sed –f scriptTest fruit_prices s/^A/a/g s/^P/p/ sed –f scriptTest fruit_prices sed –e ‘s/^A/a/g’ –e ‘s/^P/p/’ fruit_prices

Using sed in a Pipeline id uid=500(yxp) gid=100(fac) id | sed ‘s/(.*$//’ uid=500 For another example of sed in a pipe look at my facebook script1 (??? Old one)

Using sed in a Pipeline cat fruit_prices | sed ‘s/A/a/g’ ls –l | sed –n p echo Hello | sed p