Review Please turn in your homework and practicals sed

Today Regular Expressions – Again! – Review grep & sed Again! (The Revenge!) awk

The Guy Who Draws XKCD I think I have a Bash problem. What follows is an actual command from my history. cat /usr/share/dict/words | fgrep -v "'" | perl -ne print join("", ".$_."\n";' | tee lookup.txt | perl -pe 's/^([^ ]+).*/\1/g' | awk '{ print length, $0 }' | sort -n | awk '{$1=""; print $0}' | uniq -c | sort -nr | egrep "^[^0-9]+2 " | awk '{ print length, $0 }' | sort -n | awk '{$1=""; print $0}' | perl -pe 's/[ 0-9]//g' | xargs -i grep {} lookup.txt | perl - pe 's/[^ ]+ //g' | tail -n2 It’s just so hard to bite the bullet, admit that the problem has grown in scope, and move it to its own Perl/Python script. (P.S. The Guinness Book is wrong. “Conservationalists” is not a real word.)

Quick Searching The grep utility is used for matching Printing to STDOUT – an ‘eyeball’ test or redirection to another file grep ‘static string’ file >> redirection grep ‘metacharacters’ file And grep starts at the first line, at the beginning of the row, reads across the row, and as soon as it matches its regex, prints Doesn’t modify, only needs to match once

Replacement The sed utility is used for ‘substitutions’ Can either execute once (by default), or across all entries (globally) Can also print sed ‘s/from/to/g’ file sed -n ‘/regex/p’ file

awk is Awkward Final regex utility is awk For use when your data is evenly formatted Teams.txt last two lines – Seattle Mariners – There might be some guys called the Cubs too – No even formatting First few lines – City Teamname – Even formatting

Example - /etc/passwd smithj:x:561:561:Joe Smith:/home/smithj:/bin/bash smith:*:100:100:8A- 74(office):/home/smith:/usr/bin/sh jsmith:x:1001:1000:John Smith:/home/jsmith:/bin/sh Between each entry is a : Username:Password:UID:GID:Name:Homedir:Shell Empty entries are still “bounded” (ie, :: indicates one empty value)

awk Examples awk -F ‘:’ ‘{ print $1 }’ /etc/passwd awk - command -F ‘:’ - -F sets our ‘delimiter’ to the : character ‘{ print $1 }’ - action /etc/passwd - file to use Prints the first column in the /etc/passwd file – Numbering starts at 1; 0 is the whole file – /etc/passwd is ‘delimited’ by : and empty values still have a : around them

Advanced awk I will not quiz you on this The more powerful awk commands are in scripts – #!/bin/bash – awk ‘\ – BEGIN { print “File\tOwner” } \ – { print $0, “\t”, $3} \ – END { print “ – DONE –” } \ – ‘

Case Study I used awk yesterday Something close to awk -F ‘,’ ‘{ print $1\tprint $7}’ example.csv | sort \ | uniq -c | sort -nr Counting audit findings so that we could see where to concentrate

Unfortunately I had to reimage my laptop over the weekend This wiped out our vm on my system I have basic commands, but they’re not guaranteed to work There may be one question on awk on the final

Finally I have yet to see my son play T-ball I will be “sick” a week from today (Wednesday, June 4 th ) so I can see his last game All late/resubmitted work is due June 9 th Final is Monday, June 16 th during normal class time Just under 50 questions, similar to quizzes

Own Study Regex’s Grep, sed and awk