1. Regular Expressions grep Familiy of Commands
Dr. Bill M.
2016

2. INCS-745 Intrusion Detection
Topics
Grep
Searching text
Searching logs
INCS-745 Intrusion Detection
Lecture 1.1
Copyright © R. A. Mihajlovic

3. INCS-745 Intrusion Detection
Homework
Perform 6 different examples of using grep command.
Use different RegExp strings.
INCS-745 Intrusion Detection
Lecture 1.1
Copyright © R. A. Mihajlovic

4. What is grep Command grep - "general regular expression parser“
Search command for UNIX.
Used to search for text strings and regular expressions within one or more files.
man grep

5. Grep Searches for text in a file Can search for simple words: “chair”
Can look for “regular expressions”; more complex character strings such as “chair” followed by any number of spaces, followed by a digit or lowercase letter.

6. Grep usage
grep “something” somefile.txt returns all lines with the word something from somefile.txt
grep -v “something” something.txt returns all lines that don't have the word something in them
grep -i “something” something.txt returns all lines with a mixed upper and lowercase something in them.

7. Simple regular expressions
“[0-9]” look for any digit
“[a-zA-Z]” look for one upper or lowercase letter
“.” look for one character
“.*” any number of characters
“\.” a literal decimal point
“\.161:” dot, then 161, then colon
“\.161[: ]” dot, then 161, then colon or space

8. Advanced regular expressions
Look for lines that hold either “dog” or “cat”
grep -e '(dog|cat)' animalfarm.txt
Lines that have cat followed by dog on the same line, but possibly with other characters in between:
grep 'cat.*dog' animalfarm.txt
cat has to be at the beginning of the line:
grep '^cat' animalfarm.txt
Look for it at the end of the line:
grep 'cat$' animalfarm.txt

9. Ways to use it Three identical ways to search in a file:
grep promiscuous messagesF
cat messagesF | grep promiscuous
grep promiscuous < messagesF
Look for something in multiple files:
(zcat /var/log/messages.*.gz ; cat /var/log/messages ) | grep 'promiscuous' | less

10. Example: Lab Firewall SNMP probes: 161,162
Grep “\.16[12][: ].*udp” firewall.log >snmp.txt
Inbound Unix traceroute
grep “\.33[45][0-9][0-9][: ].*udp” firewall.log >traceroute.txt
FW-1
, , , 900, 18207
grep -e “(\.25[6-9][: ]|\.26[0145][: ]|\.900[: ]|\.18207[: ])” firewall.log >fw1.txt
Half-life
grep “\.27015[: ].*udp” firewall.log >halflife.txt
AIX/broken PMTU
Size 1500 icmp echo request DF
grep “icmp: echo request (DF).*len 1500” firewall.log >aix.txt
Note we only get one of these (fw1?)

11. Example: firewall.log Search
197,128 lines
“\.4040[: ]” #CipherIM: 11M!
“\.(80\|8080)[: ]” #Web 10.5M
“\.53[: ]” #DNS 0.16M
“\.22[: ].*( S \|ack)” #ssh 202K
“\.25[: ]” 30K
“arp” #26K
“ripv1” #4K
“148\.64\.147\.168” #118K
“\.123[: ]” 4K
Result: 5K, 150 lines

12. Common grep Command Options
grep [options] pattern [files]
-b Display the block number at the beginning of each line.
-c Display the number of matched lines.
-h Display the matched lines, but do not display the filenames.
-i Ignore case sensitivity.
-l Display the filenames, but do not display the matched lines.
-n Display the matched lines and their line numbers.
-s Silent mode.
-v Display all lines that do NOT match.
-w Match whole word.
grep -c Alex my_file.htm

13. How to use grep command Search file for a user
$ grep ad85 /etc/passwd
Search file ignoring word case
$ grep -i “ad85" /etc/passwd
Search recursively all files and directories under given directory
$ grep -r “ad85" /etc/

14. How to use grep command Search for a specific word in file
$ grep -w “alex" $HOME/cs265.htm
Search for 2 different words in file
$ grep -w ‘alex|victoria' $HOME/cs265.htm
Count lines that matched in file
$ grep -c 'word' $HOME/cs265.htm

15. How to use grep command Display lines that did not match a pattern
$ grep -v cs265 $HOME/cs265.htm
Number of lines that contain matched pattern
$ grep -n 'word' $HOME/cs265.htm
Display filenames that matched pattern, but not lines from the files
$ grep -l ‘word' *.htm

16. grep and Wildcards Dot ( . ) – matches 1 character
Asterisks ( * ) – matches multiple characters
Examples:
grep b.g myfile  finds the words “big”, “bag”
grep b*k myfile  finds the word “back”, “buck”, “book”

17. grep and Regular Expressions
A "regular expression" is a pattern that describes a set of strings.
Regular expressions are used when you want to search for specific lines of text containing a particular pattern.

18. grep and Regular Expressions
^ (Caret) = match expression at the start of a line, as in ^A.
$ (Dollar Sign) = match expression at the end of a line, as in A$.
\ (Back Slash) = turn off the special meaning of the next character, as in \^.
[ ] (Brackets) = match any one of the enclosed characters, as in [aeiou]. Use Hyphen "-" for a range, as in [0-9].
[^ ] = match any one character except those enclosed in [ ], as in [^0-9].

19. grep and Regular Expressions
. (Period) = match a single character of any value, except end of line.
* (Asterisk) = match zero or more of the preceding character or expression.
\{x,y\} = match x to y occurrences of the preceding.
\{x\} = match exactly x occurrences of the preceding.
\{x,\} = match x or more occurrences of the preceding.

20. grep and Regular Expressions
grep bob files {search files for lines with ‘bob'}
grep '^bob' files {‘bob' at the start of a line}
grep ‘bob$' files {‘bob' at the end of a line}
grep '^bob$' files {lines containing only ‘bob'}
grep '\^b' files {lines starting with '^b', "\" escapes the ^}
grep '[Bb]mug' files {search for ‘Bob' or ‘bob'}
grep 'B[oO][bB]' files {search for BOB, Bob, BOb or BoB }
grep '^$' files {search for empty lines}
grep '[0-9][0-9]' files {search for pairs of numeric digits}

21. INCS-745 Intrusion Detection
The End
INCS-745 Intrusion Detection
Lecture 1.1
Copyright © R. A. Mihajlovic