Regular Expressions grep Familiy of Commands Dr. Bill M. 2016

INCS-745 Intrusion Detection Topics Grep Searching text Searching logs INCS-745 Intrusion Detection Lecture 1.1 Copyright © R. A. Mihajlovic

INCS-745 Intrusion Detection Homework Perform 6 different examples of using grep command. Use different RegExp strings. INCS-745 Intrusion Detection Lecture 1.1 Copyright © R. A. Mihajlovic

What is grep Command grep - "general regular expression parser“ Search command for UNIX. Used to search for text strings and regular expressions within one or more files. man grep

Grep Searches for text in a file Can search for simple words: “chair” Can look for “regular expressions”; more complex character strings such as “chair” followed by any number of spaces, followed by a digit or lowercase letter.

Grep usage grep “something” somefile.txt returns all lines with the word something from somefile.txt grep -v “something” something.txt returns all lines that don't have the word something in them grep -i “something” something.txt returns all lines with a mixed upper and lowercase something in them.

Simple regular expressions “[0-9]” look for any digit “[a-zA-Z]” look for one upper or lowercase letter “.” look for one character “.*” any number of characters “\.” a literal decimal point “\.161:” dot, then 161, then colon “\.161[: ]” dot, then 161, then colon or space

Advanced regular expressions Look for lines that hold either “dog” or “cat” grep -e '(dog|cat)' animalfarm.txt Lines that have cat followed by dog on the same line, but possibly with other characters in between: grep 'cat.*dog' animalfarm.txt cat has to be at the beginning of the line: grep '^cat' animalfarm.txt Look for it at the end of the line: grep 'cat$' animalfarm.txt

Ways to use it Three identical ways to search in a file: grep promiscuous messagesF cat messagesF | grep promiscuous grep promiscuous < messagesF Look for something in multiple files: (zcat /var/log/messages.*.gz ; cat /var/log/messages ) | grep 'promiscuous' | less

Example: Lab Firewall SNMP probes: 161,162 Grep “\.16[12][: ].*udp” firewall.log >snmp.txt Inbound Unix traceroute grep “\.33[45][0-9][0-9][: ].*udp” firewall.log >traceroute.txt FW-1 256-259, 260-261, 264-265, 900, 18207 grep -e “(\.25[6-9][: ]|\.26[0145][: ]|\.900[: ]|\.18207[: ])” firewall.log >fw1.txt Half-life grep “\.27015[: ].*udp” firewall.log >halflife.txt AIX/broken PMTU Size 1500 icmp echo request DF grep “icmp: echo request (DF).*len 1500” firewall.log >aix.txt Note we only get one of these (fw1?)

Example: firewall.log Search 197,128 lines “\.4040[: ]” #CipherIM: 11M! “\.(80\|8080)[: ]” #Web 10.5M “\.53[: ]” #DNS 0.16M “\.22[: ].*( S \|ack)” #ssh 202K “\.25[: ]” 30K “arp” #26K “ripv1” #4K “148\.64\.147\.168” #118K “\.123[: ]” 4K Result: 5K, 150 lines

Common grep Command Options grep [options] pattern [files] -b Display the block number at the beginning of each line. -c Display the number of matched lines. -h Display the matched lines, but do not display the filenames. -i Ignore case sensitivity. -l Display the filenames, but do not display the matched lines. -n Display the matched lines and their line numbers. -s Silent mode. -v Display all lines that do NOT match. -w Match whole word. grep -c Alex my_file.htm

How to use grep command Search file for a user $ grep ad85 /etc/passwd Search file ignoring word case $ grep -i “ad85" /etc/passwd Search recursively all files and directories under given directory $ grep -r “ad85" /etc/

How to use grep command Search for a specific word in file $ grep -w “alex" $HOME/cs265.htm Search for 2 different words in file $ grep -w ‘alex|victoria' $HOME/cs265.htm Count lines that matched in file $ grep -c 'word' $HOME/cs265.htm

How to use grep command Display lines that did not match a pattern $ grep -v cs265 $HOME/cs265.htm Number of lines that contain matched pattern $ grep -n 'word' $HOME/cs265.htm Display filenames that matched pattern, but not lines from the files $ grep -l ‘word' *.htm

grep and Wildcards Dot ( . ) – matches 1 character Asterisks ( * ) – matches multiple characters Examples: grep b.g myfile  finds the words “big”, “bag” grep b*k myfile  finds the word “back”, “buck”, “book”

grep and Regular Expressions A "regular expression" is a pattern that describes a set of strings. Regular expressions are used when you want to search for specific lines of text containing a particular pattern.

grep and Regular Expressions ^ (Caret) = match expression at the start of a line, as in ^A. $ (Dollar Sign) = match expression at the end of a line, as in A$. \ (Back Slash) = turn off the special meaning of the next character, as in \^. [ ] (Brackets) = match any one of the enclosed characters, as in [aeiou]. Use Hyphen "-" for a range, as in [0-9]. [^ ] = match any one character except those enclosed in [ ], as in [^0-9].

grep and Regular Expressions . (Period) = match a single character of any value, except end of line. * (Asterisk) = match zero or more of the preceding character or expression. \{x,y\} = match x to y occurrences of the preceding. \{x\} = match exactly x occurrences of the preceding. \{x,\} = match x or more occurrences of the preceding.

grep and Regular Expressions grep bob files {search files for lines with ‘bob'} grep '^bob' files {‘bob' at the start of a line} grep ‘bob$' files {‘bob' at the end of a line} grep '^bob$' files {lines containing only ‘bob'} grep '\^b' files {lines starting with '^b', "\" escapes the ^} grep '[Bb]mug' files {search for ‘Bob' or ‘bob'} grep 'B[oO][bB]' files {search for BOB, Bob, BOb or BoB } grep '^$' files {search for empty lines} grep '[0-9][0-9]' files {search for pairs of numeric digits}

INCS-745 Intrusion Detection The End INCS-745 Intrusion Detection Lecture 1.1 Copyright © R. A. Mihajlovic