Introduction to Unix – CS 21 Lecture 13

Lecture Overview Finding files and programs which whereis find xargs Putting it all together for some complex programs

What Do You Do When You Want To Locate A File? Option 1: ls Can be very time consuming Option 2: Other programs which Used to see what you are running whereis Used to see where programs are located find A general tool to find all sorts of files in the directory

which Usage: which PROGRAMNAME Tells you what executable is being run when you run a program Useful to tell when more than one executable exists (i.e. one in /bin and one in /sbin) Only searches your PATH

Example Of which

whereis Usage: whereis PROGRAMNAME Locates the source, executable, and man page for a command Searches set paths

Example Of whereis

find Locate files based on any number of factors More than just the name When it was modified What permissions it has What size it is

Running find Usage: find [path] [expression] A little different from all other programs you’ve run Instead of a letter, options are the complete word Example: Instead of –n, you use -name

Expression Options To find -name “pattern” Matches the name of a file -iname Matches the name of a file but is case-insensitive -empty Matches empty files -type d f l

Examples

More Expression Options -atime N File was last accessed N days ago -mtime N File was last changed N days ago +N, -N +N = More than N days ago -N = Less than N days ago

Examples

Combining Expression Options -o Or -a And \( \) Grouping

Action Commands -print Simply prints out the name of the file that was found Most common action -exec Executes a command -ok Executes a command, but prompts the user first

The exec Option – A Powerful But Dangerous Option -exec COMMAND \; {} gets replaced with the name of the file that was found Example: find. –name “*.txt” –exec echo {} \; Same as find. –name “*.txt” -print

Examples

When Would You Ever Do This? The –exec option allows you to perform some command on all files that match a certain criteria Example: Someone cracked into your system and you want to check all recently modified files for any changes against a backup find –mtime –2 –exec diff {}.snapshot/{}

Problem The exec command will run the command once for every file that matches A lot of overhead May take a lot longer than you’d like Solution: xargs

The xargs Command Take values from standard in and convert them into command line parameters Usage: xargs COMMAND Example: xargs grep pattern

What? File1.txt File2.txt … Program stdin xargs Program File1.txt File2.txt …

Why? When a program outputs a list that will be used as input to another program Used mainly in pipes with find Example: find. –name *.txt | xargs grep ‘Jason’ find. –name *.txt –exec grep ‘Jason’ {} \; Calls grep many, many times less than the second call and takes less time

Another xargs example Problem: You want to kill all of your processes First, list all your processes ps aux | grep login Next, select all the PIDs cut –d’ ‘ –f2 Now pass all those PIDs to kill ps aux | grep login | cut –d’ ‘ –f2 | xargs kill -9

A Simple Cleanup Script #!/bin/bash ps aux | grep login | cut –d’ ‘ –f2 | xargs kill -9 exit 0 >./exitScript.sh &

When Would You Use Bash Versus Awk? What is bash good at? Finding and manipulating files and directories What is bash bad at? Manipulating data inside files What is awk good at? Manipulating data inside files What is awk bad at? Finding and manipulating files and directories

Building A Complex Program Start small Always try to get something working before you get something working well Build up Add functionality one part at a time and test as you go

Building A Complex Program In Bash Starting small – #!/bin/bash exit 0 Slowly add functionality #!/bin/bash read input echo $input exit 0

Building Up A Complex Program – In awk Start small { print $0 } Slowly add functionality /Pattern/ { print $0 }

Hints Break the problem down into smaller components Try printing first – always print as you go Insert temporary files or variables if needed Example: If you are required to process any number of command line parameters one at a time, start by handling just one After you can handle one, try two Generalize after you have gotten the first part down

Example

Computing Problem Find all files we haven’t changed in 2 days and move them to a folder called oldStuff Should we use bash or awk? Bash shell programming would be better

Break It Down Find all files we haven’t changed in two days find ~ –mtime +2 -print Move each file to the oldStuff folder mv file ~/oldStuff/

Put It Together We need some way to connect the first part with the second part How about a variable? Store the list of file names and then go through them one by one var=`command`

Script Solution #!/bin/bash filesToMove = `find ~ -mtime +2 –print` for f in $filesToMove do mv $f ~/oldStuff/ done exit 0

Another Computing Problem Remove all files that have the phrase “Remove Me” located on the first line Which should we use, bash or awk? We are looking inside files, but we aren’t manipulating the data, so we should use bash.

Break It Down Find all files find. –name “*” -print Get the first line head -1 Look for “Remove Me” grep “Remove Me” Remove the file if “Remove Me” was there rm file

Put It Together Variables to pass information along List of all files Temporary files or variables to store results and test Results from grep

One Script Solution #!/bin/bash allFiles=`find. –name “*” –print` for f in $allFiles do head –1 $f | grep “Remove Me” >.tmpFile if [ -s.tmpFile ] ; then rm –i $f fi done rm –f.tmpFile exit 0

Another Computing Problem Tax calculator Given a receipt, calculate the tax Receipt is a file containing the name of the product and price of the product on one line Pork Tenderloin:10.85 Should we use bash or awk? awk would be better in this case

Break It Down Calculate the tax on an individual item Get the item $2 Multiply it by the tax $2 *.0775 Sum up all the taxes

Put It Together To keep a sum, we need to store the running total in some location How about a variable? totalTax = totalTax + newTax We need to initialize the tax in the beginning and print out the tax at the end

Awk Script NF == 2 { totalTax = totalTax + $2 *.0775 } BEGIN { totalTax=0 } END { print totalTax }

Problem 4 Given a file that contains inventory, print out all items that are out of stock File format: ITEM/PRICE/QUANTITY Example: Needlenose pliers/2.50/4 Should we use bash or awk? Awk is better this time since we are checking the data

Script Solution { if ($3 == 0) { print $1 } }

Next Time Quiz #2 Bash and awk programs sed Perl Programming Introduction