1. More Unix Utilities (find, diff/cmp, at/crontab, tr)
CS465
More Unix Utilities
(find, diff/cmp, at/crontab, tr)

2. The find command
The find command walks down a subtree of the file system looking for files which match certain criteria:
wildcards
file type (e.g. file, directory, special, link)
file’s size, owner, access time / modification time
etc..

3. Format of the find command
$ find [start-dir] [criteria] [what-to-do]
start-dir defines the directory to start from.
Each subdirectory of this directory will be searched
criteria is what criteria to use to find the files
Most common – by filename
what-to-do tells what to do with the files found. Can be any Unix command.
Most common: print to a file
- display to the screen (default)

4. Starting Directories Most often used starting directories are:
$HOME users home directory
. current directory
/ root directory (searches whole system)

5. Some find criterion options
-name pattern by filename (can use wildcards)
-perm oct by octal permission settings
-user/-group by owner or group
-size/-type by size or type
-atime/-mtime by last date accessed or modified
-links by number of links to the file
! by negated criteria

6. find examples $ find ~ -name projX -print $ find /etc -user root -ls
Starting in my home directory, find any file whose name is projX and print their full pathnames
$ find /etc -user root -ls
Find all files under /etc owned by root and display in a long listing format
$ find $HOME ! -type l
Starting in my home directory, display (default) all files that are NOT link-type files

7. find examples $ ls -F pattern.txt pattern2.txt regisdir/ $ ls regisdir
pattern.txt p.txt
$ find . –name pattern.txt
./pattern.txt
./regisdir/pattern.txt
$ find . –type f
./pattern.txt
./pattern2.txt
./regisdir/p.txt
./regisdir/pattern.txt
$ find . –type d .
./regisdir

8. find examples $ ls -l $ ls -l regisdir $ find . –user jones
-rw-rw-r-- 1 jones design May pattern.txt
-rwxrwxr-x 1 jones play Feb 8 13:52 pattern2.txt
drwxrwxr-x 2 smith design Feb 8 13:42 regisdir
$ ls -l regisdir
-rwxrwxr-x 1 smith design Feb 1 10:44 p.txt
$ find . –user jones
./pattern.txt
./pattern2.txt
$ find . –group design
./pattern.txt
./regisdir
./regisdir/p.txt

9. find examples $ ls -l $ ls -l regisdir $ find . –perm 775
-rw-rw-r-- 1 jones design May pattern.txt
-rwxrwxr-x 1 jones play Feb 8 13:52 pattern2.txt
drwxrwxr-x 2 smith design Feb 8 13:42 regisdir
$ ls -l regisdir
-rwxrwxr-x 1 smith design Feb 1 10:44 p.txt
-rw-rw-r-- 1 smith design Feb 1 10:45 p2.txt
$ find . –perm 775
./pattern2.txt
./regisdir
./regisdir/p.txt
$ find . –perm 664
./pattern.txt
./regisdir/p2.txt

10. Searching the Entire System
When you search from root, redirect the error messages to a file
Prevents seeing error message for every directory you don't have permission to view

11. Root search Example $ find / -name test -print
find: cannot read dir /lost+found: Permission denied
find: cannot read dir /usr/lost+found: Permission denied
find: cannot read dir /usr/dodd/progs Permission denied
find: cannot read dir /usr/jmsmith Permission denied
/usr/kcjones/new/test
/usr/lhlester/scripts/test
find: cannot read dir /usr/mkfox Permission denied
find: cannot read dir /usr/pjsmith Permission denied
/usr/wsfrank/test $
$ find / -name test -print 2> errmsgs
/usr/kcjones/new/test
/usr/lhlester/scripts/test
/usr/wsfrank/test $

12. find Filename Wildcards
You can use filename wildcards with the name option. But if you do, you need to ADD either:
a backslash in front of the wildcard OR-
single quotes around the filename containing the wildcard
to prevent wildcard expansion by the shell (you want the find command to do the expansion of the wildcard, not the shell).

13. find Wildcarding Example
$ echo $HOME
/usr/home/jmsmith $
$ find $HOME –name 'p*.c' –print
/usr/home/jmsmith/prog1.c
/usr/home/jmsmith/hmwk/pass.c $

14. More find examples $ find $HOME -atime +30 -print
Display all files within your directories that have not been accessed within the last 30 days
NOTE: 30 accessed exactly 30 days ago accessed less than 30 days ago accessed more than 30 days ago
$ find /usr/jsmith -name '*.ksh'
- Display all Korn shell files in jsmith’s directory structure.
Remember, -print is the default action if none is specified

15. Combining find criteria
Criteria can be combined using –a and –o and parentheses (backslashed)
-a Logical AND
-o Logical OR
Example: Display all files that have a .ksh extension and have been accessed within the last 30 days.
$ find $HOME \( -name '*.ksh' -a -atime \ -30 \) -print

16. find examples $ touch soccer.doc $ ls -F
apples apples.c oranges oranges.c
pattern.txt pattern2.txt regisdir/ soccer.doc
$ ls regisdir
Other pattern.txt whole.doc
$ find . –name '*.doc' –o –name '*.txt'
./regisdir/pattern.txt
./regisdir/whole.doc
./pattern.txt
./pattern2.txt
./soccer.doc

17. Additional find actions
If you want to perform an action that is not in the list of find options, you can use find’s -exec option to execute ANY Unix command:
-exec command {} \;
Notes: {} (curly braces) represent the files found
\; (backslash semicolon) ends the exec command

18. -exec Examples
Find all file ending in .c and move them to the cprogs subdirectory.
$ find $HOME -name '*.c' \ -exec mv {} $HOME/cprogs \;
Find all files within your directories that have not been accessed within the last 30 days and delete them
$ find $HOME -atime +30 –exec rm {} \;

19. find examples $ ls -F pattern.txt pattern2.txt regisdir/ $ ls regisdir
$ find . –name 'pattern*' –exec cp {} regisdir \;
pattern.txt pattern2.txt
$ find regisdir –name '*2.txt' –exec rm {} \;
$ ls regisdir
pattern.txt

20. find Exercise
Starting in your home directory, search for all files that have been accessed within the last 3 days and print their full pathnames.
$ find $HOME -atime -3 -print
Expanding the previous exercise, search for all files that have been accessed within the last 3 days and that have a .ksh extension and copy them to subdirectory “current”, under $HOME.
$ find $HOME \( -name '*.ksh' -a -atime -3 \) –exec cp {} $HOME/current \;

21. The diff command diff compares two files
Format:
$ diff [options] file1 file2
Options:
-b ignore repeating blanks
-i ignore case
-w ignore ALL spaces and tabs
Output indicates which lines need be added (a), deleted (d) or changed (c).
Lines in file1 are identified with a (<) symbol and lines in file2 with a (>) symbol.

22. diff command output format
The output from diff is an ed editor script that would convert the first file to the second file if run with the first file as input.
Additions
firstStart a secondStart, secondStop
> lines from the second file to add to the first file
Deletions
firstStart, firstStop d lineCount
< lines from the first file to delete
Changes
firstStart, firstStop c secondStart, secondStop
< lines in the first file to be replaced
> lines in the second file to be used for the replacement

23. diff Example $ cat file1 hello $ cat file2 $ $ diff file1 file2 2d1

24. Using diff output with ed
$ cat file1
hello $
$ diff file1 file2 | ed file1
$ cat file1
hello $
$ diff file1 file2 $

25. diff examples $ diff team3 team4 $ cat team1 $cat team2 Mike Mike
John Scott
Kathy Kathy
Martin Bob
$ diff team1 team2
2c2
< John
---
> Scott
4c4
< Martin
> Bob
$ cat team3 $ cat team4
Scott Mike
Scott
$ diff team3 team4
0a1
> Mike
$ diff team4 team3
1d0
< Mike

26. Using diff with directories
Shows which files are unique to each directory
$ ls sub*
sub1:
file1 prog.c test
sub2:
file1 memo prog.c $
$ diff sub1 sub2
Only in sub2: memo
Only in sub1: test $

27. The cmp command cmp also compares two files
Instead of line by line, like diff, compares files character by character until it finds a character mismatch.
Outputs the character number and line number where the FIRST mismatch was discovered.
Example:
$ cmp filev1 filev2
filev1 filev2 differ: char 60, line 7 $

28. cmp command Format: $ cmp [options] file1 file2 Option:
-s work silently; print nothing, but return exit codes, as follows:
0 Files are identical
1 Files are different
2 Files are inaccessible

29. cmp examples $ cat team1 Mike John $ cat team2 Stephen Scott
$ cmp team1 team2
team1 team2 differ: char 1, line 1
$ cp team1 team3
$ cmp team1 team3 $
$ cmp –s team1 team3 && echo "identical"
identical

30. cmp command option
Can also create a listing of ALL character differences in the file, using -l option.
Column 1: character number (byte number) within the file
Column 2: ASCII code of character within first file
Column 3: ASCII code of character within second file

31. cmp Example $ ls file1 Hello Today is Monday $ ls file2
Today is Friday
$ ls file3 $
$ cmp file1 file2 $
$ cmp file1 file3
cmp: EOF on f3 $

32. Process Scheduling
Two Unix utilities let you schedule commands to run at specific times.
crontab - lets you schedule commands that need to run on a periodic basis.
at - lets you schedule a command to run only once.

33. The crontab file
The crontab command uses a crontab file, which contains a list of commands and when to run them.
Commands in your crontab file can be Unix utilities, shell scripts, or executable programs.
If the command generates output, the output should be re-directed to a file or terminal
Each user may register ONE crontab file (any name).
Once the file is registered, the system runs the commands at the scheduled times.

34. Crontab file specs A crontab file contains 6 columns:
Column 1 contains: minute (0-59)
Column 2 contains: hour (0-23, 24-hour clock)
Column 3 contains: day of month (1-31)
Column 4 contains: month (1-12)
Column 5 contains: weekday (1-7, where 1 = Monday)
Column 6 contains: command to run
Columns 2 to 5 may also contain a range of hours, days, or months, or the asterisk symbol. The asterisk says to match ALL hours, days, or months.

35. Sample crontab file
$ cat crontab.def
* * $HOME/monthly_cleanup.ksh
45 11 * * 1-5 echo Lunchtime!! > /dev/tty5 $
First line will run a the script file called monthly_cleanup.ksh at 0300 (3 am) on the 1st of every month.
Second line will display "Lunchtime!" to terminal 5, Monday through Friday, at 1145 (11:45 am)

36. Invoking crontab
After you create a crontab file, you must register it using the crontab command, so the system will begin running your commands.
Once your file is registered, you can list its contents using the crontab -l command.
You can stop the system from running the commands by un-registering the crontab file using the crontab -r

37. crontab Example 1
Remove .tmp files from home directory every morning at 6 am:
$ cat crontab.def
0 6 * * * rm ${HOME}/*.tmp
$ crontab crontab.def $

38. crontab Example 2 $ cat cronfile 0 3 1 * * $HOME/monthly_cleanup.ksh
45 11 * * 1-5 mail $USER < lunchmsg
$ crontab cronfile
$ crontab -l
* * $HOME/monthly_cleanup.ksh
45 11 * * 1-5 mail $USER < lunchmsg
$ crontab -r
$ crontab -l $

39. crontab Example 3 Root can view ANY user’s registered crontab file:
# crontab –l jsmith
0 3 1 * * $HOME/removeTmps
# crontab –l mjones
30 11 * * 1-5 mail $USER < lunchmsg #

40. cron Output
cron usually mails output and errors to the cron user. Other methods:
Method 1 - Discard output
10 * * * * cronscript > /dev/null 2> &1
Method 2 - Mail output to someone else
10 * * * * cronscript | mail jsmith

41. The at utility
If you only want to run a command once (instead of on a regular schedule), but at a specific time, you can use the at command.
Syntax:
$ at time < scriptfile
Time field can contain:
specific date and time
date/time relative to the current time
month names, day names, and keywords like now, tomorrow, next

42. at options Options when running at:
-c, -k and -s tell shell to run the command under a specific shell (C, Korn, or Bourne)
NOTE: Not available on acadunix -- all automatically run under Bourne shell
-m tells the system to send you mail when the command has finished running
Options after a job has been scheduled: 
-l displays scheduled jobs
-r job# de-schedules the specifed job

43. at Examples Run script1 at 2:00 am next Saturday
$ at 0200 sat < script1
Run script2 an hour from now
$ at now + 1 hour < script2
Display scheduled jobs
$ at –l
a Sat May 19 02:00:
a Fri May 18 18:26: $

44. More at Examples Run script3 at 5:00 pm tomorrow
$ at 5pm tomorrow < script3
Run script4 at 10:00 am on October 1st
$ at 1000 Oct 1 < script4
Cancel job scheduled for Saturday
$ at –l
a Sat May 19 02:00:
a Fri May 18 18:26:
$ at -r a $

45. Re-scheduling at
You can make a script re-schedule itself by making the last command in the script be another at command.
Example:
$ cat deltmp.ksh
#! /bin/ksh
rm ${HOME}/*.tmp
at now +1 day < rmtmp.ksh
$ at 6am tomorrow < rmtmp.ksh $

46. Longer Re-scheduling Script
Check to see if a specific user is logged on, and pipe the output to the checkfile file (if the user is NOT logged on, no file will be created).
Check if the checkfile file exists.
If the file does NOT exist, the script will re-schedule itself to run again in 10 minutes.
If the file does exist, the script will display a message on terminal 5 (assumed to be your terminal) and delete checkfile.

47. Script Contents $ cat check #!/bin/ksh
# $1 = userid of person to check for #
who | fgrep $1 > checkfile
if [ -s checkfile ]
then
echo $1 is logged on now > mailmsg
mail small000 < mailmsg
rm checkfile mailmsg
else
at now +10 minutes < $HOME/check $1 fi
exit 0 $

48. Permissions for at and crontab
The system administrator can allow or deny both at and crontab use for each user on the system.
The files that control access are located in the directory /usr/lib/cron
crontab is controlled by files cron.allow and cron.deny
at is controlled by files at.allow and at.deny.
If no allow files exist, then only the users listed in the deny files are denied access.

49. tr utility
Translates characters in the standard input, via substitution or deletion of characters
Format:
tr [options] [string1] [string2]
Meaning:
Match characters in string1 and substitute characters from string2.

50. tr Example #1 Using tr for case conversion: $ cat names Joe Smith
Mary Jones
$ tr '[A-Z]' '[a-z]' < names
joe smith
mary jones $

51. tr Example #2 Using tr to convert spaces to newlines: $ cat names
Joe Smith
Mary Jones
$ tr ' ' '\012' < names
Joe
Smith
Mary
Jones $

52. tr –d option The –d option deletes all characters listed in string1
Using tr to delete spaces and lowercase vowels:
$ cat names
Joe Smith
Mary Jones
$ tr –d 'aeiou ' < names > names.tr
$ cat names.tr
JSmth
MryJns $

53. tr –s option
The –s option strips out repeated characters listed in string1
Using tr to strip out blank lines:
$ cat names
Joe
Mary
$ tr –s '\012' < names $