Command Line Variations Selected Readings in Chapters 3, 5, 6 u Project 1 kickoff u Streams & Redirection: Sections 5.2, 5.3 u Pipes and Tees: Sections 5.4, 5.5 u Filenames & Wildcards: Section 3.1 u Job Control: Section 5.10 u Filter Commands: details in Chapter 6 cat, tr, head, tail, cut, paste, sort, wc (word count)

Figure 5-4 Standard Streams

Standard I/O u Standard Output (stdout) labelled (1) above –default place to which programs write u Standard Input (stdin) labelled (0) above –default place from which programs read u Standard Error (stderr) labelled (2) above –default place where errors are reported u To demonstrate -- cat –Echoes everything you typed in with an –Quits when you press Ctrl-d at a new line -- (EOF)

Figure 5-6 Redirecting Standard Output

u cat file1 file2 > file3 –concatenates file1 and file2 into file3 –file3 is created if not there –Any pre-existing file3 is clobbered if there u cat file1 file2 >> file3 –file3 is created if not there –file3 is appended to if it is there u cat > file3 –file3 is created from whatever user provides from standard input

Figure 5-7 Print who Output

Redirecting Standard Error u In bash (for shell scripts), standard error is redirected using 2> –cat myfile > yourfile 2> yourerrorfile u A better example that generates errors: ls –R / recursively list all files from root you will get many screens of output u Redirect output into a file: ls –R / > allfiles save output from above you will get many “permission denied errors” more allfiles but you can look at listing in file now u Redirect errors to a second file: ls –R / > allfiles 2> errors errors go into file errors u If you don’t care about errors ls –R / > allfiles 2> /dev/null /dev/null is a ‘bit bucket’ where bits go to die!

Figure 5-5 Redirecting Standard Input

u terra[1] $ cat newfile u A more useful example: –terra[2] $ tr string1 string2 v Read from standard input. v Character n of string1 translated to character n of string2. v Results written to standard output. –Example of use: terra[3] $ tr aeoiu eoiua terra[4] $ tr a-z A-Z file2

/dev/null u /dev/null –A virtual file that is always empty. –Copy things to here and they disappear. v cp myfile /dev/null v mv myfile /dev/null –Copy from here and get an empty file. v cp /dev/null myfile –Redirect error messages to this file v cat myfile > yourfile 2> /dev/null v Basically, all error messages are discarded.

Figure 6-1 Concept Of A Filter

Basic Filters u Filters are programs that: –Read stdin. –Modify it. –Write the results to stdout. u Filters typically do not need user input. u Example: –tr (translate): v Read stdin v Echo to stdout, translating some specified characters u Many filters can also take file names as operands for input, instead of using stdin.

Filtering using tr (translate command) u Using tr to translate what you type into upper case tr [a-z] [A-Z] (enter, type, enter etc. ctrl-D to quit) u Using tr to translate a file using input redirection tr [a-z] [A-Z] < foods (now foods will appear UC) tr [a-z] [A-Z] foods (also OK, tr accepts filename args) u Using tr to translate a file using input redirection and store results using output redirection tr [a-z] [A-Z] foodsCAP tr [a-z] [A-Z] foods > foodsCAP (also OK, see above) (now foodsCAP will hold UC output from foods)

More filters: grep, wc, sort (see Ch6) u grep patternstr –Read stdin and write lines with patternstr to stdout terra[1] $ grep "unix is easy" myfile2 terra[1] $ grep "unix is easy" myfile1 > myfile2 (also OK) –Write all lines of myfile1 containing phrase unix is easy to myfile2 u wc –Count the number of chars/words/lines on stdin –Write the resulting statistics to stdout –Example: (wc < mars.txt OR wc mars.txt ) u sort –Sort all the input lines in alphabetical order and write to the standard output.

Figure 5-8 Piping Output to Next Command

Pipes u The pipe: –Connects stdout of one program with stdin of another –General form: command1 | command2 –stdout of command1 used as stdin for command2 –Example: terra[1] $ cat foods | grep apples | wc -l u A pipe works the same as (but more efficient than) terra[2] $ cat foods > tmp1 terra[3] $ grep apples tmp2 terra[4] $ wc -l < tmp2 u Actually cat is redundant: grep apples foods | wc -l

Redirecting and Pipes (2) u Note: The name of a command always comes first on the line. u There may be a tendency to say: terra[1] $ foods > grep apples | wc -l –This is WRONG!!! –Your shell will go looking for a program named foods u To do it correctly, many alternatives! terra[1] $ cat foods | grep apples | wc -l terra[2] $ grep apples < foods | wc -l terra[3] $ grep apples foods | wc -l terra[4] $ grep -c apples foods

The tee Command (don’t worry about this) u tee - replicate the standard output –cat readme.txt | tee myfile tee stdin stdoutmyfile

Unix Filename Rules u Almost any character is valid in a file name –all the punctuation and digits –the one exception is the / (slash) character –the following are not encouraged v ? * [ ] “ ” ’ ( ) & : ; ! –the following are not encouraged as the first character v - ~ –control characters are also allowed, but are not encouraged u UPPER and lower case letters are different –A.txt and a.txt are different files

Unix Filename Extensions u No enforced extensions –The following are all legal Unix file names v a v a. v.a v … v a.b.c u Remember files beginning with dot are hidden –ls cannot see them, use ls -a u. and.. are reserved for current and parent directories

Unix Filename Extensions u Even though Unix doesn't enforce extensions, –“.” and an extension are still used for clarity v.jpg for JPEG images v.tex for LaTeX files v.sh for shell scripts v.txt for text files v.mp3 for MP3’s –some applications may enforce their own extensions v Compilers look for these extensions by default –.c means a C program file –.C or.cpp or.cc for C++ program files –.h for C or C++ header files –.o means an object file

Unix Executable Files u Executable files usually have no extensions –cannot execute file a.exe by just typing a –telling executable files from data files can be difficult u “file” command Use:file filename Result:print the type of the file Example: terra[1] $ file ~/.bashrc.bashrc: executable bash-shell script u Filenames and pathnames have limits on lengths –1024 characters typically –these are pretty long (much better than MS-DOS days and the 8.3 filenames)

Fixing Filename Mistakes u It is very easy to get the wrong stuff into filenames –Say you accidentally typed terra[3] $ cp myfile -i –What if you type terra[4] $ rm -i v The shell thinks -i is an option, not a file v Getting rid of these files can be painful u There is an easy way to fix this... –You simply type terra[5] $ rm -- -i –Many commands use “--” to say there are no more options Creates a file with name -i

Filename Wildcarding (See section 3.1) u Wildcarding is the use of “special” characters to represent or match a sequence of other characters –a short sequence of characters can match a long one –a sequence may also match a large number of sequences u Often use wildcard characters to match filenames –filename substitution – generally known as “globbing” u Wildcard characters * matches a sequence of zero or more characters –Example: a*.c*matches abc.c, abra.cpp, ? matches any single character –Example: a?.cmatches ab.c, ax.c, but not abc.c [...] matches any one character between the braces –Example: b[aei]tmatches bat, bet, or bit, not baet

Filename Wildcarding (2) u Wildcard sequences can be combined terra[6] $ mv a*.[ch] cfiles/ v mv all files beginning with a and ending with.c or.h into the directory cfiles terra[7] $ ls [abc]*.? v list files whose name begins with a, b, or c and ends with. (dot) followed by a single character u Wildcards do not cross "/" boundaries –Example: csnow*c does not match csnow/codec u Wildcards are expanded by the shell, and not by the program –Programmers of commands do not worry about searching the directory tree for matching file names –The program just sees the list of files matched

Filename Wildcarding (3) u Matching the dot –A dot (. ) at v the beginning of a filename, or v immediately following a / must be matched explicitly. –Similar to the character / –Example: terra[8] $ cat.c* u As mentioned earlier, [....] matches any one of the characters enclosed –Within “[...]”, a pair of characters separated by “-” matches any character lexically between the two v Example: terra[9] $ ls [a-z]* cat all files whose names begin with.c lists all files beginning with a character between ASCII ‘a’ and ASCII ‘z’

Processes and Job Control

Figure 5-14 Job States

Foreground and Background (1) u Unix is a multi-tasking operating system –some of these tasks are being done by other users logged in –some are being done by you in the background v e.g. watching for incoming mail u When you run a task (a Unix command, like ls or vi) it executes in the foreground of your shell –it has the “control" of your screen and keyboard

Foreground and Background (2) u If you still want to use your shell command line v terra[1] $ a_heavy_task & v [1] v terra[2] $ u When you put a task in background –task keeps running, but you continue to work at the shell in the foreground –if any output is done, it appears on your screen immediately (can be confusing) –if input is required, process prints a message and stops –when it is done, a message will be printed

Example ls –R / (again) u Kick off a long command ls –R / > listall u Suspend, Ctrl-Z u Send to background bg [1] ls –R / >listall& u Oops, still get errors! Now you have to kill a background job: kill %1 (or kill -9 %1) [1] terminated u Restart it the right way, in background, with err redirect: ls –R / > listall 2> /dev/null &

Foreground and Background (3) u Explicit background processes are needed less often with windowing systems –Just go to another window and run the command u But explicit background processes are used often... –A command needs a long time, you do not want to close that window by accident –Run a job at the background and logout –netscape& will open a new window, but leave the current shell window still available to use

A Simple Script u We use the following shell script to illustrate job control u Edit a file make_noise terra[1] $ cat > make_noise #!/bin/sh while true do date sleep 1 done ^D terra[2] $ chmod u+x make_noise u make_noise then is a shell script repeats to print the time for every second, until you terminate it using Ctrl-c.

Job Control – Suspending Jobs u csh, tcsh, and bash allow you to manage the running of different processes u Suspending jobs –the Ctrl-z special character stops the job terra[1] $ make_noise Fri May 16 14:14:43 EDT 2003 …… ^Z Suspended terra[2] $ vi readme ^Z

Job Control - Monitoring Jobs u The "jobs" command shows which of your jobs are running and/or stopped. terra[3] $ jobs [1] + Suspended make_noise [2] + Suspended vi readme u Here there are two suspended processes, the make_noise and a vi process.

Job Control – Resuming Jobs u Putting jobs back into the foreground: –Use the "fg" command to move a job into the foreground. terra[4] $ fg %2 –Puts job number 2 into the foreground. –Works with either a background or stopped job. u Putting jobs into the background: terra[5] $ bg %1

Job Control – Killing Jobs u Jobs can also be killed –Use the Unix "kill" command terra[6] $ kill %1 or if it won't die... terra[7] $ kill –9 %1 u Jobs can be stopped and continued terra[8] $ a_heavy_task & terra[9] $ stop %1 terra[10] $ bg %1

Using ps (1) u Jobs are really just a special case of Unix processes u ps can list your current processes terra[11] $ ps PID TT S TIME COMMAND 2312 pts/0 T 0:00 vi 2296 pts/0 R 0:00 tcsh 2313 pts/0 R 0:00 ps

Using ps (2) u The ps command takes a number of options v -l gives you a long listing of what is going on v -u loginid tells you about loginid's processes v use man ps to see more options u kill pid kills the process pid –TERM signal will be sent to the process pid –kill -9 or kill -KILL will send the KILL signal –Use man –s 5 kill to find out more signals