Chapter 5: The Shell The Man in the Middle

In this chapter … The command line Input, output, and redirection Process management Wildcards and expansion Builtins

Let us recall … Shell is an interpreter Sits between the user and the kernel We’ll be using bash as our point of reference

The Command Line It is what allows us to talk to the shell A command line is a string of commands and arguments ending with Enter Shell takes the command line, interprets it, then instructs the kernel

Syntax command [arg1] [arg2] … [argn] Spaces separate command and arguments Arguments can be optional Some arguments are called Options, and are proceeded by one or more hyphens (aka switches or flags)

Do I Need Arguments? Some commands work without any arguments specified (ex., ps) Others need one or more If a command expects at least one argument, often it will display a usage message if you execute the command without any arguments

Tokens Each space-delimited word in a command line is called a token and are numbered from left to right starting at zero That means the command is token 0, first arg is token 1, etc. Token can be a filename, a string, a number

Options An argument that modifies the effect or execution of the command Traditionally, options precede other arguments Most utilities use a single hyphen; some (such as several GNU utilities) also use double hyphens Sometimes you can combine options together with a a single preceding hyphen

Command Line Example ls -R -l -h public_html Could also be written as: ls –Rlh public_html LiSt Contents list directories recursively display long listing human readable format for sizes directory to perform this ls command upon

Mechanics Once enter is pressed, command line is parsed by the shell Reads the first token and tries to find the executable or builtin associated with it Passes remaining tokens to the called program, doing expansion and manipulation as necessary

Where’s the program? If not given absolute pathname, shell searches PATH variable for location of first token Can’t assume PATH includes your current working directory If it can’t locate it, shell returns command not found If no execute permissions, returns access denied

Execution Shell begins a new process and hands control over to called program Shell then goes to sleep, and waits for called program to finish running and return control Program also passes its exit status to shell

Streams Streams contain data (usually text) Three standard streams –Input –Output –Error Each stream can be associated with any number of things (remember, everything is a file!)

Standard Input Contains information to be passed to a program (a utility, a shell, etc) Most commonly is the keyboard Also commonly a file Because everything is treated as a file, programs never know exactly where input truly comes from

Standard Output Contains output from a program Again, program never sure where it’s sending output to Most commonly the screen Could also go to a printer, a file, or directly into standard input (to another program)

Standard Error Another output stream Contains error messages Usually also goes to screen You can redirect both standard out and standard error to different places

Fun with cat cat displays the contents of files Try calling cat with no argument It takes input from standard input (instead of a file) Will echo back every line you type Hit CTRL-D (EOF) to quit

Redirection Temporarily changes where standard streams point to After doing a redirection, streams return to normal There are ways to permanently change streams – beyond our scope

Redirecting Output Syntax: command [arguments] > output Output is usually a file, but can be anything (for example, a printer) The greater-than sign instructs the shell to redirect standard out to whatever is to the right If output exists, it will be overwritten – be careful!

Redirecting Output con’t To prevent overwriting, we can set the noclobber variable You can override noclobber using >| To append to the end of a file, use >> You can’t redirect output to a file you’re using as input – the shell allocates the file for the output stream before calling the program

Redirecting Errors Syntax: command [args] 2> errors You’re redirecting the second output stream (standard error) Standard output can also be written 1> You can redirect both streams: –find whizzbang > results 2> /dev/null

Redirecting Input Syntax: command [arguments] < input Input usually is a file Not all that useful if the command supports a filename for one of the arguments Can be handy if the command prompts for input … you can automate the process with a file filled with the answers

Pipes Redirects the output of one program to be the input of another Reduces the need for intermediary steps Handy for when you have several quick steps to perform upon some data Great for getting output and quickly filtering it (ie. command | grep search_string )

Getting Crazy tee allows you to send output to two different locations at once It sends output to a file, while still passing the output onto standard output Cascading tee commands can go on indefinitely

Running Jobs in the Background Thus far we’ve always run commands in the foreground Have to wait until it finishes before you can access the command line again To run in the background, place a & at the end of your command line before hitting enter

Background, con’t Once you hit enter, it will display the job number, and the process ID number (PID) You may resume issuing commands Once the job finishes, the job number and the word Done will appear on your screen Remember, to check on things, use jobs

Back and Forth Recall that pressing CNTRL-Z suspends a process, and it gives it a job number To start it again in the foreground, use: fg job_num To start it again in the background, use: bg job_num

kill Revisited Recall we could do: kill %job_num You can also: kill PID Forgot the PID? Just use ps Won’t die? Don’t forget kill -KILL

Metacharacters Special characters interpreted by the shell Also called wildcards Shell expands ambiguous file references into a list of files matching that criteria Process is called globbing or expansion

? Character Used to represent any single character For example: foo? –Would match food, fool, foot –Would NOT match fooey, footsie Can be anywhere in the filename Ex: b??kk??pper On exception – doesn’t match leading periods (for hidden files)

* Character Similar to ?, but matches any number of characters This includes zero characters Ex: foo* –Would match food, foot, footsie, foo Also does not match leading periods

[ ] Characters Used to match a list of possible characters Also known as a character class Ex: [aeiou].jpg –Matches a.jpg, e.jpg, i.jpg, etc Can also do ranges of letters / numbers –Ex: [0-9] [a-z] [A-Z]

[ ] Characters con’t You can negate a character set by placing a ! or ^ after the first bracket Ex: [^0-9] –Does not match the numbers 0-9

Metacharacter Tips You can combine multiple metacharacters of different types to get really specific The shell does all the expanding, not the program or utility Programs never see the metacharacters To let the program see them, you must quote them or escape them

Builtins Commands that are built into the shell Shell does not create a new process to run a builtin Run very quickly Shell will still do metacharacter expansion for builtins We’ll cover them in greater detail in Ch 10