Introduction to Unix – CS 21 Lecture 5

Lecture Overview Lab Review Useful commands that will illustrate today’s lecture Streams of input and output File redirection Piping between different programs Putting it all together for some powerful tools

Important Points From Lab Get used to both modes of chmod Understand how directory permissions affect access Write permission only doesn’t let you do anything Start thinking about using wildcards to speed up tasks

Good Commands To Know: wc Stands for word count Will report the number of lines, words, and characters in a file Works with all sorts of files, but only makes sense with text files Useful for all sorts of applications Either file direction or piping is required…

Example Of wc

sort Will sort the lines of a file into alphabetical order Useful on data files or output from programs Usage: sort [OPTIONS] [FILE]

Example Of sort

uniq Will go through a file and remove duplicate copies of a line Will only remove duplicate copies of lines that are adjacent Useful on files that might have repeated unnecessary data Usage: uniq [OPTIONS] [FILE]

Example Of uniq

Wouldn’t It Be Nice If We Could sort And uniq A File? No command exists that does both By the end of the lecture, we should see if this is possible Do you think this will be possible? Hint: Remember Unix was designed so that tasks could be easily automated and so that tasks could work together

What Exactly Is Input And Output? Input Everything a Unix program reads Usually from the keyboard What you type Output Everything a Unix program writes Usually to the screen What you see

Streams Of Input And Output stdin Input from the keyboard stdout Output to the screen stderr Error messages that show up on the screen

stdin Everything you type into the terminal The programs mostly expect all of their input from the user (you) typing at the keyboard

stdout The results of a running program Goes to the terminal you ran the program from

stderr Appears the same location as stdout Used for error messages Allows for the error messages to be separated from normal output

Redirecting These Streams Deciphering “echo ‘hi’ > helloFile” > Sends all output that normally would go to stdout into a file Completely overwrites whatever file was there or creates a new file if none existed

Example Of >

A Tricky Catch To Remember You must be careful because ‘>’ will overwrite files

Redirecting stdin < Makes all of the input for a file come from a file as opposed to the keyboard Allows you to input a lot of data without a lot of typing Exercise: Go to lab and run cat without any parameters

Example Of <

Redirection Of Output Without File Overwrite What if you want to add information to a file instead of overwriting it? >> Appends all output of a program to a file Will create a file if it doesn’t exist, but won’t overwrite any information currently in a file

Example Of Append Redirection

One More Stream Don’t forget about stderr 2> Not some weird smiley This says redirect stderr to a file Important to note why this notation exists 0 = stdin 1 = stdout 2 = stderr

Example Of Redirecting stderr

Example Of The Difference Between stdout And stderr

Is This Ever Useful? Short answer: Yes! Longer answer: Whenever you want to save the output of a program, redirect it Whenever you want to avoid typing input, redirect it Whenever you want to isolate any error messages, redirect it

Possible Uses For File Redirection Saving data generated by one program for use by another If you continually type the same input into different programs, place that data into a file and redirect it into the different programs

How To Remember Which Arrow Does What cat < inputFile Think of this as the inputFile dumping all of its contents into cat In other words, inputFile is going into cat cat > outputFile In this case, cat is dumping its results into outputFile In other words, cat is going into outputFile

What About Other Command Flags? The input or output file must come directly after the redirection operator Example: Right: ls –l * > listingOutput Wrong: ls > -l * listingOutput Question: How do you think this line is interpreted? As a general rule of thumb, redirection should come at the end of a line Redirection should come after all arguments to the program to avoid problems like the above

Combining These Flags Entirely possible to have both on the same line Each redirection operator assumes the next argument is what is being redirected Question: Can you have > and >> on the same line?

Answer

What Does This Command Do? ls /bin/doesntExist 2>> errorFile Appends stderr to the end of errorFile

What About These Commands?

Piping: Tying Programs Together We can combine some of the previous commands together with a pipe Pipes feed the output of one program into another | Yes, that’s a vertical bar

What Does A Pipe Look Like?

sort And uniq Answering our previous question, we see that it should be possible to both sort and uniq a file at once using pipes How exactly would we do it?

sort And uniq With Redirection

Same Example With Piping

How To Read Pipes You can have more than one pipe per line if you’d like Always read left to right, and output flows left to right

Putting It All Together What if we want to have both file redirection as well as piping? Well, things start to look ugly, but this is entirely doable

Complex Example

Really Complex Example

Is All This Stuff Worth It? Well, let’s compare this with how you do things in Windows With piping, several tasks can be combined and run in one single step What if I wanted to sort and uniq a Word file? Open the file and highlight all of the text Select Table->Sort (not intuitive) Specify how to sort and click o.k. Word doesn’t have a uniq feature…

Slowly, Your Power Grows With this information, you are slowly becoming a Unix power user Maybe you’ll save the world from dinosaurs yet…

Next Time More thorough explanation of wildcards Regular expression overview grep – what exactly is it? Quiz #1