1. 1 Commands, editing, Variables More commands, and more details –echo- uniq- find –grep- sort –sed- diff –cut- pr Editing –vi- emacs Shell Variables Environment Variables 1

2. 2 echo to show stuff on the screen “Prints” to standard output echo hello –hello echo A short line > newfile –A short line (in newfile) echo “A short line” > newfile –A short line (in newfile) hello=“Hi, there” –Hi, there echo \”hello\” –“hello” 2

3. 3 Echo (with variables and quotes) echo $hello – blank Define variable: hello=“Hi, there” echo $hello –Hi, there echo “$hello” –Hi, there echo \$hello –$hello echo ‘$hello’ –$hello 3

4. 4 grep Find Pattern in File(s) grep pattern file(s) grep -v pattern filename … grep ^pattern filename … grep pattern$ filename … grep “pattern with spaces” filename … fgrep -f patternfile filename … grep -i pattern filename … egrep “pat1|pat2|pat3” filename … 4

5. 5 cut To Get Pieces Of Each Line cut -c 5,6,7 file > newfile cut -c 5-7 file > newfile cut -c15-20,2-5,8-13 filename > newfile cut -f3 –d" " filename > newfile cut -f1,5,6 -d: /etc/passwd 5

6. 6 tr Examples To translate single characters to single characters tr abc xyz < filename a → x, b→y, c→z tr A-Z a-z –or tr ‘A-Z’ ‘a-z’ –or tr [A-Z] [a-z] translates upper case to lower case tr a-zA-Z n-za-mN-ZA-M < filename –"rot13: changes letters halfway up alphabet tr -s ‘ \t’ : –replace multiple spaces+tabs with single colon

7. 7 sed To Substitute in a File sed ‘s/string/newstring/g’ file > newfile sed ‘s?relative/path/name?new/dir/name?’ file > newfile sed ‘5,10s/string/newstring/’ file > newfile sed ‘/pat1/,/pat2/s/strg/newstrg/’ file > nf 7

8. 8 sed Delete Lines (You can also add lines, but >> is easier) sed '5,10d' file > newfile –Deletes lines 5 through 10 sed /isaacs/d file > newfile –Deletes all lines containing "isaacs" sed /here/,/there/d file > newfile –Finds first line containing "here", and deletes it and all lines up to one containing "there" 8

9. 9 uniq Eliminate Duplicate Lines uniq file > newfile eliminate duplicates (must be adjancent) sort file | uniq > newfile sort first uniq -c file | less counts identical lines 9

10. 10 sort sort file > newfile alphabetically from beginning of line sort -k3 file > newfile alphabetically from third field sort -k 3n file > newfile numeric order, third field sort -rn -k 3 file > newfile reverse numeric order ls -l | sort -r -k 5n | less show largest files) sort -t: -k1,1 -k5,5 /etc/passwd | less sort the password file, by account then name 10

11. 11 Comparing Files diff file1 file2 –reports differences between 2 files –form allows changing one to the other –5,6c7,8 means lines 5-6 n file1 match lines 7-8 in file2 –‘ ’ is line from file2 diff3 file1 file2 file3 compares 3 files cmp file1 file2 compares binary files by byte (and reports first different byte) 11

12. 12 pr Print Command pr filename format for printing pr filename | lpr actually print formatted file flags: +k start with page k -k k-column output -h make your own filename -t don’t print headers 12

13. 13 find find. -name filename -print –find a file with the name “filename” find ~isaacs -mtime +6 -print –find files in my home, older than 6 days find. -types f -exec ls -l {} \ ; –only do “ls” on regular files find. -type f | xargs ls -l 13

14. 14 awk This is an extra. I will not test on awk, but it may be useful in some of the quiz and homework questions. awk is more than a command, it is essentially a language. But if you want the full power, I would suggest learning and using Perl instead. Though harder to learn, it is much more powerful, and does all that awk does. 14

15. 15 awk definition AWK is a language for processing text files. A file is treated as a sequence of lines, called records. Each line is broken up into a sequence of fields, so we can think of the first word in a line as the first field, the second word as the second field, and so on. An AWK program is of a sequence of pattern-action statements. AWK reads the input a line at a time. A line is scanned for each pattern in the program, and for each pattern that matches, the associated action is executed. - Alfred V. Aho (one of the inventors of awk, quoted in Wikipedia, modified by me)

16. 16 How awk works. awk does actions (like arithmetic or printing) for lines specified by patterns, or before any lines are read ("BEGIN") or after all lines are read ("END) awk BEGIN {start-action} {action} END {stop-action} awk pattern1{action1} pattern2{action2} pattern2{action3} 16

17. 17 awk examples 1. awk '{print $1}' 2. awk '/isaacs/ {print}' 3. awk '/isaacs/' 4. awk '{print $8 " - " $5 " bytes"}' 5. awk 'BEGIN{sum=0} {sum=sum+$5} END {print sum}' 6. awk 'BEGIN {FS=":"} $5 ~ " - Fall13" {split ($5,xx,/,/);print xx[1]}' 17

18. 18 awk variables and actions NR - number of the current line NF - number of fields in current line FS - Field separator RS - Line (record) separator print - print a line length() - length of a string split(str,array,sep) - split string into an array tolower() - make a string lower case

19. 19 Some extra commands (Not in tests) column - make a list into columns –ls | grep somefiles | column -c 5 locate - easier to use then "find" –locate filename paste - join each line of two files together, one after the other –paste file1 file2 join - If 2 files have a field in common, combine the lines from both files if the fields match. (Must sort on the field.)

20. 20 Using vi To Create A File vi file-nameenter vi i (or a)go into insert mode – ESC(key)get out of insert mode ZZsave the file and exit (uppercase) 20

21. 21 vi Concepts Modes –command (ESC) –insert (i, a, o, etc) –ex mode (:) Screen, as a window in the text Where is the cursor? How to move around Search for a line (“/” and “?”) Exit, saving or not saving your changes 21

22. 22 vi modes Command mode –What you type is a command to vi –ESC will change you from insert to command Insert (or replace) mode –Whatever you type will go into your file –Various commands put you in insert mode ex mode –Commands to “ex” editor, start with “:” –On bottom line of screen 22

23. 23 ex mode s, d, undo, w, q :undo undo last vi operation :w write file (:w! to force) :q quit (:q! to force) :d delete current line –:5,10d delete lines 5 through 10 :s/old/new/g substitute new for old on current line –/xyz/s/old/new/g sub. new for old on all lines with “xyz” in them –:5,10s/old/new/ sub. new for 1st old in lines 5-10 23

24. 24 addressing for ex n,m (line n through line m).,$ (here to end) 1,. (line 1 to here).,.+9 (next 10 lines).-2,.+2 (5 lines: 2 before through 2nd after) /pat/ (next line containing “pat”) /pat1/,/pat2/ (lines from next containing “pat1” through one after containing “pat2”) ?pat? (previous line containing “pat”) 24

25. 25 Search for a line /abcsearch forward for pattern “abc” ?abcsearch backwards for pattern “abc” nrepeat search (in same direction) 25

26. 26 To exit vi ZZ save file and exit –DO NOT confuse with ^Z, which puts it in the background :wq write file and exit :w write new file :w! overwrite old file 26

27. 27 vi Inserting and Appending Inserting text commands. Cursor A Append at end Append after cursor aInsert before cursor i Open new line above Insert at beginning Open new line below O o I 27

28. 28 Viewing Text Within A File Basic VI COMMANDS: To enter VI type: vi filename To save a file type: :w To get out of VI type: ZZ INSERTING TEXT I - insert text before cursor o - insert text below current line a - insert text after cursor O - insert text above line To save a file type: :w To get out of VI type: ZZ 28

29. 29 vi Cursor Movements Cursor Movements Commands Try these,cursor,commands, in a file. 1G KOKO [] Cursor w E j + Return bhbh G l

30. 30 vi Screen Movement Basic VI COMMANDS: To enter VI type: vi filename To save a file type: :w To get out of VI type: ZZ INSERTING TEXT I - insert text before cursor o - insert text below current line a - insert text after cursor O - insert text above line To save a file type: :w To get out of VI type: ZZ ^b ^f H L M 30

31. 31 vi Deleting Words and Lines Deleting text in a file dw Delete word D or d$ Delete to end of line dd Delete line Cursor xDelete character at cursor

32. 32 emacs Concepts Commands use special keys: control (CTRL), escape (ESC) or meta or alt, function keys Style modes (but not like vi) Buffers (multiple windows into a file) Extensible (in Lisp) Unlimited undo Keyboard macros; key binding Files can be large (doesn’t read them into core) 32

33. 33 (emacs concepts) Automatic backups –Can set automatic saving email, directories, ftp, much more Menu control (instead of commands) Information bars 33

34. 34 Starting and Ending emacs emacs(puts you in scratch buffer) –C-x C-fto read in file emacs filename (edits “filename”) C-x C-ssave file and exit C-x ssave all files and exit C-c C-cexit (will ask if you want to) 34

35. 35 emacs help C-husual help key –F1 (function 1 key) try if C-h doesn’t work –C-h aapropos –C-h cwhat does a character do –C-h fwhat does a function do –C-h ttutorial –C-h bshow all key bindings –C-h ?show possible help questions 35

36. 36 Processes Basic unit of multi-processing ps to show your current processes ps -l to show more about them (l=long) ps -alwf shows all processes on system, in long format, child procs indented, 2 lines if necessary 36

37. 37 Shell Variables Name and Value Examples MyName=Isaacs HOME=isaacs fruit=apple MONTH=April fullname=“Stan Isaacs” 37

38. 38 To Use or See Shell Variables $ NAME to use it echo $NAME echo I am going to eat an $fruit IF fruit=apple drink=crab$fruit drink=${fruit}juice echo Fruit is $fruit, drink is $drink 38

39. 39 USES Environment Variables –to define your environment Local Variables –inside scripts –long names Parameters to shell scripts –pass arguments to shell procedures (a kind of program) 39

40. 40 Environment Variables Usually all caps Represent your personal environment Used by various UNIX programs and commands –HOME LPDEST –LOGNAME PRINTER –PATH TERM –EDITOR PS1 40

41. 41 Environment Variables env to see them all You can change some of them –frequently in startup scripts export so everybody can see them –sub-shells –scripts set shows local variables also 41

42. 42 Example d=/usr/isaacs/class/day1/exercises –cd $d –pr $d/ex3 42

43. 43 Summary of Variables Case sensitive No spaces around “=“ Quote if there are spaces in value Curly braces ({,}) to separate Usually character strings –num=100.0 is different from num=100) echo or env to see them export 43