1. Basic Text Processing, Redirection and Pipes

2. 222 Lecture Overview  Basic text processing commands head, tail, wc  Redirection and pipes  Getting to know vi  Basic vi commands  Advanced vi commands

3. 333 Text Processing Commands  UNIX is strongly text oriented  It contains many utilities to search, modify or view text files  This provides a simple mechanism for storing data and for passing it between applications and utilities

4. 444 Viewing Portions of a File – head and tail  The head command shows only the first few lines of a file  With no options, head shows first 10 lines  Command line options for head : -N – specify number of lines to show ( N is a number, not the letter N ) -cN – show first N bytes, not lines head [options] [files]

5. 555 Viewing Portions of a File – head and tail  The tail command is similar to head, but shows the last lines of the file  In addition, tail provides these options: +N – show lines starting with line N -f – output appended data as the file grows. With this option, the number of lines printed keeps growing as the file grows

6. 666 head and tail – Examples  Given a file called ' my_phones.txt ': ADAMS, Andrew 7583 BARRETT, Bruce 6466 BAYES, Ryan 6585 BECK, Bill 6346 BENNETT, Peter 7456 GRAHAM, Linda 6141 HARMER, Peter 7484 MAKORTOFF, Peter 7328 MEASDAY, David 6494 NAKAMURA, Satoshi 6453 REEVE, Shirley 7391 ROSNER, David 6830

7. 777 head and tail – Examples head -3 my_phones.txt ADAMS, Andrew 7583 BARRETT, Bruce 6466 BAYES, Ryan 6585 tail -3 my_phones.txt NAKAMURA, Satoshi 6453 REEVE, Shirley 7391 ROSNER, David 6830 head –c30 my_phones.txt ADAMS, Andrew 7583 BARRETT, Br

8. 888 head and tail – Examples tail +9 my_phones.txt MEASDAY, David 6494 NAKAMURA, Satoshi 6453 REEVE, Shirley 7391 ROSNER, David 6830 tail –c50 my_phones.txt toshi 6453 REEVE, Shirley 7391 ROSNER, David 6830

9. 999 Counting Words – wc  The wc command counts the number of bytes, words and lines in a file  Command line options for wc : -c – print the number of bytes -w – print the number of words -l – print the number of lines  With no options, all three are printed wc [options] [files]

10. 10 Lecture Overview  Basic text processing commands head, tail, wc  Redirection and pipes  Getting to know vi  Basic vi commands  Advanced vi commands

11. 11 Input and Output Redirection  Normally, when you run a command, it gets input from the keyboard ( stdin ), and sends output to the screen ( stdout ) Errors are sent to a third stream ( stderr ), which by default also points to the screen  Input and output (as well as errors) can be read from or written to a file, by using the redirection meta-characters: ' '

12. 12 Output Redirection  We use the ' > ' operator to redirect the output of a command  For example, to create a file containing a listing of '.c ' files in the current directory: ls *.c > c_files.txt cat !$ boxes.c parse_tools.c shape.c tools.c

13. 13 Input Redirection  We use the ' < ' operator to redirect the input of a command  For example, to see only the last two lines in the file ' c_files.txt ': tail -2 < c_files.txt shape.c tools.c

14. 14 Input / Output Redirection  Input and output redirection can be combined in a single command  For example: cat -n numbered.txt head -4 !$ head -4 numbered.txt 1 ADAMS, Andrew 7583 2 BARRETT, Bruce 6466 3 BAYES, Ryan 6585 4 BECK, Bill 6346

15. 15 Appending Output  When redirecting output to a file, the specified file is either created (if it does not exist), or overwritten (if it exists)  The ' >> ' operator tells the shell to append the output to the file, without overwriting it  To redirect stderr in addition to stdout, we use the operator ' >& '

16. 16 Appending Output – Example echo first line > lines.txt echo second line >> lines.txt echo third line >> lines.txt cat lines.txt first line second line third line echo fourth line > lines.txt cat lines.txt fourth line

17. 17 Forced Redirection  Output redirection fails if we try to: Overwrite an existing file Append to a non-existent file  This behavior can be overridden by using the ' ! ' modifier  When ' ! ' is added to a redirection operator, the redirection is forced to succeed

18. 18 Forced Redirection – Examples  The following will create the file ' lines.txt ' if it does not exist, or overwrite it if it does:  This will also create ' lines.txt ' if it does not exist, but append to it if it does echo first line >! lines.txt echo first line >>! lines.txt

19. 19 Pipes  The following commands count how many '.c ' files are in the current directory  This is not very efficient, and also leaves behind a temporary file as a side effect  What we really want is to link the output of ls directly to the input of wc ls *.c > c_files.txt wc -l < c_files.txt

20. 20 Pipes  A pipe takes the output of one command, and passes it as input to another Program 1 (ls *.c) Program 1 (ls *.c) Program 2 (wc –l) Program 2 (wc –l) stdout (or temp file) stdin (or temp file) Pipe

21. 21 Pipes  The symbol for a pipe is a vertical bar – ' | '  The format of a command line which uses a pipe is:  Now the two commands from the previous example can be simply combined: command1 [args] | command2 [args] ls *.c | wc -l

22. 22 Pipes  The use of pipes is actually a simple and easy form of inter-process communication  Pipes can also be chained, so complex utilities can be created by combining simple building blocks  When data passes through several pipes, these are sometimes referred to as filters

23. 23 Pipes – Examples  Print lines 6 and 7 of ' my_phones.txt ':  Another way to get the same results:  To find the most recently changed file: head -7 my_phones.txt | tail -2 GRAHAM, Linda 6141 HARMER, Peter 7484 tail +6 my_phones.txt | head -2 ls -t | head -1

24. 24 Lecture Overview  Basic text processing commands head, tail, wc  Redirection and pipes  Getting to know vi  Basic vi commands  Advanced vi commands

25. 25 Editing Files With vi  vi is the standard text editor in UNIX  Unlike most editors or word processors that you have encountered, it is modal – the same input produces different results in different modes Insert mode is for inserting text Command mode is for everything else

26. 26 Editing Files With vi  vi has hundreds of commands, whose names require memorization  It does not have menus or a GUI  All of these characteristics make vi : Very uncomfortable for beginners, but – Very powerful for experts

27. 27 Starting With vi  To edit a file, type:  You are now in command mode, so you cannot insert any text!  To enter insert mode, type: ' i '  Now you can type in text as with any other editor or word processor vi [options] [file]

28. 28 Basic vi Commands  To switch back to command mode, press Escape (this has no effect if you are already in command mode)  In command mode, you can move around using the arrow keys, or these keys: ' h ', ' j ', ' k ', ' l '  To delete a character, type ' x '

29. 29 Terminating a vi Session  To exit vi, you need to switch to a third mode, called last line mode, by typing ' : '  Then, type one of the following: w – to save your changes q – to quit an unmodified file wq – to save your changes and quit q! – to quit without saving your changes

30. 30 Learning vi  The best way to learn vi is: Start using it with a few basic commands Use available help sources to learn more features as you need them  Getting help: man vi Typing ' :help ' within vi UNIX books, the internet, etc.

31. 31 Lecture Overview  Basic text processing commands head, tail, wc  Redirection and pipes  Getting to know vi  Basic vi commands  Advanced vi commands

32. 32 vi Command Types  And now for a more detailed look  vi commands can be divided into groups: Commands for entering insert mode Commands for moving the cursor Commands for deleting or changing text Commands for searching or replacing text Miscellaneous commands

33. 33 Ways to Enter Insert Mode CommandDescription i Insert before cursor I Insert at start of line a Append after cursor A Append to end of line o Open a line below current line O Open a line above current line

34. 34 Moving the Cursor CommandDescription h, j, k, l Left, down, up, right Arrow keys Left, down, up, right w Forward to start of next word b Backward to start of previous word Ctrl-F Forward one screen Ctrl-B Backward one screen

35. 35 Moving the Cursor CommandDescription Ctrl-U Forward half a screen Ctrl-D Backward half a screen Ctrl-E Shift text down one line in window (cursor stays on same line) Ctrl-Y Shift text up one line in window (cursor stays on same line) % Move to the "mate" of this parenthesis or bracket

36. 36 Moving the Cursor CommandDescription $ Move to end of current line 0 Move to start of current line ^ Move to first non-whitespace character in current line nG, :n Go to line n gg Go to start of file G Go to end of file

37. 37 Deleting Text CommandDescription x Delete character under cursor X Delete character before cursor dd Delete entire line dw Delete current word D Delete until end of line

38. 38 Changing Text CommandDescription cc Change entire line cw Change current word C Change until end of line r Replace character under cursor R Start overwriting until ESC s Change one character and insert

39. 39 Copy and Paste CommandDescription yy Yank (copy) current line yw Yank current word p Put after (or below) cursor P (capital) Put before (or above) cursor ]p Put, with indent adjusted

40. 40 Lecture Overview  Basic text processing commands head, tail, wc  Redirection and pipes  Getting to know vi  Basic vi commands  Advanced vi commands

41. 41 Defining Range Using Motion  With some commands, it is possible to add a motion modifier which defines the range that they effect  The general format is: where motion is any of the available motion commands (e.g. d$ ) command[motion]

42. 42 Defining Range Using Motion  In the command dw, d is the command (delete), and w is the motion (word) dd (and similarly, cc ) is a special case – operates on whole lines  Examples: c$ Change until end of line dG Delete until end of file y% Yank block (from current to pair bracket)

43. 43 Repeating Commands  Most vi commands can be repeated, using the following format:  repeat defines how many times the given command should be executed  repeat and motion can be combined in a single command (e.g. 3dh ) [repeat]command

44. 44 Combining Repetition and Motion – Examples  Note: In 4yw, 4 is the repetition; in y4w, it is part of the motion 5dd Delete 5 lines, starting from current 3x Delete 3 characters, starting from current 2w Move forward 2 words 3dw Delete 3 words, starting from current 4yw Yank 4 words, starting from current y4w Same as above

45. 45 Searching for Patterns  pattern can also be a regular expression CommandDescription /pattern Search forward ?pattern Search backward n Repeat previous search N Repeat previous search in reverse direction

46. 46 Replacing Text  The following command can be used to search and replace a pattern  This searches from the current position  The g flag defines whether to replace only the first occurrence, or all matches  pattern can also be a regular expression :s/search_string/replace_string/[g]

47. 47 Additional Commands CommandDescription. Repeat previous command u Undo last command U Undo all changes on current line Ctrl-R Redo last undo ~ Switch case and advance cursor J Merge current and next lines

48. 48 Additional Commands CommandDescription >> Indent line one tab-stop << Un-indent line one tab-stop '' Go back to position before last move mama Mark current position as a 'a'a Jump to position marked as a

49. 49 Commands in Last Line Mode CommandDescription :r file Insert file in cursor position :w file Write current file with new name :f Display details of current file :!command Run a shell command :r !command Run command, and insert result :set var Set a vi environment variable

50. 50 Last Line Mode – Examples :!pwd Show current directory :!ls List contents of current directory :r !date Insert current date in cursor position :set nu Display line numbers :set nonu Do not display line numbers :set ai Auto-indent new lines according to indentation of previous ones :set noai Do not auto-indent new lines

51. 51 The vi Startup File –.exrc  vi provides various customization options  Whenever vi is started, it first reads a file named.exrc from your home directory  This file may contain any last line mode command, such as: set – to set vi environment variables map – to define new behavior for keys

52. 52 A Sample.exrc File set autoindent set number set tabstop=4 set shiftwidth=4 set expandtab set showmatch map ^\ :r ~/template/header.txt^M map ^_ :r !date +\%D^M map #6 :r ~/template/header.txt^M/Date^M:r !date +"\%b \%d, \%Y"^MkJjjj0 map #9 :w!%^M:!chmod u+x %^M:!%^M