BIF713 Additional Utilities. Linux Utilities  You have learned many Linux commands. Here are some more that you can use:  Data Manipulation (Reg Exps)

Slides:



Advertisements
Similar presentations
Jump to first page Unix Commands Monica Stoica Jump to first page Introduction to Unix n Unix was born in 1969 at Bell Laboratories, a research subdivision.
Advertisements

Working with Files How to create, view, copy, rename and print files.
Cosc 4750 Getting Started in UNIX Don’t be afraid of the prompt, in linux it can be your best friend. In some cases, the only way to do certain things.
ATS Programming Short Course I INTRODUCTORY CONCEPTS Tuesday, Jan. 27 th, 2009 Essential Unix Commands.
Linux+ Guide to Linux Certification, Second Edition
Guide to Linux Installation and Administration, 2e1 Chapter 6 Using the Shell and Text Files.
Lecture 02CS311 – Operating Systems 1 1 CS311 – Lecture 02 Outline UNIX/Linux features – Redirection – pipes – Terminating a command – Running program.
CSCI 330 T HE UNIX S YSTEM File operations. OPERATIONS ON REGULAR FILES 2 CSCI The UNIX System Create Edit Display Contents Display Contents Print.
UNIX Filters.
Shell Script Examples.
7/17/2009 rwjBROOKDALE COMMUNITY COLLEGE1 Unix Comp-145 C HAPTER 2.
Chapter 4: UNIX File Processing Input and Output.
BILKENT UNIVERSITY DEPARTMENT OF COMPUTER TECHNOLOGY AND INFORMATION SYSTEMS CTIS156 INFORMATION TECHNOLOGIES II CHAPTER 10: ADVANCED FILE PROCESSING.
COMP1070/2002/lec4/H.Melikian COMP1070 Lecture #5  Files and directories in UNIX  Various types of files  File attributes  Notion of pathname  Commands.
Advanced File Processing
1 THE UNIX FILE SYSTEM By Chokechai Chuensukanant ID COSC 513 Operating System.
Linux environment ● Graphical interface – X-window + window manager ● Text interface – terminal + shell.
3 File Processing Mauro Jaskelioff. Introduction More UNIX commands for handling files Regular Expressions and Searching files Redirection and pipes Bash.
Lesson 11-Locating, Printing, and Archiving User Files.
BIF703 Miscellaneous Commands. File related commands  grep - print lines matching a pattern  head - output the first part of files  tail - output the.
Agenda User Profile File (.profile) –Keyword Shell Variables Linux (Unix) filters –Purpose –Commands: grep, sort, awk cut, tr, wc, spell.
File Processing. Introduction More UNIX commands for handling files Regular Expressions and Searching files Redirection and pipes Bash facilities.
Unix Basics Chapter 4.
Unix Talk #2 (sed). 2 You have learned…  Regular expressions, grep, & egrep  grep & egrep are tools used to search for text in a file  AWK -- powerful.
Introduction to Unix (CA263) File Processing. Guide to UNIX Using Linux, Third Edition 2 Objectives Explain UNIX and Linux file processing Use basic file.
Unix programming Term: III B.Tech II semester Unit-II PPT Slides Text Books: (1)unix the ultimate guide by Sumitabha Das (2)Advanced programming.
Title Slide CSS 404/504 The UNIX Operating System (2) By Ralph B. Bisland, Jr.
The UNIX Shell. The Shell Program that constantly runs at terminal after a user has logged in. Prompts the user and waits for user input. Interprets command.
LIN Unix Lecture 3 Hana Filip. LIN UNIX Resources UNIX Tutorials UNIX help for.
Agenda Chapter 1: Linux (Unix) Features Commands (Chapters 2 & 3) Command Structure / Command line editing man, passwd, cal, date, whereis, which Working.
Jozef Goetz, expanded by Jozef Goetz, 2009 Credits: Parts of the slides are based on slides created by UNIX textbook authors, Syed M. Sarwar, Robert.
Linux+ Guide to Linux Certification Chapter Four Exploring Linux Filesystems.
Linux+ Guide to Linux Certification, Third Edition
The Shell Chapter 7. Overview The Command Line Standard IO Redirection Pipes Running a Program in the Background Killing (a process!)
1 Lecture 5 Additional useful commands COP 3353 Introduction to UNIX.
Additional UNIX Commands. 222 Lecture Overview  Multiple commands and job control  More useful UNIX utilities.
Week 3 Exploring Linux Filesystems. Objectives  Understand and navigate the Linux directory structure using relative and absolute pathnames  Describe.
Advanced File Processing. 2 Objectives Use the pipe operator to redirect the output of one command to another command Use the grep command to search for.
UNIX Shell Script (1) Dr. Tran, Van Hoai Faculty of Computer Science and Engineering HCMC Uni. of Technology
Chapter Five Advanced File Processing Guide To UNIX Using Linux Fourth Edition Chapter 5 Unix (34 slides)1 CTEC 110.
Chapter Five Advanced File Processing. 2 Objectives Use the pipe operator to redirect the output of one command to another command Use the grep command.
Course materials may not be reproduced in whole or in part without the prior written permission of IBM. 5.1 © Copyright IBM Corporation 2008 Unit 10 Linux.
Agenda Regular Expressions (Appendix A in Text) –Definition / Purpose –Commands that Use Regular Expressions –Using Regular Expressions –Using the Replacement.
Linux file system "On a UNIX system, everything is a file; if something is not a file, it is a process." Sorts of files (on a Linux system) Directories:
I/O Redirection and Regular Expressions February 9 th, 2004 Class Meeting 4.
Chapter Five Advanced File Processing. 2 Lesson A Selecting, Manipulating, and Formatting Information.
Agenda Basic Unix Commands (Chapters 2 & 3) Miscellaneous Commands: whereis, which, whoami, finger, passwd, cal, date Working with Files: cat, more, less.
I/O Redirection & Regular Expressions CS 2204 Class meeting 4 *Notes by Doug Bowman and other members of the CS faculty at Virginia Tech. Copyright
ULI101 More Linux Commands Introduction to UNIX/Linux and the Internet
Agenda Basic Unix Commands (Chapters 2 & 3) Miscellaneous Commands: which, passwd, date, ps / kill Working with Files: file, touch, cat, more, less, grep,
UNIX Commands RTFM: grep(1), egrep(1) & fgrep(1) Gilbert Detillieux April 13, 2010 MUUG Meeting.
Linux+ Guide to Linux Certification, Second Edition Chapter 4 Exploring Linux Filesystems.
A Brief Overview of Unix Brandon Bohrer. Topics What is Unix? – Quick introduction Documentation – Where to get it, how to use it Text Editors – Know.
Chapter Linux Basics. Acknowledgements This presentation was prepared by – Banyat Settapanich – Bahran Madaen This presentation will be updated later.
CSCI 330 UNIX and Network Programming Unit II Basic UNIX Usage: File System.
Lesson 6-Using Utilities to Accomplish Complex Tasks.
File Processing. Introduction More UNIX commands for handling files Regular Expressions and Searching files Redirection and pipes Bash facilities.
Linux Tutorial Lesson Two *Getting Help in Linux *Data movement and manipulation *Relative and Absolute path *Processes Note: see chapter 1,2,3 from Linux.
Filters and Utilities. Notes: This is a simple overview of the filtering capability Some of these commands are very powerful ▫Only showing some of the.
Lesson 5-Exploring Utilities
Linux Commands Help HANDS ON TRAINING Author: Muhammad Laique
Some Linux Commands.
Agenda Basic Unix Commands (Chapters 2 & 3) Miscellaneous Commands:
Basic UNIX OLC Training.
Guide To UNIX Using Linux Third Edition
Lecture 5 Additional useful commands COP 3353 Introduction to UNIX 1.
Unix Talk #2 (sed).
Linux Shell Script Programming
Lecture 5 Additional useful commands COP 3353 Introduction to UNIX 1.
Presentation transcript:

BIF713 Additional Utilities

Linux Utilities  You have learned many Linux commands. Here are some more that you can use:  Data Manipulation (Reg Exps)  Miscellaneous  Printer Commands 

Data Manipulation  grep - print lines matching a pattern  head - output the first part of files  tail - output the last part of files  sort - sort lines of text files  diff - find differences between two  files  file - determine file type

4 Regular Expressions  Definition: A Regular Expression refers to a set of one or more strings of characters. Regular expressions can be used with Linux / UNIX commands such as grep, egrep, man, more, less, vi.

5 Regular Expressions  Simple Regular Expressions A Simple Regular Expression is a considered to be a “simple” match using patterns involving all or portions of characters Here are some examples: grep –i “435” course_codes (Matches strings of text containing 435 such as “OPS435”) grep –i “there” file.txt (Matches strings of text containing pattern “there” such as “there”, “therefore”, “there they go again…”

6 Regular Expressions  Complex Regular Expressions A Complex Regular Expression uses special characters with letters and numbers in order to provide an ambiguous regular expression match. This is similar to File Name Expansion, except we are matching strings (not filename), and the rules for searching symbols differ…

7 Regular Expressions  Complex Regular Expression Symbols:. Matches any character *Matches one or more occurrences of that preceeding character ^Anchors search to beginning of character string (line) $Anchors search to end of character string (line) [.. ] Matches any single character that belongs in square brackets. For opposite, use the notation [^.. ]

8 Regular Expressions  Complex Regular Expressions - Examples Explain what the following Linux commands using complex regular expressions will do:  grep “^the” file.txt grep “the$” file.txt grep “^the$” file.txt grep “^[0-9]*” file.txt  grep “^[0-9]*$” file.txt grep “^[0-9].*$” file.txt

9 Regular Expressions  Complex Regular Expressions – Examples / Continued… Explain what the following Linux commands using complex regular expressions will do:  var=“I am hungry”; echo $var | grep “^[A-Z].*[a-z]” file.txt var=“I am hungry”; echo $var | grep “^[a-z].*[a-z]” file.txt var=“I like OPS435”; echo $var | grep “^[^0-9].*[0-9]” file.txt var=“OPS435”; echo $var | grep “..” file.txt  var=“OPS435”; echo $var | grep “^..$” file.txt var=“OPS435”; echo $var | grep “^……$” file.txt  var=“OPS435A”; echo $var | grep “^……$” file.txt

10 Regular Expressions  Extended Regular Expressions Symbols to include advanced (extended) features such as repetition, and grouping. Use egrep or grep –E when using grep with extended regular expressions. You can use complex and regular expressions together as long as you use grep command as egrep or grep –E…

11 Regular Expressions  Extended Regular Expression Symbols: {1,} One or more occurrences of preceding character {0,1} Zero or one occurrence of preceding character {2,5} Between 2 and 5 occurrences of preceding character (pattern){1,} One or more occurrences of “pattern” NOTE: There are some short-cut symbols: + = {1,} ? = {0,1}

12 Regular Expressions  Extended Regular Expressions - Examples Explain what the following Linux commands using extended regular expressions will do:  grep “^[+-]{0,1}[0-9]{1,}$” file.txt  egrep “^[+-]?[0-9]+$” file.txt  egrep “^[+-]?[0-9]?$” file.txt  grep -Ei “^(the){2,}” file.txt

13 Regular Expressions  Other Utilities (vi): Regular expressions can be used with the vi editor to make it easier to search for complex patterns that are contained in files. The commands to search and replace regular expressions are typed in “last-line mode”

14 Regular Expressions Examples (using vi within “last-line” mode): :s/pattern1/pattern2/ - substitutes pattern2 for the first occurrence of pattern1 :1,$ s/pattern1/pattern2/ - substitutes pattern2 for the first occurrence of pattern1 in each line of the file ($ means last line,. means current line) :.,.+10 s/pattern1/pattern2/ - substitutes 11 lines starting with current line :s/pattern1/ *** & ***/ - & is the value of the string matched by pattern1, and is the only character with special meaning in the second pattern

15 Regular Expressions  Other Utilities (man, more, less): Regular expressions can be used with the man, more and less utilities to navigate to a location that matches a pattern Examples:  /^The/  /ward$/  /^[0-9].*[0-9]$/  /therefore.*done/

Miscellaneous  who – show who is logged in  date – print or set the system date  and time  which – show the full path of (shell)  commands  finger – user information lookup program  mail – send and receive mail

Print commands  lpr – print files  lpq – show print queue status  lprm – cancel print jobs

grep  grep takes a pattern, read standard input or a list of files, and outputs the lines containing matches for the pattern. Example: grep foo * Print lines in any of the files in the current directory that contain the pattern “foo”.

grep options Major options for grep: -l display name of the file that has matching line -r search all the files in the current directory and all its subdirectory for the given pattern -n prefix each output with line number -w search for matching word -v output lines that do not contain the given pattern

grep examples  grep -r foo. Print all the lines in all the files in the current directory and all its subdirectories that contains the pattern “foo”.  grep -lr foo. Similar as above but only print the names of the files that contains the pattern “foo”

grep examples Print all the lines in the file “bar” that contains the pattern “foo” misc]$ grep foo bar This line contains the word foo and bar. Do you like to play football or basket ball? Same as above but prefix each line of output with the line number within the file “bar” misc]$ grep -n foo bar 3:This line contains the word foo and bar. 4:Do you like to play football or basket ball?

grep examples Print all the lines in the file “bar” that contains the word “foo” misc]$ grep -w foo bar This line contains the word foo and bar. Print all the lines in the file “bar” that does not contain the pattern “foo” misc]$ grep -v foo bar The name of this file is called bar. This file has only five line. This is the end of the file.

head & tail head bar Display the first 10 line of the file “bar” head -5 bar Display the first 5 lines of the file “bar” tail bar Display the last 10 lines of the file “bar” tail -5 bar Display the last 5 lines of the file “bar”

sort Sort line of text file cat numbers [root] sort numbers [root] sort -n numbers Numeric order String order

sort examples cat numbers sort numbers sort -k2 numbers Sort by the 1st field Sort by the 2 nd field

sort examples cat numbers sort numbers sort -r numbers Sort in reverse order

diff Display the differences between two files Syntax:  diff [options] file1 file2  When using “diff” without any options, it produces a series of lines containing: Add (a) Delete (d), and Change (c) instructions Each of these lines is followed by the lines from the file that you need to add, delete, or change to make the files the same.

diff examples cat file1 blue red white yellow orange cat file2 blue yellow black red orange $diff file1 file2 2,3d1 < red < white 4a3,4 > black > red 1. Delete line 2 through 3 from file1 2. Append lines 3 through 4 from file2 after line 4 in file1 Steps to convert file1 to file2:

file Displays the classification (type) of a file. Considered useful if a file extension is missing, or if user is unsure of file extension.  Syntax: file [option] file-list Examples: file assign01.html assign01.html: HTML document text file a.out a.out: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped file mydoc.doc mydoc.doc: Microsoft Office Document file 1 1: empty

who Shows which users are logged on to server Phobos: /home/rchan>$ who rchan pts/0 Oct 30 02:08 (toronto-hse-ppp3) sslui pts/1 Oct 30 01:11 (CPE00112f0fe590-) Phobos: /home/rchan>$ who -H Name Line Time Hostname rchan pts/0 Oct 30 02:08 (toronto-hse-ppp3) sslui pts/1 Oct 30 01:11 (CPE00112f0fe590-) Phobos: /home/rchan>$ who -qH Name Hostname rchan (toronto-hse-ppp3) sslui (CPE00112f0fe590-) Total users: 2

who options Major options for grep: -H displays column headings -T includes message reception status: + message reception on - message reception off -i includes column indicating number of minutes of user inactivity.

who examples Show who is logged on who -H Name Line Time Hostname root pts/0 Mar 02 09:11 ( ) msaul pts/1 Mar 02 09:21 (CPE0040f4df2fef-) who -i root pts/0 Mar 02 09:11 0: ( ) msaul pts/1 Mar 02 09: who -T root - pts/0 Mar 02 09:11 msaul + pts/1 Mar 02 09:21 Less than 1 minute inactivity Allows message reception Column headings

date Displays the system time and date week8]$ date Sun Oct 30 01:48:10 EST 2005 week8]$ date +"%D" 10/30/05 week8]$ date +"%T" 01:54:05 week8]$ date +"%D %T" 10/30/05 01:54:13  Refer to the man page for more formatting codes

which Shows the full path of (shell) commands week8]$ which mkdir /bin/mkdir week8]$ which type /usr/bin/which: no type in (/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/home/ray/bin)

finger User information lookup program Phobos: /home/rchan>$ finger rchan Login name: rchan In real life: Raymond Chan Directory: /home/rchan Shell: /usr/bin/ksh On since Oct 30 02:08:55 on pts/0 from toronto-hse-ppp3 (messages off) No Plan.

mail Send and receive mail To read your mail on phobos, type the “mail” command by itself: Phobos: /home/rchan>$ mail Mail [5.2 UCB] [AIX 4.1] Type ? for help. "/var/spool/mail/rchan": 1 message 1 new >N 1 rchan Wed Oct 26 00:24 10/340 "Mail testing"

Sending mail To send a file called “letter” through on phobos to the user “rchan”: Phobos: /home/rchan>$ mail -s “subject” rchan < letter

lpr, lpq, lprm  lpr – submit file for printing lpr [ -P printer-name ] [ -# copies ] file-name [ -P printer-name] : send files to the named printer [ -# copies ] :sets the number of copies to print between 1 and 100 file-name : name of file to be printed

printer queue status  lpq - show printer queue status lpq [ -P printer-name] [ -a ] [ -l] [ -P printer-name] : show status on the named printer [ -a ] : reports jobs on all printers [ -l ] : display more verbose (long) format

Cancel Print Jobs  lprm – cancel print jobs lprm [ - ] [ -P printer-name] [ job ID(s)] [ - ] : all print jobs [ -P printer-name] : print jobs on the named printer [ job ID(s) ]: jobs to be cancel