CIT 140: Introduction to ITSlide #1 CSC 140: Introduction to IT Advanced File Processing Part 2.

Slides:



Advertisements
Similar presentations
UNIX Chapter 10 Advanced File Processing Mr. Mohammad Smirat.
Advertisements

การใช้ระบบปฏิบัติการ UNIX พื้นฐาน บทที่ 4 File Manipulation วิบูลย์ วราสิทธิชัย นักวิชาการคอมพิวเตอร์ ศูนย์คอมพิวเตอร์ ม. สงขลานครินทร์ เวอร์ชั่น 1 วันที่
CSCI 330 T HE UNIX S YSTEM Regular Expressions. R EGULAR E XPRESSION A pattern of special characters used to match strings in a search Typically made.
The Unix File System. What are the three parts of every file on a Unix filesystem? And where is each stored? Filename - stored in directories Inode -
File Security. Viewing Permissions ls –l Permission Values.
Linux+ Guide to Linux Certification, Second Edition
Linux Linux File System.
CS 497C – Introduction to UNIX Lecture 16: - File Attributes Chin-Chih Chang
Guide To UNIX Using Linux Third Edition
T UTORIAL OF U NIX C OMMAND & SHELL SCRIPT S 5027 Professor: Dr. Shu-Ching Chen TA: Samira Pouyanfar Spring 2015.
UNIX Files and Security Software Tools. Slide 2 File Systems l What is a file system? A means of organizing information on the computer. A file system.
Links Software Tools. Lecture 4 / Slide 2 Links l A link is a pointer to a file. l In fact, in UNIX all filenames are just links to a file. Most files.
Unix Filters Text processing utilities. Filters Filter commands – Unix commands that serve dual purposes: –standalone –used with other commands and pipes.
UNIX Filters.
Shell Script Examples.
Chapter 4: UNIX File Processing Input and Output.
BILKENT UNIVERSITY DEPARTMENT OF COMPUTER TECHNOLOGY AND INFORMATION SYSTEMS CTIS156 INFORMATION TECHNOLOGIES II CHAPTER 10: ADVANCED FILE PROCESSING.
Adv. UNIX: Filters/41 Advanced UNIX v Objectives –to discuss five useful filters: tr, grep, awk, sed, and find Special Topics in Comp.
Filesystem Hierarchy Standard (FHS) –Standard of outlining the location of set files and directories on a Linux system –Gives Linux software developers.
System Programming Regular Expressions Regular Expressions
June 1, 1999Manipulating Files1 Introduction to UNIX E. Manipulating Files.
Agenda User Profile File (.profile) –Keyword Shell Variables Linux (Unix) filters –Purpose –Commands: grep, sort, awk cut, tr, wc, spell.
Chapter Four UNIX File Processing. 2 Lesson A Extracting Information from Files.
CIT 140: Introduction to ITSlide #1 CSC 140: Introduction to IT I/O Redirection.
Guide To UNIX Using Linux Fourth Edition
LIN 6932 Unix Lecture 6 Hana Filip. LIN 6932 HW6 - Part II solutions posted on my website see syllabus.
Unix Talk #2 (sed). 2 You have learned…  Regular expressions, grep, & egrep  grep & egrep are tools used to search for text in a file  AWK -- powerful.
Introduction to Unix (CA263) File Processing. Guide to UNIX Using Linux, Third Edition 2 Objectives Explain UNIX and Linux file processing Use basic file.
Unix programming Term: III B.Tech II semester Unit-II PPT Slides Text Books: (1)unix the ultimate guide by Sumitabha Das (2)Advanced programming.
CIT 500: IT Fundamentals Text Processing 1. Topics 1.Displaying files: cat, less, od, head, tail 2.Creating and appending 3.Concatenating files 4.Comparing.
Regular expressions Used by several different UNIX commands, including ed, sed, awk, grep A period ‘.’ matches any single characters.X. matches any X.
CS 403: Programming Languages Fall 2004 Department of Computer Science University of Alabama Joel Jones.
System Administration Introduction to Unix Session 2 – Fri 02 Nov 2007 Reference:  chapter 1, The Unix Programming Environment, Kernighan & Pike, ISBN.
Week 3 Exploring Linux Filesystems. Objectives  Understand and navigate the Linux directory structure using relative and absolute pathnames  Describe.
UNIX Shell Script (1) Dr. Tran, Van Hoai Faculty of Computer Science and Engineering HCMC Uni. of Technology
Scripting Languages Course 2 Diana Trandab ă ț Master in Computational Linguistics - 1 st year
Lesson 9-Setting and Using Permissions. Overview Describing file permissions. Using execute permissions with a file. Changing file permissions using mnemonics.
Module 6 – Redirections, Pipes and Power Tools.. STDin 0 STDout 1 STDerr 2 Redirections.
Course materials may not be reproduced in whole or in part without the prior written permission of IBM. 5.1 © Copyright IBM Corporation 2008 Unit 10 Linux.
Introduction to Unix – CS 21 Lecture 6. Lecture Overview Homework questions More on wildcards Regular expressions Using grep Quiz #1.
BIF713 Additional Utilities. Linux Utilities  You have learned many Linux commands. Here are some more that you can use:  Data Manipulation (Reg Exps)
Linux file system "On a UNIX system, everything is a file; if something is not a file, it is a process." Sorts of files (on a Linux system) Directories:
Introduction to Unix – CS 21 Lecture 12. Lecture Overview A few more bash programming tricks The here document Trapping signals in bash cut and tr sed.
Chapter Five Advanced File Processing. 2 Lesson A Selecting, Manipulating, and Formatting Information.
Chapter Four I/O Redirection1 System Programming Shell Operators.
UNIX An Introduction. Brief History UNIX UNIX Created at Bell Labs, 1969 Created at Bell Labs, 1969 BSD during mid 70s BSD during mid 70s AT&T began offering.
Introduction to Linux Instructor: Bennett M. Tanyag PART – 3 Unit 1.
CIT 140: Introduction to ITSlide #1 CIT 140: Introduction to IT Shell Programming.
1 Lecture 9 Shell Programming – Command substitution Regular expressions and grep Use of exit, for loop and expr commands COP 3353 Introduction to UNIX.
By Corey Stokes 9/14/10. What is grep? Global Regular Expression Print grep is a command line search utility in Unix Try: Search for a word in a.cpp file.
BASH – Text Processing Utilities Erick, Joan © Sekolah Tinggi Teknik Surabaya 1.
Linux Commands C151 Multi-User Operating Systems.
CIT 140: Introduction to ITSlide #1 CSC 140: Introduction to IT Advanced File Processing.
Why UNIX? In the 1980s, UNIX became popular In the 1980s, UNIX became popular Customer demand for open systems: Customer demand for open systems: Application.
Linux+ Guide to Linux Certification, Second Edition Chapter 4 Exploring Linux Filesystems.
A Brief Overview of Unix Brandon Bohrer. Topics What is Unix? – Quick introduction Documentation – Where to get it, how to use it Text Editors – Know.
CIT 140: Introduction to ITSlide #1 CSC 140: Introduction to IT File Processing.
ORAFACT Text Processing. ORAFACT Searching Inside Files grep - searches for patterns within files grep [options] [[-e] pattern] filename [...] -n shows.
Filters and Utilities. Notes: This is a simple overview of the filtering capability Some of these commands are very powerful ▫Only showing some of the.
Advanced File Processing
The UNIX Shell Learning Objectives:
C151 Multi-User Operating Systems
Chapter 6 Filters.
CS 403: Programming Languages
INTRODUCTION TO UNIX: The Shell Command Interface
Tutorial of Unix Command & shell scriptS 5027
Tutorial of Unix Command & shell scriptS 5027
Tutorial of Unix Command & shell scriptS 5027
Chapter Four UNIX File Processing.
Software I: Utilities and Internals
Presentation transcript:

CIT 140: Introduction to ITSlide #1 CSC 140: Introduction to IT Advanced File Processing Part 2

CIT 140: Introduction to ITSlide #2 Topics 1.Searching for files: find 2.Searching files: grep and egrep 3.Cutting and pasting: cut and paste 4.Encryption and decryption: tr and crypt

CIT 140: Introduction to ITSlide #3 Searching for Files: find find path expression Search for files that match expression under path. Example expressions: -atime +xWas file accessed > x days ago? -atime –xWas file accessed < x days ago? -mtime +/- xWas file modified >/< x days ago? -name xDoes filename match x? -perm xDoes file match permissions x? -size +/-xIs file larger/smaller than x blocks? -type cIs file a regular file | link | directory? -user nDoes file belong to user n?

CIT 140: Introduction to ITSlide #4 Basic Find Options > find /usr/include -name stdio.h /usr/include/stdio.h > find /usr/include -name stdio.h -ls rw-r--r-- 1 root bin Apr /usr/include/stdio.h > find /usr/include -type l /usr/include/iso/assert_iso.h /usr/include/iso/errno_iso.h... /usr/include/uil /usr/include/xil > find /usr/include -type d /usr/include /usr/include/xfn...

CIT 140: Introduction to ITSlide #5 More Find Options > find /usr/include -mtime -7 > find /usr/include -mtime +7 /usr/include... /usr/include/xfn/FN_bindingset.h > find /usr/include -perm u=rw,g=rw,o=r > find /usr/include -perm u=rw,g=r,o=r /usr/include/zconf.h /usr/include/zlib.h... > find /usr -name '*.h' find: cannot read dir /usr/lost+found: Permission denied /usr/openwin/share/include/X11/DPS/ColorSB.h /usr/openwin/share/include/X11/DPS/ColorSBP.h...

CIT 140: Introduction to ITSlide #6 Combining find Expressions > find /usr/include -perm u=rw,g=r,o=r -user waldenj > find /usr/include -perm u=rw,g=r,o=r -user root /usr/include/zconf.h... /usr/include/xfn/FN_compound_name.h > find /kernel -type f -user root -size ls rwxr-xr-x 2 root sys May /kernel/drv/ip rwxr-xr-x 2 root sys May /kernel/drv/sparcv9/ip rwxr-xr-x 2 root sys May /kernel/strmod/ip rwxr-xr-x 2 root sys May /kernel/strmod/sparcv9/ip rwxr-xr-x 1 root sys May /kernel/genunix

CIT 140: Introduction to ITSlide #7 Combining find Expressions with -o > find /usr \( -name gnome -o -name dict \) -ls drwxr-xr-x 3 root other 512 Dec /usr/demo/gnome drwxr-xr-x 2 root bin 512 Dec /usr/share/lib/dict drwxr-xr-x 33 root other 1024 Dec /usr/share/gnome drwxr-xr-x 4 root other 1536 Dec /usr/share/gnome/pixmaps/nautilus/gnome lrwxrwxrwx 1 root root 16 Dec /usr/dict ->./share/lib/dict drwxr-xr-x 3 root other 512 Dec /usr/gnome rw-r--r-- 1 bin bin 1480 May /usr/local/share/terminfo/g/gnome

CIT 140: Introduction to ITSlide #8 Searching Files: grep grep [-i] [-l] [-n] [-v] pattern file1 [file2,...] Search for pattern in the file arguments. -iIgnore case of letters in files. -lPrint only the names of files that contain matches. -nPrint line numbers along with matching lines. -vPrint only nonmatching lines.

CIT 140: Introduction to ITSlide #9 Simple Searches > grep catt /usr/dict/words backscatter... wildcatter > grep -c catt /usr/dict/words 8 > grep -c -v catt /usr/dict/words > wc -l /usr/dict/words /usr/dict/words > grep -n catt /usr/dict/words 1814:backscatter :wildcatter > grep -l catt /usr/include/*.h /usr/include/exec_attr.h /usr/include/xti.h

CIT 140: Introduction to ITSlide #10 Regular Expressions ^Beginning of line $End of line [a-z]Character range (all lower case) [0-9]Character range (digits) [aeiou]Character range (vowels).Any character *Zero or more of previous pattern a|bMatch a or b

CIT 140: Introduction to ITSlide #11 Regular Expression Searches > egrep '^dogg' /usr/dict/words dogging doggone > egrep 'dogg$' /usr/dict/words > egrep 'mann$' /usr/dict/words Boltzmann Hermann Neumann Riemann Schumann > egrep '^mann' /usr/dict/words manna mannequin mannerism

CIT 140: Introduction to ITSlide #12 Regular Expression Searches > egrep 'catt|dogg' /usr/dict/words backscatter cattail... dogging doggone... wildcatter > egrep '^(catt|dogg)‘ /usr/dict/words cattail cattle cattleman cattlemen dogging doggone

CIT 140: Introduction to ITSlide #13 Character classes > grep '[0-9]' /usr/dict/words 10th 1st 2nd 3rd 4th 5th 6th 7th 8th 9th > grep -c '^[xz]' /usr/dict/words 49 > grep -c '[xz]$' /usr/dict/words 196

CIT 140: Introduction to ITSlide #14 How many words words have 4 sequential vowels? > grep '[aeiou][aeiou][aeiou][aeiou]' /usr/dict/words aqueous Hawaiian obsequious onomatopoeia pharmacopoeia prosopopoeia queue Sequoia

CIT 140: Introduction to ITSlide #15 Cutting Text: cut cut [-d char] [-f #] [-s] file1 [file2,...] Print specified fields of files. -c#1-#2Print characters #1 through #2 of each line. -d charUse specified character as field separator. (Tab is default separator.) -f #Print field # -f #1,#2Print field #1 and field #2 -f #1-#2Print all fields #1 through #2 -sDon’t print lines that don’t contain field separator.

CIT 140: Introduction to ITSlide #16 Cutting with space-separated fields > who longa pts/13 Oct 3 11:11 ( ) waldenj pts/16 Oct 8 16:23 ( ) trutat1 pts/14 Oct 8 16:51 ( ) > who | cut -c1-8 longa waldenj trutat1 > who | cut -c1-14 longa pts waldenj pts trutat1 pts > who | cut -d " " -f1 longa waldenj trutat1

CIT 140: Introduction to ITSlide #17 Cutting :-separated fields > tail /etc/passwd | cut -d: -f1,6 cribbett:/export/home7/cribbett matracia:/export/home7/matracia muellerm:/export/home7/muellerm nadaudm:/export/home7/nadaudm kohuss:/export/home7/kohuss cit1403:/export/home7/cit1403 stevenso:/export/home7/stevenso mousem:/export/home7/mousem krugd:/export/home1/krugd frankc:/export/home7/frankc

CIT 140: Introduction to ITSlide #18 Pasting Text: paste paste [-d char] [-f #] [-s] file1 [file2,...] Horizontally concatenate files. -d charUse specified character as field separator. (Tab is default separator.) -f #Print field # -f #1,#2Print field #1 and field #2 -f #1-#2Print all fields #1 through #2 -sDon’t print lines that don’t contain field separator.

CIT 140: Introduction to ITSlide #19 Cutting and Pasting > who >who.txt > cut -d " " -f1 who.txt >users.txt > cat users.txt longa waldenj trutat1 > cut -c25-31 who.txt >cut.txt > cat cut.txt Oct 3 Oct 8 > paste users.txt cut.txt longa Oct 3 waldenj Oct 8 trutat1 Oct 8

CIT 140: Introduction to ITSlide #20 Encryption Encryption is the process of encoding a message so that its meaning is not obvious. Decryption is the reverse process, transforming an encrypted message back into its original form.

CIT 140: Introduction to ITSlide #21 ROT13 ROT13 (“ROTate by 13 places”) is a simple Caesar cipher used for obscuring text by replacing each letter with the letter thirteen places down the alphabet. Not secure, but used to obscure quiz or puzzle answers so you don’t accidentally read them.

CIT 140: Introduction to ITSlide #22 ROT13 in UNIX > tr A-Za-z N-ZA-Mn-za-m rot13.txt > cat rot13.txt Fhaqnl Zbaqnl Ghrfqnl Jrqarfqnl Guhefqnl Sevqnl Fngheqnl > tr A-Za-z N-ZA-Mn-za-m <rot13.txt Sunday Monday Tuesday Wednesday Thursday Friday Saturday

CIT 140: Introduction to ITSlide #23 Encrypting Files: crypt crypt key ciphertext.txt Encrypt the file plaintext.txt using the specified key and store the result in ciphertext.txt. crypt key <ciphertext.txt Decrypt the file ciphertext.txt using the specified key, which must be identical to the key used to originally encrypt the file.

CIT 140: Introduction to ITSlide #24 Example of crypt > crypt hsk45$ crypt.txt > cat crypt.txt ~¿RNiÊV·wVæT®Ýz}ðqpà¾r´6 àÐÚ©ÖÓ'ÔÙóéö`­'îõkóòÇ > crypt hsk45$ <crypt.txt Sunday Monday Tuesday Wednesday Thursday Friday Saturday > crypt foobar <crypt.txt º´­Z Æ'98Åéïkai£7`Íôãâ^w²ª®9Z*A53=QeúSU9Ð _¾ø6`!>ü