UNIX commands Head More (press Q to exit) Cat – Example cat file – Example cat file1 file2 Grep – Grep –v ‘expression’ – Grep –A 1 ‘expression’ – Grep.

Slides:



Advertisements
Similar presentations
Unix Trix for Emprirical CL1 CSA405: Unix Trix for Empirical CL How to use Unix as a toolbox for NLP applications.
Advertisements

EMT 2390L Lecture 4 Dr. Reyes Reference: The Linux Command Line, W.E. Shotts.
A Guide to Unix Using Linux Fourth Edition
 *, ? And [ …] . Any single character  ^ beginning of a line  $ end of the line.
CIS 240 Introduction to UNIX Instructor: Sue Sampson.
Introduction to Unix – CS 21 Lecture 13. Lecture Overview Finding files and programs which whereis find xargs Putting it all together for some complex.
T UTORIAL OF U NIX C OMMAND & SHELL SCRIPT S 5027 Professor: Dr. Shu-Ching Chen TA: Samira Pouyanfar Spring 2015.
Grep, comm, and uniq. The grep Command The grep command allows a user to search for specific text inside a file. The grep command will find all occurrences.
© 2006 KDnuggets [16/Nov/2005:16:32: ] "GET /jobs/ HTTP/1.1" "
Using Unix Shell Scripts to Manage Large Data
Unix Files, IO Plumbing and Filters The file system and pathnames Files with more than one link Shell wildcards Characters special to the shell Pipes and.
Unix Filters Text processing utilities. Filters Filter commands – Unix commands that serve dual purposes: –standalone –used with other commands and pipes.
UNIX Filters.
CS 124/LINGUIST 180 From Languages to Information Unix for Poets (in 2014) Dan Jurafsky (From Chris Manning’s modification of Ken Church’s presentation)
Review for Exam 1 Exam 1 on June 24 CSC 3320.
1 Day 16 Sed and Awk. 2 Looking through output We already know what “grep” does. –It looks for something in a file. –Returns any line from the file that.
Advanced File Processing
– Command-Line Data Analysis and Reporting 9/4/ Command-Line Data Analysis and Reporting - Rediscovering the Prompt
Interpreting logs and reports IIPC GA 2014 Crawl engineers and operators workshop Bert Wendland/BnF.
System Programming Regular Expressions Regular Expressions
LIN 6932 Unix Lecture 6 Hana Filip. LIN 6932 HW6 - Part II solutions posted on my website see syllabus.
Introduction to Unix (CA263) File Processing. Guide to UNIX Using Linux, Third Edition 2 Objectives Explain UNIX and Linux file processing Use basic file.
Unix programming Term: III B.Tech II semester Unit-II PPT Slides Text Books: (1)unix the ultimate guide by Sumitabha Das (2)Advanced programming.
Review Please turn in your homework and practicals sed.
CS 403: Programming Languages Fall 2004 Department of Computer Science University of Alabama Joel Jones.
OPERATING SYSTEMS DESIGN UNIX BASICS & SHELL SCRIPTING.
Advanced File Processing. 2 Objectives Use the pipe operator to redirect the output of one command to another command Use the grep command to search for.
Chapter Five Advanced File Processing Guide To UNIX Using Linux Fourth Edition Chapter 5 Unix (34 slides)1 CTEC 110.
Chapter Five Advanced File Processing. 2 Objectives Use the pipe operator to redirect the output of one command to another command Use the grep command.
Module 6 – Redirections, Pipes and Power Tools.. STDin 0 STDout 1 STDerr 2 Redirections.
Awk Dr. Tran, Van Hoai Faculty of Computer Science and Engineering HCMC Uni. of Technology
I/O Redirection and Regular Expressions February 9 th, 2004 Class Meeting 4.
Introduction to Unix – CS 21 Lecture 12. Lecture Overview A few more bash programming tricks The here document Trapping signals in bash cut and tr sed.
Chapter 9 Basic File Processing. Displaying File Contents cat, cat w/append tac nl pr more less head tail.
Chapter Five Advanced File Processing. 2 Lesson A Selecting, Manipulating, and Formatting Information.
Getting the most out of the workshop Ask questions!!! Don’t sit next to someone you already know Work with someone with a different skillset and different.
I/O Redirection & Regular Expressions CS 2204 Class meeting 4 *Notes by Doug Bowman and other members of the CS faculty at Virginia Tech. Copyright
Advanced Text Processing. 222 Lecture Overview  Character manipulation commands cut, paste, tr  Line manipulation commands sort, uniq, diff  Regular.
– Command-Line Data Analysis and Reporting 12/7/ Command-Line Data Analysis and Reporting - Rediscovering the Prompt
CS 124/LINGUIST 180 From Languages to Information Unix for Poets (in 2013) Christopher Manning Stanford University.
16-Dec-15Advanced Programming Spring 2002 sed and awk Henning Schulzrinne Dept. of Computer Science Columbia University.
– Introduction to the Shell 1/21/2016 Introduction to the Shell – Session Introduction to the Shell – Session 3 · Job control · Start,
CS 124/LINGUIST 180 From Languages to Information
Xuan Guo Review 2 Xuan Guo CSC Xuan Guo Redirection Command Substitution Subshell Quoting Termination and Exit codes Exporting Variables Background.
1 Lecture 10 Introduction to AWK COP 3344 Introduction to UNIX.
Computing on TSCC Make a folder for the class and move into it –mkdir –p /oasis/tscc/scratch/username/biom262_harismendy –cd /oasis/tscc/scratch/username/biom262_harismendy.
ORAFACT Text Processing. ORAFACT Searching Inside Files grep - searches for patterns within files grep [options] [[-e] pattern] filename [...] -n shows.
Lesson 6-Using Utilities to Accomplish Complex Tasks.
In the last class, Filters and delimiters The sample database pr command head and tail commands cut and paste commands.
Filters and Utilities. Notes: This is a simple overview of the filtering capability Some of these commands are very powerful ▫Only showing some of the.
SIMPLE FILTERS. CONTENTS Filters – definition To format text – pr Pick lines from the beginning – head Pick lines from the end – tail Extract characters.
Introduction to Textutils sort, uniq, wc, cut, grep, sed, awk ● Steve Walsh ● Linux Users of Victoria ● November, 2007.
Linux 201 Training Module Linux Adv File Mgmt.
Tutorial of Unix Command & shell scriptS 5027
Lesson 5-Exploring Utilities
Using Unix Shell Scripts to Manage Large Data
regular expressions - grep
Advanced File Processing
Chapter 6 Filters.
Linux command line basics III: piping commands for text processing
Unix Scripting Session 4 March 27, 2008.
CS 403: Programming Languages
Tutorial of Unix Command & shell scriptS 5027
Tutorial of Unix Command & shell scriptS 5027
The Linux Command Line Chapter 6
Guide To UNIX Using Linux Third Edition
Tutorial of Unix Command & shell scriptS 5027
Unix Talk #2 grep/egrep/fgrep (maybe add more to this one….)
MeasureCamp VI *NIX for ETL
Software I: Utilities and Internals
Presentation transcript:

UNIX commands Head More (press Q to exit) Cat – Example cat file – Example cat file1 file2 Grep – Grep –v ‘expression’ – Grep –A 1 ‘expression’ – Grep –B 2 ‘expression’ – Example: grep –v ‘#’ file.txt to remove comments Expression metacharacters – $ end of line – $ beginning of line – [AB] A or B – * any character – Example: ‘CDKN*’ or ‘chr[1-7]’

UNIX commands Cut –cut –f 1 –cut –f 3 –d ‘:’ Sort –sort –n –sort –nr (or sort –n –r) –sort –k 2 uniq –uniq –uniq -c wc –wc –l file.txt –Example: cut –f 1 file | sort | uniq -c

UNIX commands Sed –Sed ‘s/foo/bar/g’ file : find and replace Awk –Awk ‘$3>2000’ file : select row with 3 rd field>2000 –Awk ‘{if ($3>2000) print $1,$2}’ file only print first 2 columns –Awk ‘{sum+=$3} END {print sum}’ file print sum of column 3 –Awk ‘{sum+=$3} END {print sum/NR}’ file print average of column 3 Join –join –j 1 sorted_file1 sorted_file2