Uniq command 6/12/2015Gary DeRoest1 report or filter out repeated lines in a file Note: the file needs to be sorted so that repeated lines are adjacent.

Slides:



Advertisements
Similar presentations
Decision Maths 1 Sorting Algorithms Bubble Sort A V Ali : 1.Start at the beginning of the data set. 2.Compare the first two elements,
Advertisements

Topological Sort and Hashing
การใช้ระบบปฏิบัติการ UNIX พื้นฐาน บทที่ 4 File Manipulation วิบูลย์ วราสิทธิชัย นักวิชาการคอมพิวเตอร์ ศูนย์คอมพิวเตอร์ ม. สงขลานครินทร์ เวอร์ชั่น 1 วันที่
Creating a new ASCII file Output usually goes to screen Output can be redirected to a file using the output redirection operator > (greater than symbol)
CIS 240 Introduction to UNIX Instructor: Sue Sampson.
 *, ? And [ …] . Any single character  ^ beginning of a line  $ end of the line.
CS 497C – Introduction to UNIX Lecture 25: - Simple Filters Chin-Chih Chang
Unix Utilities (sort/uniq) CS465 – Unix. The sort command Sorts lines Default behavior: Do a case-sensitive, ascii- alphabetic line sort, starting at.
Selection Sort
Grep, comm, and uniq. The grep Command The grep command allows a user to search for specific text inside a file. The grep command will find all occurrences.
Introduction to Unix – CS 21 Lecture 5. Lecture Overview Lab Review Useful commands that will illustrate today’s lecture Streams of input and output File.
Unix Filters Text processing utilities. Filters Filter commands – Unix commands that serve dual purposes: –standalone –used with other commands and pipes.
UNIX Filters.
2 $ command Command Line Options ls –a –l hello hi Command Arguments.
1 Day 16 Sed and Awk. 2 Looking through output We already know what “grep” does. –It looks for something in a file. –Returns any line from the file that.
LIN 6932 Unix Lecture 6 Hana Filip. LIN 6932 HW6 - Part II solutions posted on my website see syllabus.
Jozef Goetz, expanded by Jozef Goetz, 2009 Credits: Parts of the slides are based on slides created by UNIX textbook authors, Syed M. Sarwar, Robert.
CS 403: Programming Languages Lecture 21 Fall 2003 Department of Computer Science University of Alabama Joel Jones.
CS 403: Programming Languages Fall 2004 Department of Computer Science University of Alabama Joel Jones.
Chapter Five Advanced File Processing. 2 Objectives Use the pipe operator to redirect the output of one command to another command Use the grep command.
Module 6 – Redirections, Pipes and Power Tools.. STDin 0 STDout 1 STDerr 2 Redirections.
Java™ How to Program, 9/e © Copyright by Pearson Education, Inc. All Rights Reserved.
Pipes and Filters Copyright © Software Carpentry 2010 This work is licensed under the Creative Commons Attribution License See
Chapter 9 Basic File Processing. Displaying File Contents cat, cat w/append tac nl pr more less head tail.
Pipes & Filters Architecture Pattern Source: Pattern-Oriented Software Architecture, Vol. 1, Buschmann, et al.
Chapter Four I/O Redirection1 System Programming Shell Operators.
Searching and Sorting. Why Use Data Files? There are many cases where the input to the program may come from a data file.Using data files in your programs.
Math – What is a Function? 1. 2 input output function.
Selection Sort
Week 9 - Nov 7, Week 9 Agenda I/O redirection I/O redirection pipe pipe tee tee.
CSC 352– Unix Programming, Spring 2015 February 2015 Unix Filters.
CSC 4630 Meeting 17 March 21, Exam/Quiz Schedule Due to ice, travel, research and other commitments that we all have: –Quiz 2, scheduled for Monday.
ORAFACT Text Processing. ORAFACT Searching Inside Files grep - searches for patterns within files grep [options] [[-e] pattern] filename [...] -n shows.
Uniq The uniq command is useful when you need to find duplicate lines in a file. The basic format of the command is uniq in_file out_file In this format,
HANGMAN- Software/ Hardware Integration Project Idea was to design a working C++ program to play a Hangman word guessing game Idea was to design a working.
FILTERS USING REGULAR EXPRESSIONS – grep and sed.
Lesson 6-Using Utilities to Accomplish Complex Tasks.
In the last class, Filters and delimiters The sample database pr command head and tail commands cut and paste commands.
1 CSE 390a Lecture 2 Exploring Shell Commands, Streams, and Redirection slides created by Marty Stepp, modified by Jessica Miller & Ruth Anderson
CSC 4630 Perl 3 adapted from R. E. Beck. Problem But we worked on it first: Input: Read from a text file named in a command line argument Output: List.
SIMPLE FILTERS. CONTENTS Filters – definition To format text – pr Pick lines from the beginning – head Pick lines from the end – tail Extract characters.
1 Linux Commands. 2 Path You specify a file or directory by its path name:  the full, or absolute, path name or the one relative to a location. The full.
regular expressions - grep
Week 10.
CompSci 6 Introduction to Computer Science
(optional - but then again, all of these are optional)‏
Chapter 6 Filters.
Vi Editor.
CS 403: Programming Languages
INTRODUCTION TO UNIX: The Shell Command Interface
Exploring Shell Commands, Streams, and Redirection
Command Line Parameters
CSE 390a Lecture 2 Exploring Shell Commands, Streams, and Redirection
The Linux Command Line Chapter 6
شاخصهای عملکردی بیمارستان
مدل زنجیره ای در برنامه های سلامت
فرق بین خوب وعالی فقط اندکی تلاش بیشتر است
Chapter 8: Sorting in Linear Time
Function Notation “f of x” Input = x Output = f(x) = y.
Exploring Shell Commands, Streams, and Redirection
Y/I/Q channel blurring with 5x5 mean filter
CSE 390a Lecture 2 Exploring Shell Commands, Streams, and Redirection
Counter Integrated Circuits (I.C.s)
Module 6 Working with Files and Directories
Lab 7: Filtering.
Chapter 8: Sorting in Linear Time
Exploring Shell Commands, Streams, and Redirection
Software I: Utilities and Internals
GregWroblicky_script_Part01.py Source Datasets.
Exploring Shell Commands, Streams, and Redirection
Presentation transcript:

uniq command 6/12/2015Gary DeRoest1 report or filter out repeated lines in a file Note: the file needs to be sorted so that repeated lines are adjacent to each other. -c Precede each output line with the count of the number of times the line occurred. -d Don't output lines that are not repeated in the input. -u Don't output lines that are repeated in the input. uniq [-c -d -u] filename

uniq 6/12/2015Gary DeRoest2 report or filter out repeated lines in a file bob frank sue trish uniq names names bob frank sue trish

uniq 6/12/2015Gary DeRoest3 report or filter out repeated lines in a file frank sue uniq -u names names bob frank sue trish

uniq 6/12/2015Gary DeRoest4 report or filter out repeated lines in a file bob trish uniq -d names names bob frank sue trish

uniq 6/12/2015Gary DeRoest5 report or filter out repeated lines in a file 2 bob 1 frank 1 sue 2 trish uniq -c names names bob frank sue trish