Rossella Lau Lecture 7, DCO20105, Semester A,2005-6 DCO 20105 Data structures and algorithms  Lecture 7: Big-O analysis Sorting Algorithms  Big-O analysis.

Slides:



Advertisements
Similar presentations
Introduction to Algorithms Quicksort
Advertisements

Garfield AP Computer Science
Practice Quiz Question
CSCE 3110 Data Structures & Algorithm Analysis
DIVIDE AND CONQUER APPROACH. General Method Works on the approach of dividing a given problem into smaller sub problems (ideally of same size).  Divide.
Sorting Algorithms. Motivation Example: Phone Book Searching Example: Phone Book Searching If the phone book was in random order, we would probably never.
Data Structures Data Structures Topic #13. Today’s Agenda Sorting Algorithms: Recursive –mergesort –quicksort As we learn about each sorting algorithm,
Sorting Chapter Sorting Consider list x 1, x 2, x 3, … x n We seek to arrange the elements of the list in order –Ascending or descending Some O(n.
Data Structures and Algorithms PLSD210 Sorting. Card players all know how to sort … First card is already sorted With all the rest, ¶Scan back from the.
Chapter 19: Searching and Sorting Algorithms
Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved Sorting.
Lecture 25 Selection sort, reviewed Insertion sort, reviewed Merge sort Running time of merge sort, 2 ways to look at it Quicksort Course evaluations.
Chapter 11 Sorting and Searching. Copyright © 2005 Pearson Addison-Wesley. All rights reserved Chapter Objectives Examine the linear search and.
1 Algorithm Efficiency and Sorting (Walls & Mirrors - Remainder of Chapter 9)
E.G.M. Petrakissorting1 Sorting  Put data in order based on primary key  Many methods  Internal sorting:  data in arrays in main memory  External.
CHAPTER 11 Sorting.
Algorithm Efficiency and Sorting
Sorting CS-212 Dick Steflik. Exchange Sorting Method : make n-1 passes across the data, on each pass compare adjacent items, swapping as necessary (n-1.
Rossella Lau Lecture 1, DCO20105, Semester A, DCO Data structures and algorithms  Lecture 1: Introduction What this course is about:  Data.
© 2006 Pearson Addison-Wesley. All rights reserved10 A-1 Chapter 10 Algorithm Efficiency and Sorting.
CHAPTER 7: SORTING & SEARCHING Introduction to Computer Science Using Ruby (c) Ophir Frieder at al 2012.
1 Data Structures and Algorithms Sorting. 2  Sorting is the process of arranging a list of items into a particular order  There must be some value on.
Recursion, Complexity, and Searching and Sorting By Andrew Zeng.
Sorting HKOI Training Team (Advanced)
CHAPTER 09 Compiled by: Dr. Mohammad Omar Alhawarat Sorting & Searching.
Recursion, Complexity, and Sorting By Andrew Zeng.
Fall 2013 Instructor: Reza Entezari-Maleki Sharif University of Technology 1 Fundamentals of Programming Session 17 These.
Chapter 12 Recursion, Complexity, and Searching and Sorting
C++ Programming: Program Design Including Data Structures, Fourth Edition Chapter 19: Searching and Sorting Algorithms.
Chapter 19: Searching and Sorting Algorithms
HKOI 2006 Intermediate Training Searching and Sorting 1/4/2006.
Merge Sort. What Is Sorting? To arrange a collection of items in some specified order. Numerical order Lexicographical order Input: sequence of numbers.
C++ Programming: From Problem Analysis to Program Design, Second Edition Chapter 19: Searching and Sorting.
Elementary Sorting Algorithms Many of the slides are from Prof. Plaisted’s resources at University of North Carolina at Chapel Hill.
CSE 373 Data Structures and Algorithms
1 Joe Meehean.  Problem arrange comparable items in list into sorted order  Most sorting algorithms involve comparing item values  We assume items.
Sorting. Pseudocode of Insertion Sort Insertion Sort To sort array A[0..n-1], sort A[0..n-2] recursively and then insert A[n-1] in its proper place among.
Sorting Chapter Sorting Consider list x 1, x 2, x 3, … x n We seek to arrange the elements of the list in order –Ascending or descending Some O(n.
CS 61B Data Structures and Programming Methodology July 21, 2008 David Sun.
Chapter 18: Searching and Sorting Algorithms. Objectives In this chapter, you will: Learn the various search algorithms Implement sequential and binary.
3 – SIMPLE SORTING ALGORITHMS
Chapter 8 Sorting and Searching Goals: 1.Java implementation of sorting algorithms 2.Selection and Insertion Sorts 3.Recursive Sorts: Mergesort and Quicksort.
Review 1 Selection Sort Selection Sort Algorithm Time Complexity Best case Average case Worst case Examples.
Sorting and Searching by Dr P.Padmanabham Professor (CSE)&Director
1 Searching and Sorting Searching algorithms with simple arrays Sorting algorithms with simple arrays –Selection Sort –Insertion Sort –Bubble Sort –Quick.
Chapter 9 Sorting 1. The efficiency of data handling can often be increased if the data are sorted according to some criteria of order. The first step.
Chapter 9 Sorting. The efficiency of data handling can often be increased if the data are sorted according to some criteria of order. The first step is.
Intro To Algorithms Searching and Sorting. Searching A common task for a computer is to find a block of data A common task for a computer is to find a.
HEAPS. Review: what are the requirements of the abstract data type: priority queue? Quick removal of item with highest priority (highest or lowest key.
Searching and Sorting Searching: Sequential, Binary Sorting: Selection, Insertion, Shell.
ICS201 Lecture 21 : Sorting King Fahd University of Petroleum & Minerals College of Computer Science & Engineering Information & Computer Science Department.
PREVIOUS SORTING ALGORITHMS  BUBBLE SORT –Time Complexity: O(n 2 ) For each item, make (n –1) comparisons Gives: Comparisons = (n –1) + (n – 2)
Week 13 - Wednesday.  What did we talk about last time?  NP-completeness.
Chapter 9: Sorting1 Sorting & Searching Ch. # 9. Chapter 9: Sorting2 Chapter Outline  What is sorting and complexity of sorting  Different types of.
Quicksort This is probably the most popular sorting algorithm. It was invented by the English Scientist C.A.R. Hoare It is popular because it works well.
Sorting Algorithms Written by J.J. Shepherd. Sorting Review For each one of these sorting problems we are assuming ascending order so smallest to largest.
Sorting & Searching Geletaw S (MSC, MCITP). Objectives At the end of this session the students should be able to: – Design and implement the following.
SORTING Chapter 8. Chapter Objectives  To learn how to use the standard sorting methods in the Java API  To learn how to implement the following sorting.
WHICH SEARCH OR SORT IS BETTER?. COMPARING ALGORITHMS Time efficiency refers to how long it takes an algorithm to run Space efficiency refers to the amount.
Advanced Sorting 7 2  9 4   2   4   7
Data Structures and Algorithms
Description Given a linear collection of items x1, x2, x3,….,xn
10.3 Bubble Sort Chapter 10 - Sorting.
Sorting Chapter 13 Nyhoff, ADTs, Data Structures and Problem Solving with C++, Second Edition, © 2005 Pearson Education, Inc. All rights reserved
Quicksort and Mergesort
Quicksort analysis Bubble sort
8/04/2009 Many thanks to David Sun for some of the included slides!
Discrete Mathematics CMP-101 Lecture 12 Sorting, Bubble Sort, Insertion Sort, Greedy Algorithms Abdul Hameed
Sub-Quadratic Sorting Algorithms
Algorithm Course Algorithms Lecture 3 Sorting Algorithm-1
Presentation transcript:

Rossella Lau Lecture 7, DCO20105, Semester A, DCO Data structures and algorithms  Lecture 7: Big-O analysis Sorting Algorithms  Big-O analysis on different ways of multiplication  Sorting Algorithms: selection sort, bubble sort, insertion sort, radix sort, partition sort, merge sort  Comparison of different sorting algorithms -- By Rossella Lau

Rossella Lau Lecture 7, DCO20105, Semester A, Performance re-visit  For multiplication, we can use (at least) three different ways:  The one we used to use in primary school  bFunction(m, n) in slide 9 of Lecture 6  funny(a, b) in slide 10 of Lecture 6 int funny(int a, int b) { if ( a == 1 ) return b; return a & 1 ? funny ( a>>1, b<<1) + b : funny ( a>>1, b<<1); } int bFunction( int n, int m) { if (!n) return 0; return bFunction (n-1, m) + m; }

Rossella Lau Lecture 7, DCO20105, Semester A, Performance analysis The traditional waybFunction(n,m)funny(a, b) Memorizing time table is required (memory) Memorizing time table is not required Memorizing is not required for binary system (in a computer) Number of operations: Multiplications: #(n) * #(m) – memory recall Additions: #(n) #(n) is the number of digits of n Number of operations: number of n additions Number of operations: Shift operations: 2 * #b(a) Additions: worst case, #b(a) #b(a) is the number of digits of a in binary

Rossella Lau Lecture 7, DCO20105, Semester A, Execution time vs memory  The traditional multiplication has the least operations but it requires the most memory at O (log10 n)  bFunction() does not require additional memory but it spends a terrible amount of time getting the result : O(n)  funny() does not require additional memory and it has a bit more operations at O (log2 n)  The traditional way may have less operations but hard to say if it really outperforms funny() since memory load may not be faster than shift operation

Rossella Lau Lecture 7, DCO20105, Semester A, Ordering of data  In order to search a record efficiently, records are stored in the order of key values  A key is a field or some fields of a record that can uniquely identify the record in a file  Usually, only the key values are stored in memory and the corresponding record is loaded into the memory only when it is necessary  The key values, therefore, usually are sorted in a special order to allow efficient searching

Rossella Lau Lecture 7, DCO20105, Semester A, Classification of sorting methods  Comparison-Based Methods  Insertion Sorts  Selection Sorts  Heapsort (tree sorting) – in future lesson  Exchange sorts Bubble sort Quick sort  Merge sorts  Distribution Methods: Radix sorting

Rossella Lau Lecture 7, DCO20105, Semester A, Selection sort  Selection: choose the smaller element from a list and place it in the 1 st position.  The process is from the first element to the second to last element on a list and for each element to apply the “selection” on the sub-list starting from the element being processed.  Ford’ text book slides 2-9 in Chapter 3

Rossella Lau Lecture 7, DCO20105, Semester A, Bubble sort  To pass through the array n-1 times, where n is the number of data in the array  For each pass:  compare each element in the array with its successor  interchange the two elements if they are not in order  The algorithm bubble (int x[], int n) { for (i=0; i<n-1; i++) for (j=0; j<n-1; j++) if (x[j] > x[j+1]) SWAP (x[j], x[j+1]) }

Rossella Lau Lecture 7, DCO20105, Semester A, An example trace of bubble sort Given data sequence: The first pass: Subsequent passes: Pass2: Pass3: Pass4: Pass5: Pass6: Pass7:

Rossella Lau Lecture 7, DCO20105, Semester A, Improvement can be made  At pass i, the last i elements should be in proper positions since, at the first pass the largest element should be placed at the end of the array. At the second pass, the second large element should be placed before the last element, and so on.  The comparison only requires from x[0] to x[n-i-1]  The array has already been sorted at the fifth iteration and the sixth and seventh are redundant  Therefore, once no exchange is required in an iteration, the array is already sorted and the subsequent iterations are redundant

Rossella Lau Lecture 7, DCO20105, Semester A, The improved algorithm for bubble sort void bubble1 (int x[], int n) { exchange = TRUE; for (i=0; i<n-1 && exchange; i++) { exchange = FALSE; for (j=0; i<n-i-1; j++) if (x[j] > x[j+1]) { exchange = TRUE; SWAP(x[j], x[j+1]); }/*end if */ }/* end for i */ }

Rossella Lau Lecture 7, DCO20105, Semester A, Performance considerations of bubble sort  For the first version, it requires (n-1) comparisons in (n-1) passes  the total number of comparisons is n 2 - 2n +1, i.e., O(n 2 )  For the improved version, it requires (n-1) + (n-2) (n-k) for k (<n) passes  the total number of comparisons is (2kn-k 2 -k)/2. However, the average k is O(n) yielding the overall complexity as O(n 2 ) and the overhead (set and check exchange) introduced should also be considered  It only requires little additional space

Rossella Lau Lecture 7, DCO20105, Semester A, Insertion sort  Insert an item into a previous sorted order one by one for each of the data.  It is similar to repeatedly picking up playing cards and inserting them into the proper position in a partial hand of cards

Rossella Lau Lecture 7, DCO20105, Semester A, An example trace of insertion sort ……

Rossella Lau Lecture 7, DCO20105, Semester A, The algorithm of insertion sort void insertsort(x,n) int x[], n) {for (k=1; k<n; k++) { y = x[k]; for (i = k-1; i >=0 && y<x[i]; i--) x[i+1] = x[i]; x[i+1] = y; } /* end for k */ }  The checking of i>=0 is time consuming. Setting a sentinel in the beginning of the array will prevent y from going beyond the array void insertsort(int x[], int m) /* m is n+1, data from x[1] */ X[0] = MAXNEGINT {for (k=2; k < m; k++) { y = x[k]; for (i = k-1; y<x[i]; i--) x[i+1] = x[i]; x[i+1] = y; } /* end for k */ }

Rossella Lau Lecture 7, DCO20105, Semester A, Performance analysis of insertion sort  If the original sequence is already in order, only one comparison is made on each pass ==> O(n)  If the original sequence is in a reversed order, it requires n comparison in each pass ==> O(n 2 )  The complexity is from O(n) to O(n 2 )  It requires little additional space

Rossella Lau Lecture 7, DCO20105, Semester A, Quick sort  It is also called partition exchange sort  In each step, the original sequence is partitioned into 3 parts: a. all the items less than the partitioning element b. the partitioning element in its final position c. all the items greater than the partitioning element  The partitioning process continues in the left and right partitions

Rossella Lau Lecture 7, DCO20105, Semester A, The partitioning in each step of quicksort  To pick one of the elements as the partitioning element, p, usually the first element of the sequence  To find the proper position for p while partitioning the sequence into 3 parts a) it employs two indexes, down and up b) down goes from left to right to find elements greater than p c) up goes from right to left to find elements less than p d) elements found by up and down are exchanged e) process until up and down are matched or passed each other f) the position of p should be pointed by up g) exchange p with the element pointed by up

Rossella Lau Lecture 7, DCO20105, Semester A, An example trace of quicksort (12) 25 ( ) Subsequent processes: ( ) ( ) (33 37) 48 ( ) 12 25()33 (37) 48 (57 86) 92() (57 86) ()57(86) _ down, _ up

Rossella Lau Lecture 7, DCO20105, Semester A, The algorithm for quicksort void quickSort(int x[], int left, int right) { int down, up, partition; down=left; up=right+1; partition=x[left]; while (down<up) { while (x[++down] <= partition); while (x[--up] > partition); if (down<up) SWAP(x[down], x[up]) } x[left] = x[up]; x[up] = partition; if (left < up - 1) quickSort(x, left, up-1); if (down < right) quickSort(x, down, right); }

Rossella Lau Lecture 7, DCO20105, Semester A, Performance considerations of quicksort  Quciksort got its name because it quickly puts an element into its proper position by employing two indexes to speed up the partioning process and to minimize the exchange  Each pass reduces the comparisons about a half  total number of comparisons is about O(nlog 2 n)  It requires spaces for the recursive process or stacks for an iterative process, it is about O(log 2 n)

Rossella Lau Lecture 7, DCO20105, Semester A, Merge  Merge means to combine two or more sorted sequences into another sorted sequence  The merging of two sequences, for example, are as follows | | | | | | | | | | | | | | | | | | | | _| _|

Rossella Lau Lecture 7, DCO20105, Semester A, Merge sort  It employs the merging technique in the following way: 1. Divide the sequence into n parts 2. Merge adjacent parts yielding the sequence n/2 parts 3. Merge adjacent parts again yielding the sequence n/4 parts Process goes on until the sequence becomes 1 part

Rossella Lau Lecture 7, DCO20105, Semester A, An example of merge sort 8 parts merge parts merge parts merge

Rossella Lau Lecture 7, DCO20105, Semester A, Performance considerations of merge sort  There are only log 2 n passes yielding a complexity of O(nlogn)  It never requires n* log 2 n comparison while quicksort may require O(n 2 ) at the worst case  However, it requires about double of assignment statements as quicksort  It also requires more additional spaces, about O(n), than quicksort's O(log 2 n)

Rossella Lau Lecture 7, DCO20105, Semester A, Radix Sort  It is based on the values of the actual digits of its octal position  Starting from the least significant digit to the most significant digit  define 10 vectors for each digit and number the vectors from v0 to v9 for digit 0 to 9 respectively  scan the data sequence once and add x i into the significant digit's respective vector  new data sequence is as follows: remove elements from each vector from the beginning one by one until it is empty from q0 to q9  After the above actions, the new data sequence is the sorted sequence!

Rossella Lau Lecture 7, DCO20105, Semester A, An example of radix sort

Rossella Lau Lecture 7, DCO20105, Semester A, Performance considerations of radix sort  It does not require any comparison between data  It requires number of digits, log10 m, passes   O(n*log10 m)  O(n), treating log10 m a constant  It requires 10 times of the memory for numbers  It seems that radix sort has the “best” performance; however, it is not popularly used because  It consumes a terrible amount of memory  Log10 m depends on the digit (length) of a key and may not be treated as a small constant when the key length is long

Rossella Lau Lecture 7, DCO20105, Semester A, The real life sort for vector based data  Although quick sort is known to be the fastest in many cases, the library will not usually directly use quick sort as the sort method  Usually, a carefully designed library will implement its sort method with quick sort and insertion sort  Quick sort divides partitions until a partition is about the size from 8 to 16, insertion is applied to the partition since the partitions usually are near being sorted

Rossella Lau Lecture 7, DCO20105, Semester A, The real life sort for non vector data  Quick sort requires a container with random access  A container such as a linked list does not support random access and cannot apply quick sort  Merge sort is preferred to be applied

Rossella Lau Lecture 7, DCO20105, Semester A, Sample timing of sort methods  Ford’s prg15_2.cpp & d_sort.h Timing for some sample runs : timeSort.out

Rossella Lau Lecture 7, DCO20105, Semester A, Summary  Bubble sort and insertion sort have complexity of O(n 2 ) but insertion sort is still preferred for short data stream  Partition sort, merge sort have a less complexity at O(n logn)  Radix sort seemed at O(n) complexity but it consumes more memory and may depend on the key length  Many times, the trade off is space

Rossella Lau Lecture 7, DCO20105, Semester A, Reference  Ford: 3.1, 4.4,  Data Structures using C and C++ by Yedidyah Langsam, Moshe J. Augenstein & Aaron M. Tenenbaum: Chapter 6  Example programs: Ford: prg15_2.cpp, d_sort.h, -- END --