Sorting CS 110: Data Structures and Algorithms First Semester, 2010-2011.

Slides:



Advertisements
Similar presentations
Introduction to Algorithms Quicksort
Advertisements

Garfield AP Computer Science
Divide and Conquer Sorting Algorithms
Algorithms Analysis Lecture 6 Quicksort. Quick Sort Divide and Conquer.
Bucket Sort and Radix Sort CS /02/05 BucketSort Slide 2 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights.
CSE 3101: Introduction to the Design and Analysis of Algorithms
Quick Sort, Shell Sort, Counting Sort, Radix Sort AND Bucket Sort
CSCE 3110 Data Structures & Algorithm Analysis
Using Divide and Conquer for Sorting
Sorting Algorithms. Motivation Example: Phone Book Searching Example: Phone Book Searching If the phone book was in random order, we would probably never.
Quicksort CS 3358 Data Structures. Sorting II/ Slide 2 Introduction Fastest known sorting algorithm in practice * Average case: O(N log N) * Worst case:
Sorting Algorithms and Average Case Time Complexity
Sorting Chapter Sorting Consider list x 1, x 2, x 3, … x n We seek to arrange the elements of the list in order –Ascending or descending Some O(n.
CHAPTER 11 Sorting.
Sorting Chapter 10.
TTIT33 Algorithms and Optimization – Dalg Lecture 2 HT TTIT33 Algorithms and optimization Lecture 2 Algorithms Sorting [GT] 3.1.2, 11 [LD] ,
CS 104 Introduction to Computer Science and Graphics Problems Data Structure & Algorithms (3) Recurrence Relation 11/11 ~ 11/14/2008 Yang Song.
Sorting CS-212 Dick Steflik. Exchange Sorting Method : make n-1 passes across the data, on each pass compare adjacent items, swapping as necessary (n-1.
Sorting Chapter 10. Chapter 10: Sorting2 Chapter Objectives To learn how to use the standard sorting methods in the Java API To learn how to implement.
CSE 373 Data Structures Lecture 19
CSE 373 Data Structures Lecture 15
Divide-And-Conquer Sorting Small instance.  n
Sorting HKOI Training Team (Advanced)
CHAPTER 09 Compiled by: Dr. Mohammad Omar Alhawarat Sorting & Searching.
CSC 213 Lecture 12: Quick Sort & Radix/Bucket Sort.
Fall 2013 Instructor: Reza Entezari-Maleki Sharif University of Technology 1 Fundamentals of Programming Session 17 These.
CSCE 3110 Data Structures & Algorithm Analysis Sorting (I) Reading: Chap.7, Weiss.
C++ Programming: Program Design Including Data Structures, Fourth Edition Chapter 19: Searching and Sorting Algorithms.
HKOI 2006 Intermediate Training Searching and Sorting 1/4/2006.
1 Lecture 16: Lists and vectors Binary search, Sorting.
Merge Sort. What Is Sorting? To arrange a collection of items in some specified order. Numerical order Lexicographical order Input: sequence of numbers.
Chapter 10 B Algorithm Efficiency and Sorting. © 2004 Pearson Addison-Wesley. All rights reserved 9 A-2 Sorting Algorithms and Their Efficiency Sorting.
Sorting Chapter 10. Chapter Objectives  To learn how to use the standard sorting methods in the Java API  To learn how to implement the following sorting.
Sorting Fun1 Chapter 4: Sorting     29  9.
Sorting. Pseudocode of Insertion Sort Insertion Sort To sort array A[0..n-1], sort A[0..n-2] recursively and then insert A[n-1] in its proper place among.
Sorting CS /02/05 L12: Sorting Slide 2 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved The.
© 2006 Pearson Addison-Wesley. All rights reserved10 A-1 Chapter 10 Algorithm Efficiency and Sorting.
Sorting CS 105 See Chapter 14 of Horstmann text. Sorting Slide 2 The Sorting problem Input: a collection S of n elements that can be ordered Output: the.
Survey of Sorting Ananda Gunawardena. Naïve sorting algorithms Bubble sort: scan for flips, until all are fixed Etc...
Sorting Chapter Sorting Consider list x 1, x 2, x 3, … x n We seek to arrange the elements of the list in order –Ascending or descending Some O(n.
CS 361 – Chapters 8-9 Sorting algorithms –Selection, insertion, bubble, “swap” –Merge, quick, stooge –Counting, bucket, radix How to select the n-th largest/smallest.
Sorting Sorting: –Task of rearranging data in an order. –Order can be: Ascending Order: –1,2,3,4,5,6,7,8,9 Descending Order: –9,8,7,6,5,4,3,2,1 Lexicographic.
Algorithms and their Applications CS2004 ( ) Professor Jasna Kuljis (adapted from Dr Steve Swift) 6.1 Classic Algorithms - Sorting.
Sorting CSIT 402 Data Structures II. 2 Sorting (Ascending Order) Input ›an array A of data records ›a key value in each data record ›a comparison function.
Chapter 18: Searching and Sorting Algorithms. Objectives In this chapter, you will: Learn the various search algorithms Implement sequential and binary.
Searching and Sorting Recursion, Merge-sort, Divide & Conquer, Bucket sort, Radix sort Lecture 5.
Bucket Sort and Radix Sort
1 Introduction to Sorting Algorithms Sort: arrange values into an order Alphabetical Ascending numeric Descending numeric Two algorithms considered here.
1 Sorting (Bubble Sort, Insertion Sort, Selection Sort)
Review 1 Selection Sort Selection Sort Algorithm Time Complexity Best case Average case Worst case Examples.
Divide And Conquer A large instance is solved as follows:  Divide the large instance into smaller instances.  Solve the smaller instances somehow. 
Divide and Conquer Sorting Algorithms CS /02/05 HeapSort Slide 2 Copyright 2005, by the authors of these slides, and Ateneo de Manila University.
Sorting and Searching by Dr P.Padmanabham Professor (CSE)&Director
Quick sort, lower bound on sorting, bucket sort, radix sort, comparison of algorithms, code, … Sorting: part 2.
Data Structures - CSCI 102 Selection Sort Keep the list separated into sorted and unsorted sections Start by finding the minimum & put it at the front.
1 Ch.19 Divide and Conquer. 2 BIRD’S-EYE VIEW Divide and conquer algorithms Decompose a problem instance into several smaller independent instances May.
Young CS 331 D&A of Algo. Topic: Divide and Conquer1 Divide-and-Conquer General idea: Divide a problem into subprograms of the same kind; solve subprograms.
Nirmalya Roy School of Electrical Engineering and Computer Science Washington State University Cpt S 122 – Data Structures Sorting.
Computer Science 1620 Sorting. cases exist where we would like our data to be in ascending (descending order) binary searching printing purposes selection.
PREVIOUS SORTING ALGORITHMS  BUBBLE SORT –Time Complexity: O(n 2 ) For each item, make (n –1) comparisons Gives: Comparisons = (n –1) + (n – 2)
Chapter 9: Sorting1 Sorting & Searching Ch. # 9. Chapter 9: Sorting2 Chapter Outline  What is sorting and complexity of sorting  Different types of.
INTRO2CS Tirgul 8 1. Searching and Sorting  Tips for debugging  Binary search  Sorting algorithms:  Bogo sort  Bubble sort  Quick sort and maybe.
Liang, Introduction to Java Programming, Tenth Edition, (c) 2013 Pearson Education, Inc. All rights reserved. 1 Chapter 23 Sorting.
Advanced Sorting 7 2  9 4   2   4   7
Chapter 11 Sorting Acknowledgement: These slides are adapted from slides provided with Data Structures and Algorithms in C++, Goodrich, Tamassia and Mount.
Algorithm Design and Analysis (ADA)
Sorting.
Sorting Algorithms Ellysa N. Kosinaya.
Analysis of Algorithms
CS203 Lecture 15.
Presentation transcript:

Sorting CS 110: Data Structures and Algorithms First Semester,

Learning Objectives ► Explain and implement sorting algorithms ► Explain pros and cons of each implementation

The Sorting Problem Input: a collection S of n elements that can be ordered Output: the same collection of elements arranged in increasing (or non-decreasing) order *typically, S would be stored in an array, and the problem is to rearrange the elements in that array for now, let’s assume we are sorting a collection of integers

Example

Some Sorting Algorithms ► Insertion sort ► Selection sort ► Bubble sort ► Quick sort ► Merge sort ► Heap sort ► Bucket sort ► Radix sort O ( n 2 ) O ( n log n ) O ( n )

Insertion Sort ► Strategy: treat each s[i] as an incoming element that you will insert into the already sorted sequence s[0],s[i],…s[i-1] ► Requires locating the proper position of the incoming element and adjusting elements to the right ► Best case: the array is already sorted so that no “insertions” are carried out  O( n ) ► Worst case: the array is in decreasing order; incoming elements are always inserted at the beginning  O( n 2 )

Insertion Sort for i  1 to n-1 do temp  s[i] // incoming element j  i // adjust elements to the right while ( j > 0 && s[j-1] > temp ) s[j]  s[j-1]; j--; s[j] = temp; // insert incoming element

Insertion Sort Example i=1i=2i=3i=4i=5i=6i=

Selection Sort ► Strategy: locate the minimum element, place it at the first position, locate the next minimum and place it at the second position … ► Requires a scan ( O( n ) ) for each of the n elements  O( n 2 ) best and worst case ► Variation: can repeatedly select the maximum instead and place it at the last position

Selection Sort for i  0 to n-2 do lowIndex  i // determine for j  i+1 to n-1 do // minimum if (s[j] < s[lowIndex] ) lowIndex  j swap( s[i], s[lowIndex] ) // place minimum // in proper place Why not n-1?

Selection Sort Example i=0i=1i=2i=3i=4i=5i=

Selection Sort Variation ► Repeatedly select maximum instead for i  n-1 downto 1 do highIndex  i // determine for j  0 to i-1 do // maximum if (s[j] > s[highIndex] ) highIndex  j swap( s[i], s[highIndex] ) // place maximum // in proper place

Bubble Sort ► Essentially selection sort but the sort is carried out by swapping adjacent elements only ► Minimum elements are repeatedly “bubbled-up”, maximum elements are repeatedly “bubbled-down” the array ► O( n 2 ) because of the comparisons (actual swaps are carried out only when elements are out of place)

Bubble Sort for i  n-1 down to 1 do for j  0 to i-1 do if (s[j] > s[j+1] ) swap( s[j], s[j+1] ) Puts the ith element in its proper place Repeatedly positions the maximum element

Exercise: Bubble Sort Perform a trace for this array

Bubble Sort Variation for i  0 to n-2 do for j  n-1 to i+1 do if (s[j] < s[j-1] ) swap( s[j], s[j-1] ) Puts the ith element in its proper place Repeatedly positions the MINIMUM element

Time Complexity Summary AlgorithmBest CaseWorst Case Insertion SortO( n )O(n 2 ) Selection Sort O(n 2 ) Bubble SortO(n 2 )

Improved Sorting Strategy ► Divide-and-Conquer ► Given the collection of n elements to sort: perform the sort in three steps ► Divide step: split the collection S into two subsets, S1 and S2 ► Recursion step: sort S1 and S2 separately ► Conquer step: combine the two lists into one sorted list

Quick Sort and Merge Sort ► Two algorithms adopt this divide-and-conquer strategy ► Quick sort ► Work is carried out in the divide step using a pivot element ► Conquer step is trivial ► Merge sort ► Divide step is trivial – just split the list into two equal parts ► Work is carried out in the conquer step by merging two sorted lists

Quick Sort: Divide Step ► In the divide step, select a pivot from the array (say, the last element) ► Split the list/array S using the pivot: S1 consists of all elements pivot S1S2

Quick Sort: Conquer Step ► After sorting S1 and S2, combine the sorted lists so that S1 is on the left, the pivot is in the middle, and S2 is on the right S1 sortedS2 sorted

Quick Sort with Recur Step S1S S1 sortedS2 sorted Divide Recur Conquer

Implementing Quick Sort ► It is preferable if we can perform quick-sort in-place; i.e., we sort by swapping elements in the array, perhaps using some temporary variables ► Plan: algorithm QSort( S, a, b ) sorts the sublist in the array S from index a to index b ► QSort( L, 0, n-1 ) will sort an array L of length n ► Within the QSort( S, a, b ) algorithm, there will be recursive calls to QSort on smaller ranges within the range a…b.

Algorithm QSort Algorithm QSort( S, a, b ) if ( a < b ) p  S[b] rearrange S so that: S[a]…S[x-1] are elements < p S[x] = p S[x+1]…S[b] are elements > p QSort( S, a, x-1 ) QSort( S, x+1, b ) base case: a…b range contains 0 or 1 element

Rearranging a Sublist in S p  S[b], l  a, r  b - 1 while l <= r do // find an element larger than the pivot while l <= r and S[l] <= p do l  l + 1 // find an element smaller than the pivot while r >= l and S[r] >= p do r  r – 1 if l < r then swap ( S[l], S[r] ) // swap the two elements swap( S[l], S[b] ) // place pivot in proper place

Time Complexity of Quick Sort ► First, note that rearranging a sublist takes O( n ) time, where n is the length of the sublist ► Requires scanning the list from both ends until both l and r pointers meet ► O( n ) even if loops are nested within a loop ► Rearranging sublists is all that the quick sort algorithm does ► Need to find out how often the sort would perform the rearrange operation

Time Complexity of Quick Sort Suppose the pivots always split the lists into two lists of roughly equal size 1 list of length n 2 lists of length n/2 4 lists of length n/4 n lists of length 1

Time Complexity of Quick Sort ► Each level takes O( n ) time for sublist rearranging ► Assuming an even split caused by each pivot, there will be around log n levels ► Therefore, quick sort takes O( n log n ) time ► But…

Time Complexity of Quick Sort ► In the worst case, the pivot might split the lists such that there is only 1 element in one partition (not an even split) ► There will be n levels ► Each level requires O( n ) time for sublist rearranging ► Quick sort takes O( n 2 ) time in the worst case

Merge Sort ► Another sorting algorithm using the divide-and-conquer paradigm ► This time, the hard work is carried out in the conquer phase instead of the divide phase ► Divide: split the list S[0..n-1] by taking the middle index m ( = (0 + n-1) / 2 ) ► Recursion: recursively sort S[0..m] and S[m+1..n-1] ► Conquer: merge the two sorted lists (how?)

Merge Sort S1S S1 sortedS2 sorted Divide Recur Conquer

Merge Sort Time Complexity ► Divide step ensures that the sublist split is done evenly ► O( log n ) levels ► Conquer/merge step takes O( n ) time per level ► Time complexity is O( n log n ), guaranteed ► Disadvantage: hard to carry out the merge step in-place; temporary array/list is necessary if we want a simple implementation

Time Complexity Summary AlgorithmBest CaseWorst Case Quick sortO( n log n )O(n 2 ) Merge sortO( n log n )

Time Complexity of Sorting ► Several sorting algorithms have been discussed and the best ones, so far: ► Merge sort: O( n log n ) ► Quick sort (best one in practice): O( n log n ) on average, O( n 2 ) worst case ► Can we do better than O( n log n )? ► No. ► It can be proven that any comparison-based sorting algorithm will need to carry out at least O( n log n ) operations

Restrictions on the Problem ► Suppose the values in the list to be sorted can repeat but the values have a limit (e.g., values are digits from 0 to 9) ► Sorting, in this case, appears easier ► Is it possible to come up with an algorithm better than O( n log n )? ► Yes ► Strategy will not involve comparisons

Bucket Sort ► Idea: suppose the values are in the range 0..m-1; start with m empty buckets numbered 0 to m-1, scan the list and place element s[i] in bucket s[i], and then output the buckets in order ► Will need an array of buckets, and the values in the list to be sorted will be the indexes to the buckets ► No comparisons will be necessary

Bucket Sort: Example

Bucket Sort Algorithm Algorithm BucketSort( S ) ( values in S are between 0 and m-1 ) for j  0 to m-1 do// initialize m buckets b[j]  0 for i  0 to n-1 do// place elements in their b[S[i]]  b[S[i]] + 1// appropriate buckets i  0 for j  0 to m-1 do// place elements in buckets for r  1 to b[j] do// back in S S[i]  j i  i + 1

Time Complexity ► Bucket initialization: O( m ) ► From array to buckets: O( n ) ► From buckets to array: O( n ) ► Even though this stage is a nested loop, the sum of the number of elements in each bucket is n ► Since m will likely be small compared to n, Bucket sort is O( n ) ► Strictly speaking, time complexity is O ( n + m )

Sorting Integers ► Can we perform bucket sort on any array of (non-negative) integers? ► Yes, but note that the number of buckets will depend on the maximum integer value ► If you are sorting 1000 integers and the maximum value is , you will need 1 million buckets! ► Time complexity is not really O( n ) because m is much > than n. Actual time complexity is O( m ) ► Can we do better?

Radix Sort ► Idea: repeatedly sort by digit—perform multiple bucket sorts on S starting with the rightmost digit ► If maximum value is , only ten buckets (not 1 million) will be necessary ► Use this strategy when the keys are integers, and there is a reasonable limit on their values ► Number of passes (bucket sort stages) will depend on the number of digits in the maximum value

Radix Sort Example: first pass

Radix Sort Example: second pass

Radix Sort Example: 1 st & 2 nd passes sort by rightmost digit sort by leftmost digit

Radix Sort and Stability ► Radix sort works as long as the bucket sort stages are stable sorts ► Stable sort: in case of ties, relative order of elements are preserved in the resulting array ► Suppose there are two elements whose first digit is the same; for example, 52 & 58 ► If 52 occurs before 58 in the array prior to the sorting stage, 52 should occur before 58 in the resulting array ► This way, the work carried out in the previous bucket sort stages is preserved

Time Complexity ► If there is a fixed number p of bucket sort stages (six stages in the case where the maximum value is ), then radix sort is O( n ) ► There are p bucket sort stages, each taking O( n ) time ► Strictly speaking, time complexity is O( pn ), where p is the number of digits (note that p = log 10 m, where m is the maximum value in the list)

About Radix Sort ► Note that only 10 buckets are needed regardless of number of stages since the buckets are reused at each stage ► Radix sort can apply to words ► Set a limit to the number of letters in a word ► Use 27 buckets (or more, depending on the letters/characters allowed), one for each letter plus a “blank” character ► The word-length limit is exactly the number of bucket sort stages needed

Summary ► O( n 2 ) algorithms (insertion-, selection-, bubble- sort) are easy to code but are inefficient ► Quick sort has an O( n 2 ) worst case but works very well in practice; O( n log n ) on the average ► Merge sort is difficult to implement in-place but O( n log n ) complexity is guaranteed ► Note: it doesn’t perform well in practice

Summary ► Bucket sort and Radix sort are O( n ) algorithms only because we have imposed restrictions on the input list to be sorted ► Sorting, in general, can be done in O( n log n ) time ► Later this semester ► A guaranteed O( n log n ) algorithm called Heap sort that is a reasonable alternative to quick sort