Ch8: Sorting in Linear Time Ming-Te Chi Algorithms Ch8: Sorting in Linear Time Ming-Te Chi Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time Overview This week topics：sorting Next topics：searching and related subjects. Computer is powerful in manipulating data Store data and perform operations on data Search/retrieve Insert Delete Other operations Ch8 Sorting in Linear Time

Sorting – how fast can we sort? Selection or insertion sort Merge sort Heap sort Quick sort . . . Complexity (time) is either O(n log n) or O(n2) Ch8 Sorting in Linear Time

Lower bounds for sorting to examine all the input. All sorts seen so far are We’ll show that is a lower bound for comparison sorts. Ch8 Sorting in Linear Time

Sorting based on Comparison Selection sort, insertion sort, merge sort, heap sort, quick sort All have time complexity Based on comparison between the input elements Comparison sorts can be viewed abstractly in terms of decision tree Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time Decision tree Fully binary tree Abstraction of any comparison sort. Represents comparisons made by a specific sorting algorithm on inputs of a given size Abstracts away everything else: control and data movement. We’re counting only comparisons Ch8 Sorting in Linear Time

Example: Decision Tree for Sorting Three Element {a, b, c} a:b a ≤ b a > b a:c b:c b ≤ c a ≤ c a > c b > c b:c cab a:c cba b ≤ c a ≤ c b > c a > c abc acb bac bca Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time Each leaf is labeled by the permutation of orders that the algorithm determines Any correct sorting algorithm must be able to produce each permutation of its input. A correct comparison sort algorithm requires that each of the permutations must appear as one of the leaves of the decision tree. Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time Each of these leaves must be reachable from the root by a path corresponding to an actual execution of the comparison sort. The execution of the sorting algorithm corresponds to tracing a path from the root of the decision tree to a leaf. Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time How many leaves? What is the height of the tree? Ch8 Sorting in Linear Time

Properties of the decision tree At least n! leaves For any comparison sort 1 tree for each n View the tree as if the algorithm splits in two at each node, based on the information it has determined up to that point. The tree models all possible execution traces. Ch8 Sorting in Linear Time

Properties of the decision tree (continue) Length of the longest path from root to leaf? The worst-case number of comparisons that the corresponding sorting algorithm performs. Depends on the algorithm Insertion sort: Merge sort: Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time

Bucket Sort: linear time sorting Allocate sufficient number of buckets & put element in corresponding buckets at the end, scan the buckets in order & collect all elements n elements, ranges from 1 to m  m buckets O(m+n) Best case: O(n), Worst case: O(m) Ch8 Sorting in Linear Time

Sorting in linear time: 8.2 Counting sort (similar to bucket sort) Depends on a key assumption: n numbers to be sorted are integers in {0, 1, . . . , k}. Input: A[1 . . n], where A[ j ] ∈ {0, 1, . . . , k} for j = 1, 2, . . . , n. (Array A and values n and k are given as parameters.) Output: B[1 . . n], sorted. (B is assumed to be already allocated and is given as a parameter.) Auxiliary storage: C[0 . . k] Ch8 Sorting in Linear Time

Sorting in linear time: 8.2 Counting sort Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time How big a k is practical? Good for sorting 32-bit values? No. 16-bit? Probably not. 8-bit? Maybe, depending on n. 4-bit? Probably (unless n is really small) Example: playing cards. (撲克牌) Counting sort (bucket sort) will be used in radix sort. Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time 8.3 Radix sort Used by the card-sorting machines one can now find only in computer museum. Sort the least significant digit first RADIX_SORT(A,d) 1 for i  1 to d 2 do use a stable sort to sort array A on digit i Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time Analysis: Radix sort Assume that we use counting sort as the intermediate sort. per pass (digits in range 0, . . . , k) d passes total If , time complexity is Not in-place (in-place: only constant number of elements of the input array are ever stored outside the array. Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time Sample question: What is the decision tree? Explain how it can be used to prove the lower bound of the comparison based sorting algorithms. It is well known that radix sort has the time complexity of O(d n), where d is the maximal number of digits of all the inputs and n is the total number of inputs. Are these two results contradict to each other? Justify your answer. Ch8 Sorting in Linear Time

Bucket sort (new version) Divide the interval [0, 1) into n equal-sized subintervals (buckets). Distribute the n input numbers into the buckets. Input: n-element array A, with Needs an auxiliary array B[0..n-1] of linked lists. Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time Analysis The running time of bucket sort is where ni is the number of elements placed in the bucket B[i]. Why? Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time Analysis (continue) Taking expectations of both sides and using linearity of expectation, we have Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time We claim that We define indicator random variables Xij = I {A[j] falls in bucket i} for i = 0, 1, …, n-1 and j = 1, 2,…,n. thus, Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time Indicator random variable Xij is 1 with probability 1/n and 0 otherwise, and therefore When k  j, the variables Xij and Xik are independent, and hence Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time We can conclude that the expected time for bucket sort is (n)+n·O(2-1/n)= (n). Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time

Ch8 Sorting in Linear Time Remarks: Decision tree proves that O(n log n) is required for sorting Radix sort (or bucket sort) states that O(n) is enough for sorting Is there anything wrong? Ch8 Sorting in Linear Time