Dr. Yingwu Zhu Chapter 9, p 213-220 Linear Time Selection Dr. Yingwu Zhu Chapter 9, p 213-220

Terminology Order statistic The i-th order statistic of a set of n elements is the i-th smallest element Median n is odd, (n+1)/2 –th smallest n is even, floor((n+1)/2), usually lower median Selection problem Given a set of n elements and i such that 1<=i<=n Find the i-th smallest element

Where to Build a Road? Given (x,y) coordinates of N houses, where should you build road parallel to x-axis to minimize construction cost of building driveways? 1 2 3 Chazelle: Amazingly enough, the road costs nothing to built. It is the driveways that cost money. Driveway cost is proportional to its distance to road. (Obviously, they will be perpendicular.) 4 5 6 7 8 9 10 11 12

Where to Build a Road? Given (x,y) coordinates of N houses, where should you build road parallel to x-axis to minimize construction cost of building driveways? n1 = nodes on top. n2 = nodes on bottom. 1 Decreases total cost by (n2-n1)  2 3 4  5 6 7 8 9 10 11 12

Where to Build a Road? Given (x,y) coordinates of N houses, where should you build road parallel to x-axis to minimize construction cost of building driveways? n1 = nodes on top. n2 = nodes on bottom. 1 2 3 4 5 6 7 8 9 10 11 12

Where to Build a Road? Given (x,y) coordinates of N houses, where should you build road parallel to x-axis to minimize construction cost of building driveways? n1 = nodes on top. n2 = nodes on bottom. 1 2 3 4 5 6 7 8 9 10 11 12

Where to Build a Road? Given (x,y) coordinates of N houses, where should you build road parallel to x-axis to minimize construction cost of building driveways? Solution: put street at median of y coordinates. 1 2 3 4 5 6 7 8 9 10 11 12

Order Statistics Given N linearly ordered elements, find ith smallest element. Min: i = 1 Max: i = N Median: i = (N+1) / 2 and (N+1) / 2 O(N) for min or max. O(N log N) comparisons by sorting. O(N log i) with heaps. Can we do in worst-case O(N) comparisons? Surprisingly, yes. (Blum, Floyd, Pratt, Rivest, Tarjan, 1973) Assumption to make presentation cleaner. All items have distinct values.

Randomized Select Similar to quicksort, but throw away useless "half" at each iteration. Select ith smallest element from A[L..H]. if (L == H) return A[L] x  random_partition(A, L, H) k  x-L+1 if (i == k) return x else if (i < k) return Rand_Select(A, L, x-1, i) else if (i > k) return Select(A, x+1, H, i-k) Rand_Select (A, L, H, i) x = pivot position

Summary The expected running time: O(n) The worst-case running time: O(n2) However, because it is randomized, no particular input elicits the worst-case behavior

The worst-case linear time selection: partition Divide N elements into N/5 groups of 5 elements each, plus extra. 22 45 14 28 29 44 39 09 23 10 52 50 05 06 38 11 15 03 26 37 53 25 54 40 02 12 16 30 19 55 13 41 48 01 18 43 17 47 46 24 20 33 32 31 34 04 07 51 49 35 27 21 08 36 N = 54

The worst-case linear time selection: partition Divide N elements into N/5 groups of 5 elements each, plus extra. Brute force sort each of the 5-element groups. 22 45 14 28 29 44 39 09 23 10 52 50 05 06 38 11 15 03 26 37 53 25 54 40 02 12 16 30 19 55 13 41 48 01 18 43 17 47 46 24 20 33 32 31 34 04 07 51 49 35 27 21 08 36 22 45 29 28 14 10 44 39 23 09 06 52 50 38 05 11 37 26 15 03 25 54 53 40 02 16 53 30 19 12 13 48 41 18 01 24 47 46 43 17 31 34 33 32 20 07 51 49 35 04

The worst-case linear time selection: partition Divide N elements into N/5 groups of 5 elements each, plus extra. Brute force sort each of the 5-element groups. Find x = "median of medians" by Select() on N/5 medians. 14 09 05 03 02 12 01 17 20 04 36 32 is also median the algorithm could return 22 10 06 11 25 16 13 24 31 07 27 28 28 23 38 15 40 19 18 43 32 35 08 29 39 50 26 53 30 41 46 33 49 21 45 44 52 37 54 53 48 47 34 51

The worst-case linear time selection: partition Divide N elements into N/5 groups of 5 elements each, plus extra. Brute force sort each of the 5-element groups. Find x = "median of medians" by Select() on N/5 medians. Partition the array using x 14 09 05 03 02 12 01 17 20 04 36 32 is also median the algorithm could return 22 10 06 11 25 16 13 24 31 07 27 28 28 23 38 15 40 19 18 43 32 35 08 29 39 50 26 53 30 41 46 33 49 21 45 44 52 37 54 53 48 47 34 51

The worst-case linear time selection: partition Divide N elements into N/5 groups of 5 elements each, plus extra. Brute force sort each of the 5-element groups. Find x = "median of medians" by Select() on N/5 medians. Partition the array using x , follow 3 cases in Rand_Select() 14 09 05 03 02 12 01 17 20 04 36 32 is also median the algorithm could return 22 10 06 11 25 16 13 24 31 07 27 28 28 23 38 15 40 19 18 43 32 35 08 29 39 50 26 53 30 41 46 33 49 21 45 44 52 37 54 53 48 47 34 51

Selection Analysis Crux of proof: delete roughly 30% of elements by partitioning. At least 1/2 of 5 element medians  x at least  N / 5 / 2 = N / 10 medians  x 14 09 05 03 02 12 01 17 20 04 36 CLR Instructors manual median of medians 22 10 06 11 25 16 13 24 31 07 27 28 23 38 15 40 19 18 43 32 35 08 29 39 50 26 53 30 41 46 33 49 21 45 44 52 37 54 53 48 47 34 51

Selection Analysis Crux of proof: delete roughly 30% of elements by partitioning. At least 1/2 of 5 element medians  x at least  N / 5 / 2 = N / 10 medians  x At least 3 N / 10 elements  x. 14 09 05 03 02 12 01 17 20 04 36 median of medians 22 10 06 11 25 16 13 24 31 07 27 28 23 38 15 40 19 18 43 32 35 08 29 39 50 26 53 30 41 46 33 49 21 45 44 52 37 54 53 48 47 34 51

Selection Analysis Crux of proof: delete roughly 30% of elements by partitioning. At least 1/2 of 5 element medians  x at least  N / 5 / 2 = N / 10 medians  x At least 3 N / 10 elements  x. At least 3 N / 10 elements  x. 14 09 05 03 02 12 01 17 20 04 36 median of medians 22 10 06 11 25 16 13 24 31 07 27 28 23 38 15 40 19 18 43 32 35 08 29 39 50 26 53 30 41 46 33 49 21 45 44 52 37 54 53 48 47 34 51

Selection Analysis Crux of proof: delete roughly 30% of elements by partitioning. At least 1/2 of 5 element medians  x at least  N / 5 / 2 = N / 10 medians  x At least 3 N / 10 elements  x. At least 3 N / 10 elements  x.  Select() called recursively (Case 2 or 3) with at most N - 3 N / 10 elements. C(N) = # comparisons on a file of size N. Now, solve recurrence. Apply master theorem? Assume N is a power of 2? Assume C(N) is monotone non-decreasing? would be nice to assume C(N) is a power of 5 and a power of 10 to eliminate floors, but unfortunately not many such integers :) CLRS assumes that C(N) is monotone. Theirs is (I think). Ours is not.

Selection Analysis Analysis of selection recurrence. T(N) = # comparisons on a file of size  N. T(N) is monotone, but C(N) is not! Claim: T(N)  20cN. Base case: N < 50. Inductive hypothesis: assume true for 1, 2, . . . , N-1. Induction step: for N  50, we have: For n  50, 3 N / 10  N / 4.

Linear Time Selection Postmortem Practical considerations. Constant (currently) too large to be useful. Practical variant: choose random partition element. O(N) expected running time ala quicksort. Open problem: guaranteed O(N) with better constant. Quicksort. Worst case O(N log N) if always partition on median. Justifies practical variants: median-of-3, median-of-5.