Order Statistics

Order Statistic ith order statistic: ith smallest element of a set of n elements. Minimum: first order statistic. Maximum: nth order statistic. Median: “half-way point” of the set. Unique, when n is odd – occurs at i = (n+1)/2. Two medians when n is even. Lower median, at i = n/2. Upper median, at i = n/2+1. For consistency, “median” will refer to the lower median.

Selection Problem Selection problem: Input: A set A of n distinct numbers and a number i, with 1 i  n. Ouput: the element x  A that is larger than exactly i – 1 other elements of A. Can be solved in O(n lg n) time. How? We will study linear-time algorithms. For the special cases when i = 1 and i = n. For the general problem.

Minimum (Maximum) No. of comparisons: n – 1. Can we do better? Minimum (A) 1. min  A[1] 2. for i  2 to length[A] 3. do if min > A[i] 4. then min  A[i] 5. return min Maximum can be determined similarly. T(n) = (n). No. of comparisons: n – 1. Can we do better?

Minimum (Maximum) No. of comparisons: n – 1. Minimum (A) 1. min  A[1] 2. for i  2 to length[A] 3. do if min > A[i] 4. then min  A[i] 5. return min Maximum can be determined similarly. T(n) = (n). No. of comparisons: n – 1. Can we do better? Why not? Minimum(A) has worst-case optimal # of comparisons.

Selection Problem Selection problem: Input: A set A of n distinct numbers and a number i, with 1 i  n. Ouput: the element x  A that is larger than exactly i – 1 other elements of A. Seems harder than min or max, but has the same asymptotic running time. QuickMedian: expected linear time (randomized) BFPRT73: linear time (deterministic) Key idea: geometric series sum to linear

Randomized Quicksort: review Quicksort(A, p, r) if p < r then q := Rnd-Partition(A, p, r); Quicksort(A, p, q – 1); Quicksort(A, q + 1, r) fi Rnd-Partition(A, p, r) i := Random(p, r); A[r]  A[i]; x, i := A[r], p – 1; for j := p to r – 1 do if A[j]  x then i := i + 1; A[i]  A[j] fi od; A[i + 1]  A[r]; return i + 1 A[p..r] 5 A[p..q – 1] A[q+1..r] Partition 5  5  5

Selection in Expected Linear Time Key idea: Quicksort, but recur only on list containing i Exploit two abilities of Randomized-Partition (RP). RP returns the index k in the sorted order of a randomly chosen element (pivot). If the order statistic i = k, then we are done. Else reduce the problem size using its other ability. RP rearranges all other elements around the pivot. If i < k, selection can be narrowed down to A[1..k – 1]. Else, select the (i – k)th element from A[k+1..n].

Randomized-Select Randomized-Select(A, p, r, i) // select ith order statistic. 1. if p = r 2. then return A[p] 3. q  Randomized-Partition(A, p, r) 4. k  q – p + 1 5. if i = k 6. then return A[q] 7. elseif i < k 8. then return Randomized-Select(A, p, q – 1, i) 9. else return Randomized-Select(A, q+1, r, i – k)

Randomized-Select Example 8 1 5 3 4 Goal: Find 3rd smallest element

Randomized-Select Example 8 1 5 3 4

Randomized-Select Example 8 1 5 3 4 1 3 8 5 4

Randomized-Select Example 8 1 5 3 4 1 3 8 5 4 8 5 4

Randomized-Select Example 8 1 5 3 4 1 3 8 5 4 8 5 4

Randomized-Select Example 8 1 5 3 4 1 3 8 5 4 4 5 8

Randomized-Select Example 8 1 5 3 4 1 3 8 5 4 4 5 8 4

Analysis Worst-case Complexity: Average-case Complexity: (n2) – As we could get unlucky and always recurse on a subarray that is only one element smaller than the previous subarray. (T(n) = T(n-1) + (n) ) Average-case Complexity: (n) – Intuition: Because the pivot is chosen at random, we expect that we get rid of half of the list each time we choose a random pivot q. Why (n) and not (n lg n)?

Analysis Average-case Complexity - more intuition If we get rid of 10% of the list each time we choose a random pivot q... T(n) = T(9/10 n) + (n)

Analysis Average-case Complexity - more intuition If we get rid of 10% of the list each time we choose a random pivot q... T(n) = T(9/10 n) + (n) Let a  1 and b > 1 ,T(n) = a T(n/b) + c nk, n  0 1. If a > bk, then T(n) = ( nlog_b a ). 2. If a = bk, then T(n) = ( nk lg n ). 3. If a < bk, then T(n) = ( nk ).

Analysis Average-case Complexity - more intuition T(n) = (n) If we get rid of 10% of the list each time we choose a random pivot q... T(n) = T(9/10 n) + (n) T(n) = (n) Let a  1 and b > 1 ,T(n) = a T(n/b) + c nk, n  0 1. If a > bk, then T(n) = ( nlog_b a ). 2. If a = bk, then T(n) = ( nk lg n ). 3. If a < bk, then T(n) = ( nk ).

Selection in Worst-Case Linear Time Algorithm Select: Like RandomizedSelect, finds the desired element by recursively partitioning the input array. Unlike RandomizedSelect, is deterministic. Uses a variant of the deterministic Partition routine. Partition is told which element to use as the pivot. Achieves linear-time complexity in the worst case by Guaranteeing that the split is always “good” at each Partition. How can a good split be guaranteed?

Choosing a Pivot Median-of-Medians: Divide the n elements into n/5 groups. n/5 groups contain 5 elements each. 1 group may contain n mod 5 < 5 elements. Determine the median of each of the groups. Recursively find the median x of the n/5 medians. n/5 groups of 5 elements each. n/5th group of n mod 5 elements.

Example

Algorithm Select Determine the median-of-medians x (on previous slide.) Partition input array around x (Partition from Quicksort). Let k be the index of x that Partition returns. If k = i, then return x. Else if i < k, apply Select recursively to A[1..k–1] to find the ith smallest element. Else if i > k, apply Select recursively to A[k+1..n] to find the (i – k)th smallest element.

Worst-case Split Arrows point from larger to smaller elements. n/5 groups of 5 elements each. Elements < x n/5th group of n mod 5 elements. Median-of-medians, x Elements > x

Worst-case Split Assumption: Elements are distinct. Why? At least  n/5 /2 groups have 3 of their 5 elements ≥ x. Ignore the last group if it has fewer than 5 elements. Hence, the no. of elements ≥ x is at least 3(n–4)/10. Likewise, the no. of elements ≤ x is at least 3(n–4)/10. Thus, in the worst case, Select is called recursively on at most (7n+12)/10 elements.

Recurrence for worst-case running time T(Select)  T(Median-of-medians) + T(Partition) + T(recursive call to select) T(n)  O(n) + T(n/5) + O(n) + T((7n+12)/10) = T(n/5) + T(7n/10+1.2) + O(n) Base: for n  24, assume we just use Insertionsort. So T(n)  24n for all n  24. T(Median-of-medians) T(Partition) T(recursive call)

Solving the recurrence Base: for all n  24, T(n)  24n For n > 24, T(n) ≤ an+ T(n/5) + T(7n/10+1.2) We want to find c>0 so for all n>0 T(n) ≤ cn. Base implies c ≥ 24 T(n) ≤ an+ T(n/5) + T(7n/10+1.2) ?≤ an+ c n/5 + c 7n/10 + 1.2c = cn – (c n/10 – an – 1.2c) = cn – ((c/20 – a)n + (n/20 – 1.2)c) ≤ cn, as long as c ≥ 20a. So, c = max(24, 20a) works

Conclusions We can find the ith largest in an unordered list in Θ(n) worst-case time Let us do Quicksort in worst-case Θ(n lg n) . That constant, 20× partition cost, was high; use RandomizedSelect (aka QuickMedian) in practice.