Order Statistics(Selection Problem) A more interesting problem is selection:  finding the i th smallest element of a set We will show: –A practical randomized algorithm with O(n) expected running time –A cool algorithm of theoretical interest only with O(n) worst-case running time Naive algorithm: Sort and index i-th element T(n) = Θ(nlgn)+Θ(1) = Θ(nlgn) using merge sort or heapsort (not quicksort) BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)1

Selection in Expected Linear Time Randomized algorithm Divide and conquer Similar to randomized quicksort – Like quicksort: Partitions input array recursively – Unlike quicksort: – Only works on one side of the partition – Quicksort works on both sides of the partition – Expected running times: – SELECT: E[n] = Θ(n) – QUICKSORT: E[n] = Θ(nlgn) BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)2

Selection in Expected Linear Time BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)3

Randomized Selection BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)4 Key idea: use partition() from quicksort –But, only need to examine one subarray –This savings shows up in running time: O(n) We will again use a slightly different partition than the book: q = RandomizedPartition(A, p, r)  A[q]  A[q] q pr

Randomized Selection RandomizedSelect(A, p, r, i) if (p == r) then return A[p]; q = RandomizedPartition(A, p, r) k = q - p + 1; if (i == k) then return A[q]; // not in book if (i < k) then return RandomizedSelect(A, p, q-1, i); else return RandomizedSelect(A, q+1, r, i-k); BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)5  A[q]  A[q] k q pr

Randomized Selection ● Analyzing RandomizedSelect() ■ Worst case: partition always 0:n-1 T(n) = T(n-1) + O(n) = O(n 2 ) (arithmetic series) ○ No better than sorting! ■ “Best” case: suppose a 9:1 partition T(n) = T(9n/10) + O(n) = O(n)(Master Theorem, case 3) ○ Better than sorting! BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)6

Randomized Selection ● Average case ■ For upper bound, assume i th element always falls in larger side of partition: ■ Let’s show that T(n) = O(n) by substitution BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)7 What happened here?

Randomized Selection Assume T(n)  cn for sufficiently large c: BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)8 The recurrence we started with Substitute T(n)  cn for T(k) “Split” the recurrence Expand arithmetic series Multiply it out

Randomized Selection BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)9 What happened here? Subtract c/2 What happened here? Assume T(n)  cn for sufficiently large c: The recurrence so far Multiply it out Rearrange the arithmetic What we set out to prove

Worst-Case Linear-Time Selection ● Randomized algorithm works well in practice ● What follows is a worst-case linear time algorithm, really of theoretical interest only ● Basic idea: ■ Generate a good partitioning element ■ Call this element x BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)10

Selection in Worst Case Linear Time SELECT(S, n, i) // return i-th element in set S with n elements if n≤5 then SORT S and return the i-th element DIVIDE S into  n/5  groups first  n/5  groups are of size 5, last group is of size n mod 5 FIND median set M={m 1, …, m  n/5  } m j = median of j-th group x ← SELECT(M,  n/5 ,  n/5  /2  +1) PARTITION set S around the pivot x into L and R if i ≤ |L| then return SELECT(L, |L|, i) else return SELECT(R, n–|L|, i–|L|) BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)11

Choosing the pivot BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)12

Analysis BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)13

Analysis BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)14

Analysis BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)15

Selection in Worst Case Linear Time SELECT(S, n, i) // return i-th element in set S with n elements if n≤5 then SORT S and return the i-th element DIVIDE S into  n/5  groups first  n/5  groups are of size 5, last group is of size n mod 5 FIND median set M={m 1, …, m  n/5  } m j = median of j-th group x ← SELECT(M,  n/5 ,  n/5  /2  +1) PARTITION set S around the pivot x into L and R if i ≤ |L| then return SELECT(L, |L|, i) else return SELECT(R, n–|L|, i–|L|) BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)16

Selection in Worst Case Linear Time BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)17