1. Randomized Algorithms for Selection and Sorting Prepared by John Reif, Ph.D. Analysis of Algorithms

2. Randomized Algorithms for Selection and Sorting a)Randomized Sampling b)Selection by Randomized Sampling c)Sorting by Randomized Splitting: Quicksort and Multisample Sorts

3. Readings Main Reading Selections: CLR, Chapters 9

4. Comparison Problems Input set X of N distinct keys total ordering < over X Problems 1)For each key x X rank(x,X) = |{x’ X| x’ < x}| + 1 2)For each index i {1, …, N) select (i,X) = the key x X where i = rank(x,X) 3)Sort (X) = (x 1, x 2, …, x n ) where x i = select (i,X)

5. Randomized Comparison Tree Model

6. Algorithm Samplerank s (x,X) begin Let S be a random sample of X-{x} of size s output 1+ N/s [rank(x,S)-1] end

7. Algorithm Samplerank s (x,X) (cont’d) Lemma 1 –The expected value of samplerank s (x,X) is rank (x,X)

8. Algorithm Samplerank s (x,X) (cont’d)

9. S is Random Sample of X of Size s

10. More Precise Bounds on Randomized Sampling (cont’d)

11. More Precise Bounds on Randomized Sampling Let S be a random sampling of X Let r i = rank(select(i,S),X)

12. More Precise Bounds on Randomized Sampling (cont’d) Proof We can bound r i by a Beta distribution, implying Weak bounds follow from Chebychev inequality The Tighter bounds follow from Chernoff Bounds

13. Subdivision by Random Sampling Let S be a random sample of X of size s Let k 1, k 2, …, k s be the elements of S in sorted order These elements subdivide X into

14. Subdivision by Random Sampling (cont’d) How even are these subdivisions?

15. Subdivision by Random Sampling (cont’d) Lemma 3 If random sample S in X is of size s and X is of size N, then S divides X into subsets each of

16. Subdivision by Random Sampling (cont’d) Proof The number of (s+1) partitions of X is The number of partitions of X with one block of size  v is

17. Subdivision by Random Sampling (cont’d) So the probability of a random (s+1) partition having a block size  v is

18. Randomized Algorithms for Selection “canonical selection algorithm” Algorithm can select (i,X) input set X of N keys index i  {1, …, N} [0] if N=1 then output X [1] select a bracket B of X, so that select (i,X) B with high prob. [2] Let i 1 be the number of keys less than any element of B [3] output can select (i-i 1, B) B found by random sampling

19. Hoar’s Selection Algorithm Algorithm Hselect (i,X) where 1  i  N begin if X = {x} then output x else choose a random splitter k  X let B = {x  X|x < k} if |B| i then output Hselect(i,B) else output Hselect(i-|B|, X-B) end

20. Hoar’s Selection Algorithm (cont’d) Sequential time bound T(i,N) has mean

21. Hoar’s Selection Algorithm (cont’d) Random splitter k  X Hselect (i,X) has two cases

22. Hoar’s Selection Algorithm (cont’d) Inefficient: each recursive call requires N comparisons, but only reduces average problem size to ½ N

23. Improved Randomized Selection By Floyd and Rivest Algorithm FRselect(i,X) begin if X = {x} then output x else choose k 1, k 2  X such that k 1 < k 2 let r 1 = rank(k 1, X), r 2 = rank(k 2, X) if r 1 > i then FRselect(i, {x  X|x < k 1 }) else if r 2 > i then FRselect(i-r 1, {x  X|k 1 x k 2 }) else FRselect(i-r 2, {x  X|x > k 2 }) end

24. Choosing k 1, k 2 We must choose k 1, k 2 so that with high likelihood, k 1  select(i,X)  k 2

25. Choosing k 1, k 2 (cont’d) Choose random sample S  X size s Define

26. Choosing k 1, k 2 (cont’d) Lemma 2 implies

27. Expected Time Bound

28. Small Cost of Recursions Note With prob  1 - 2N - each recursive call costs only O(s) = o(N) rather than N in previous algorithm

29. Randomized Sorting Algorithms “canonical sorting algorithm” Algorithm cansort(X) begin if x={x} then output X else choose a random sample S of X of size s Sort S S subdivides X into s+1 subsets X 1, X 2, …, X s+1 output cansort (X 1 ) cansort (X 2 )… cansort(X s+1 ) end

30. Using Random Sampling to Aid Sorting Problem: –Must subdivide X into subsets of nearly equal size to minimize number of comparisons –Solution: random sampling!

31. Hoar’s Randomized Sorting Algorithm Uses sample size s=1 Algorithm quicksort(X) begin if |X|=1 then output X else choose a random splitter k  X output quicksort ({x  X|x k}) end

32. Expected Time Cost of Hoar’s Sort Inefficient: Better to divide problem size by ½ with high likelihood!

33. Improved Sort using Better choice of splitter is k = sample select s (N/2, N) Algorithm samplesort s (X) begin if |X|=1 then output X choose a random subset S of X size s = N/log N k  select (S/2,S) cost time o(N) output samplesort s ({x  X|x k}) end

34. Randomized Approximant of Mean By Lemma 2, rank(k,X) is very nearly the mean:

35. Improved Expected Time Bounds Is optimal for comparison trees!

36. Open Problems in Selection and Sorting 1)Improve Randomized Algorithms to exactly match lower bounds on number of comparisons 2)Can we de randomize these algorithms – i.e., give deterministic algorithms with the same bounds?

37. Randomized Algorithms for Selection and Sorting Prepared by John Reif, Ph.D. Analysis of Algorithms