1. 10 Algorithms in 20th Century Science, Vol. 287, No. 5454, p. 799, February 2000 Computing in Science & Engineering, January/February 2000 1946: The Metropolis Algorithm for Monte Carlo 1947: Simplex Method for Linear Programming // Com S 477/577, 418/518 1950: Krylov Subspace Iteration Method 1951: The Decompositional Approach to Matrix Computations 1957: The Fortran Optimizing Compiler 1959: QR Algorithm for Computing Eigenvalues 1962: Quicksort Algorithms for Sorting 1965: Fast Fourier Transform // Com S 477/577 1977: Integer Relation Detection 1987: Fast Multipole Method With the greatest influence on the development and practice of science and engineering …

2. Quicksort C. A. R. Hoare 1962 Best practical sorting algorithm! p r pq prqq+1r A divide conquer After partition, A[i]  A[j], p  i  q and q+1  j  r. No work is needed on combining A[p..q] and A[q+1..r]!

3. How to Partition? 6 2 7 4 9 5 10 pivot x i = p–1p j = r+1r 6 2 7 4 9 5 10 j i  x x  x x 5 2 7 4 9 6 10 ij

4. Partitioning 5 2 7 4 9 6 10 ij  x x  x x 5 2 4 7 9 6 10 ji jiq = A[p..q] A[q+1..r]

5. The Procedure Partition(A, p, r) x  A[p] i  p – 1 j  r + 1 while true do repeat j  j – 1 until A[j]  x repeat i  i + 1 until A[i]  x if i < j then A[i]  A[j] else return j // different from the procedure in the textbook

6. The Quicksort Algorithm Quicksort(A, p, r) if p < r then q  Partition(A, p, r) Quicksort(A, p, q) Quicksort(A, q+1, r)

7. Performance of Quicksort Depends on how balanced the partitioning is. Worst Case: p = q < r Let n = r – p + 1 n 2 … n – 1 1 1 1 1 n – 2 1  (n ) time 2

8. A Worst-Case Example The input array is already sorted. 2, 4, 5, 9, 11 2 4, 5, 9, 11 4 5, 9, 11 5 9, 11 9 11

9. Best Case q – p + 1 = (r – p + 1) / 2 recursion tree n n/2 n/4 n/4 cost cn lg n  (n lg n)

10. Balanced Partitioning Suppose always a 9-to-1 proportional split. T(n) = T(9n/10) + T(n/10) + n n n/10 9n/10 n/100 9n/100 9n/100 81n/100 81n/1000 729n/1000 1 1 cn  cn log n 10 log n 10/9 T(n) =  (n lg n)

11. Average Case Imagine a bad split followed by a good split. n 1 n – 1 (n – 1) / 2 Cost of two-step partitioning: n + n – 1 =  (n) n (n – 1) / 2 + 1 (n – 1) / 2 similar to Cost of one-step partitioning: n =  (n) T(n) =  (n lg n)

12. Comparison of Sorting Methods Method Space Average Max n=16 n=10000 Insertion sort n(1+  ) 1.25n + 13.25n 2.5n 433 1248615 Merge sort n(1+  ) 14.43n lnn + 4.92n 14.4n lnn 761 104716 Heapsort n 23.08n lnn + 0.01n 24.5n lnn 1068 159714 Quicksort n + 2  lgn 11.67n lnn – 1.74n  2n 470 81486 Median-of-3 n + 2  lgn 10.63n lnn – 2.12n  n 487 74574 Quicksort 2 2 22 (Use the median of three randomly picked elements as the pivot.) D. E. Knuth, “The Art of Computer Programming”, Vol 3, 2 nd ed., p.382, 1998. Implemented on the MIX Computer.

13. Issue with Quicksort Quicksort has the best average-case behavior. All permutations of the input sequence are assumed to be equally likely. Not necessarily true in practice! Make its behavior independent of the input ordering!

14. Randomization An algorithm is randomized if its behavior depends on  the input  random numbers No particular input causes its worst-case behavior. Choose the pivot at random! Randomized-Partition(A, p, r) i  Random(p, r) A[p]  A[i] return Partition(A, p, r)

15. An Example 6 2 7 4 9 5 10 p r Execution 1: Random(p, r) = p + 3 4 2 7 6 9 5 10 iijjjjjj 2 4 7 6 9 5 10 j

16. Cont’d 6 2 7 4 9 5 10 p r Execution 2: Random(p, r) = p + 2 7 2 6 4 9 5 10 iijjjjiiii 5 2 6 4 9 7 10 ijj

17. Randomized Quicksort Randomized-Quicksort(A, p, r) if p < r then q  Randomized-Partition(A, p, r) Randomized-Quicksort(A, p, q) Randomized-Quicksort(A, q+1, r) A random sequence of good and bad choices can yield an efficient algorithm. Height of recursion tree:  (lg n).

18. Analysis of Partitioning Assumption for simplification: All elements are distinct. x…… prq A:A: After random partitioning A[k]  x p  k  q A[k]  x q+1  k  r rank(x) is # elements  x

19. Ranks and Probabilities  rank(x) = 1 x p = q  rank(x)  2 …x…… p q p – q + 1 = rank(x) – 1 Probability P{rank(x) = i } = 1/n, 1  i  n = r – p + 1  P{q = p} = 2/n, if rank(x) = 1, 2  P{q – p + 1 = i} = 1/n, if rank(x) = i+1, i = 2, …, n–1

20. Average-case Running Time of Randomized-Quicksort T(n) = (1/n) ( T(1) + T(n – 1) +  ( T(q) + T(n – q) ) ) +  (n) rank(x) = 1 rank(x) = q + 1 T(n) = O(n lg n) But T(n) cannot better the best case running time  (n lg n) of quicksort. (see a separate.pdf handout) Running time is  (n lg n)! q = 1 n – 1 So T(n) =  (n lg n).