10 Algorithms in 20th Century Science, Vol. 287, No. 5454, p. 799, February 2000 Computing in Science & Engineering, January/February : The Metropolis Algorithm for Monte Carlo 1947: Simplex Method for Linear Programming // Com S 477/577, 418/ : Krylov Subspace Iteration Method 1951: The Decompositional Approach to Matrix Computations 1957: The Fortran Optimizing Compiler 1959: QR Algorithm for Computing Eigenvalues 1962: Quicksort Algorithms for Sorting 1965: Fast Fourier Transform // Com S 477/ : Integer Relation Detection 1987: Fast Multipole Method With the greatest influence on the development and practice of science and engineering …

Quicksort C. A. R. Hoare 1962 Best practical sorting algorithm! p r pq prqq+1r A divide conquer After partition, A[i]  A[j], p  i  q and q+1  j  r. No work is needed on combining A[p..q] and A[q+1..r]!

How to Partition? pivot x i = p–1p j = r+1r j i  x x  x x ij

Partitioning ij  x x  x x ji jiq = A[p..q] A[q+1..r]

The Procedure Partition(A, p, r) x  A[p] i  p – 1 j  r + 1 while true do repeat j  j – 1 until A[j]  x repeat i  i + 1 until A[i]  x if i < j then A[i]  A[j] else return j // different from the procedure in the textbook

The Quicksort Algorithm Quicksort(A, p, r) if p < r then q  Partition(A, p, r) Quicksort(A, p, q) Quicksort(A, q+1, r)

Performance of Quicksort Depends on how balanced the partitioning is. Worst Case: p = q < r Let n = r – p + 1 n 2 … n – n – 2 1  (n ) time 2

A Worst-Case Example The input array is already sorted. 2, 4, 5, 9, , 5, 9, , 9, ,

Best Case q – p + 1 = (r – p + 1) / 2 recursion tree n n/2 n/4 n/4 cost cn lg n  (n lg n)

Balanced Partitioning Suppose always a 9-to-1 proportional split. T(n) = T(9n/10) + T(n/10) + n n n/10 9n/10 n/100 9n/100 9n/100 81n/100 81n/ n/ cn  cn log n 10 log n 10/9 T(n) =  (n lg n)

Average Case Imagine a bad split followed by a good split. n 1 n – 1 (n – 1) / 2 Cost of two-step partitioning: n + n – 1 =  (n) n (n – 1) / (n – 1) / 2 similar to Cost of one-step partitioning: n =  (n) T(n) =  (n lg n)

Comparison of Sorting Methods Method Space Average Max n=16 n=10000 Insertion sort n(1+  ) 1.25n n 2.5n Merge sort n(1+  ) 14.43n lnn n 14.4n lnn Heapsort n 23.08n lnn n 24.5n lnn Quicksort n + 2  lgn 11.67n lnn – 1.74n  2n Median-of-3 n + 2  lgn 10.63n lnn – 2.12n  n Quicksort (Use the median of three randomly picked elements as the pivot.) D. E. Knuth, “The Art of Computer Programming”, Vol 3, 2 nd ed., p.382, Implemented on the MIX Computer.

Issue with Quicksort Quicksort has the best average-case behavior. All permutations of the input sequence are assumed to be equally likely. Not necessarily true in practice! Make its behavior independent of the input ordering!

Randomization An algorithm is randomized if its behavior depends on  the input  random numbers No particular input causes its worst-case behavior. Choose the pivot at random! Randomized-Partition(A, p, r) i  Random(p, r) A[p]  A[i] return Partition(A, p, r)

An Example p r Execution 1: Random(p, r) = p iijjjjjj j

Cont’d p r Execution 2: Random(p, r) = p iijjjjiiii ijj

Randomized Quicksort Randomized-Quicksort(A, p, r) if p < r then q  Randomized-Partition(A, p, r) Randomized-Quicksort(A, p, q) Randomized-Quicksort(A, q+1, r) A random sequence of good and bad choices can yield an efficient algorithm. Height of recursion tree:  (lg n).

Analysis of Partitioning Assumption for simplification: All elements are distinct. x…… prq A:A: After random partitioning A[k]  x p  k  q A[k]  x q+1  k  r rank(x) is # elements  x

Ranks and Probabilities  rank(x) = 1 x p = q  rank(x)  2 …x…… p q p – q + 1 = rank(x) – 1 Probability P{rank(x) = i } = 1/n, 1  i  n = r – p + 1  P{q = p} = 2/n, if rank(x) = 1, 2  P{q – p + 1 = i} = 1/n, if rank(x) = i+1, i = 2, …, n–1

Average-case Running Time of Randomized-Quicksort T(n) = (1/n) ( T(1) + T(n – 1) +  ( T(q) + T(n – q) ) ) +  (n) rank(x) = 1 rank(x) = q + 1 T(n) = O(n lg n) But T(n) cannot better the best case running time  (n lg n) of quicksort. (see a separate.pdf handout) Running time is  (n lg n)! q = 1 n – 1 So T(n) =  (n lg n).