CS200: Algorithm Analysis

MORE ON SORTING How fast can we sort n items if we use a comparison sort?____________ Comparison sorts require comparison of elements to induce order: Mergesort, Quicksort, Heapsort, Insertion sort...etc. A comparison sort must do at least n comparisons in order to compare n elements: but what about the large gap between n and n log n? Can we do better than n log n?

Why must comparison sorts must be at least n log n? Argument uses a Decision Tree abstraction that represents the comparisons performed by a sorting algorithm on an input of size n. Show the decision tree for insertion sort on A = <a1,a2,a3> (next slide). Note that there are n! leaves in the tree corresponding to all possible permutations of n input elements. Each interior node is annotated with the relationship (≤, >) between ai and aj; elements in the set to be sorted. Each leaf is annotated by a permutation of n.

Tracing Insertion sort requires finding a path from the root to a leaf in the decision tree. Trace Insertion sort for A= <9,2,6>. The longest path for a decision tree that models Insertion sort is Q(n2) and for MergeSort is Q(n log n). This represents the worst-case number of comparisons. A lower bound on the height of a decision tree thus gives a lower bound on the running time of a comparison sort.

Theorem: Any decision tree that sorts n elements has a height of h = W(n log n). Proof: a decision tree has n! leaves. A decision tree must be a binary tree because a split at a sub-root depends on the two relationships; ≤ and >. A binary tree of height h has no more than 2h leaves (by definition, see Appendix B, done in lecture). n! ≤ 2h, lemma proved next, see Appendix B. Take log of both sides, log(n!) ≤ log(2h) h ≥ log(n!) for n! use, n! > (n/e)n By Stirlings approx. h ≥ log(n/e)n log(n/e)n = n log n - n log e n log e is a lower order term. h = W(n log n)

Lemma - Now to prove the lemma: Proof: By induction on h. Basis: h = 0. Tree is just one node, which is a leaf. 2h = 1. Inductive step: Assume true for height = h − 1. Extend tree of height h − 1 by making as many new leaves as possible. Each leaf becomes parent to two new leaves. # of leaves for height h = 2·(# of leaves for height h−1) = 2 · 2h−1 (ind. hypothesis) • • • • = 2h.

By Theorem the following holds: mergesort, heapsort and median partitioned quicksort are optimal sorting algorithms.

Summary Lower bound on sorts Theorem 8.1 and corollary 8.2