1. CS200: Algorithm Analysis

2. MORE ON SORTING
How fast can we sort n items if we use a comparison sort?____________
Comparison sorts require comparison of elements to induce order: Mergesort, Quicksort, Heapsort, Insertion sort...etc.
A comparison sort must do at least n comparisons in order to compare n elements: but what about the large gap between n and n log n? Can we do better than n log n?

3. Why must comparison sorts must be at least n log n?
Argument uses a Decision Tree abstraction that represents the comparisons performed by a sorting algorithm on an input of size n.
Show the decision tree for insertion sort on A = <a1,a2,a3> (next slide). Note that there are n! leaves in the tree corresponding to all possible permutations of n input elements.
Each interior node is annotated with the relationship (≤, >) between ai and aj; elements in the set to be sorted.
Each leaf is annotated by a permutation of n.

5. Tracing Insertion sort requires finding a path from the root to a leaf in the decision tree.
Trace Insertion sort for A= <9,2,6>.
The longest path for a decision tree that models Insertion sort is Q(n2) and for MergeSort is Q(n log n). This represents the worst-case number of comparisons.
A lower bound on the height of a decision tree thus gives a lower bound on the running time of a comparison sort.

6. Theorem: Any decision tree that sorts n elements has a height of h = W(n log n).
Proof: a decision tree has n! leaves. A decision tree must be a binary tree because a split at a sub-root depends on the two relationships; ≤ and >. A binary tree of height h has no more than 2h leaves (by definition, see Appendix B, done in lecture).
n! ≤ 2h, lemma proved next, see Appendix B. Take log of both sides,
log(n!) ≤ log(2h)
h ≥ log(n!) for n! use,
n! > (n/e)n By Stirlings approx.
h ≥ log(n/e)n
log(n/e)n = n log n - n log e n log e is a lower order term.
h = W(n log n)

7. Lemma - Now to prove the lemma:
Proof: By induction on h. Basis: h = 0. Tree is just one node, which is a leaf. 2h = 1. Inductive step: Assume true for height = h − 1. Extend tree of height h − 1 by making as many new leaves as possible. Each leaf becomes parent to two new leaves. # of leaves for height h = 2·(# of leaves for height h−1) = 2 · 2h−1 (ind. hypothesis) • • • • = 2h.

8. By Theorem the following holds: mergesort, heapsort and median partitioned quicksort are optimal sorting algorithms.

9. Summary
Lower bound on sorts
Theorem 8.1 and corollary 8.2