Princeton University COS 423 Theory of Algorithms Spring 2001 Kevin Wayne Average Case Analysis

2 Beyond Worst Case Analysis Worst-case analysis. n Analyze running time as function of worst input of a given size. Average case analysis. n Analyze average running time over some distribution of inputs. n Ex: quicksort. Amortized analysis. n Worst-case bound on sequence of operations. n Ex: splay trees, union-find. Competitive analysis. n Make quantitative statements about online algorithms. n Ex: paging, load balancing.

3 Average Case Analysis Average case analysis. n Analyze average running time over some distribution of inputs. n Ex: quicksort. – O(N log N) if input is assumed to be in random order. – leads to randomized algorithm with O(N log N) expected running time, independent of input n Major disadvantage: hard to quantify what input distributions will look like in practice.

4 Quicksort Quicksort. n Assume all elements are unique. n Assume input is a random permutation of inputs. n Denote i th largest element by i. void quicksort(Item a[], int left, int right) { int m; if (right > left) { m = partition(a, left, right); quicksort(a, left, m - 1); quicksort(a, m + 1, right); } quicksort.c

5 Quicksort: BST Representation of Pivots 10 pivot elements

6 Probability that i = 2 and j = 7 get compared. n Let x be pivot element that separates i and j. n Case 1: x  {3, 4, 5, 6}  i and j not compared. Quicksort: Average Case Analysis

7 Probability that i = 2 and j = 7 get compared. n Let x be pivot element that separates i and j. n Case 1: x  {3, 4, 5, 6}  i and j not compared. n Case 2: x  {2, 7}  i and j are compared. Quicksort: Average Case Analysis

8 Probability that i = 2 and j = 7 get compared. n Let x be pivot element that separates i and j. n Case 1: x  {3, 4, 5, 6}  i and j not compared. n Case 2: x  {2, 7}  i and j are compared. Quicksort: Average Case Analysis

9 Quicksort: average case analysis. n Expected # of comparisons = Remark: same analysis if we choose partition element uniformly at random. n Running time independent of input. Quicksort: Average Case Analysis

10 Comparison Based Sorting Lower Bound a1 < a2 a1 < a3 a2 < a3 a1 < a3 a2 < a32, 1, 3 2, 3, 13, 2, 11, 3, 23, 1, 2 1, 2, 3 YES NO Decision Tree of Program

11 Comparison Based Sorting Lower Bound Lower bound =  (N log 2 N): applies to comparison-based algorithms. n Tree height h determines worst case running time. n N! different input orderings. n One (or more) leaves corresponds to each ordering. n Binary tree with N! leaves has height: Stirling's formula