1 Lower Bound on Comparison-based Search We have now covered lots of searching methods –Contiguous Data (Arrays) Sequential search Binary Search –Dynamic Data (Linked Lists and Trees) Sequential search on linked lists Search Trees: BST, AVL tree, Splay Tree, B Trees, … Question: –How fast can a searching algorithm be made using only comparison of keys? –That is, what’s the lower bound on comparison- based searching?

2 Comparison (Decision) Trees A comparison tree (also called decision tree or search tree) of an algorithm is obtained by tracing through the actions of the algorithm Represent each key comparison by a vertex of the tree, which we denote by a circle Put the key against which we are comparing the target inside the vertex Branches (lines) drawn down a vertex represent possible outcomes of the comparison and are labeled accordingly

3 Comparison (Decision) Trees When the algorithm terminates, we put either – F (for failure) OR –The location where the target is found at the end of the appropriate branch, which we call a leaf and denote by a square

4 Sequential Search – Flashback // Return the index of the array containing the key or –1 if key not found // int LinearSearch(int A[], int noOfKeys, int key){ for (int k=0; k < noOfKeys; k++){ if (A[k] == key) return k; // Key found. Return the index of the array } //end-for return –1; // Key not found }

5 Comparison Tree for Sequential Search A[0] A[1] A[2] A[3] A[4] A[N] F = = = = = = The # of steps for the algorithm to terminate in a particular search is the # of internal (circular) vertices traversed going from the top of the tree down the appropriate path to a leaf Height = N Means that in the worst case the algorithm will take O(N) steps to terminate. So its running time is O(N)

6 Binary Search - Flashback // Return the index of the array containing the key or –1 if key not found int BinarySearch(int A[], int noOfKeys, int key){ left = 0; right = noOfKeys-1; while (left <= right){ int middle = (left+right)/2; // Index of the key to test against if (A[middle] == key) return middle; // Key found. Return the index else if (key < A[middle]) right = middle – 1; // Eliminate the right side else left = middle+1; // Eliminate the left side } //end-while return –1; // Key not found } //end-BinarySearch

7 Comparison Tree for Binary Search A[3]A[1]A[5]A[0]A[2] A[1] A[0] F F A[3] A[2] F F A[4]A[6] A[5] A[4] F F A[6] F F A: left right Height = log 2 (7) = 3 = = = = = = = < < < < < < < > > > >> > >

8 Comparison Tree for Binary Search A[middle] = Target less than key in the middle: Search (left, right = middle-1) Target greater than key in the middle: Search (left=middle+1, right) Since we divide the data set in half with each comparison, the maximum height of the comparison tree will be log 2 N. So the running time of binary search is O(log 2 N). Height = log 2 (N) < >

9 Lower Bound on Comparison-based Algorithms Theorem: –Suppose that an algorithm uses ONLY comparison of keys to solve a problem. –If there are “n” possible outcomes, then the algorithm must take AT LEAST log 2 N comparison of keys in the worst case to solve the problem.

10 Lower Bound on Comparison-based Algorithms – Informal Proof O1 O2 C O3 O4 C O5 O6 C O n-1 OnOn C C O5 O6 C ………………… C C ……………………………………………. C …………………. Height = log 2 (N) Notice that the comparison tree that would result in the smallest height is a complete binary tree. The height of a complete binary tree of N leaves is log 2 N This completes the informal proof.

11 Lower Bound on Comparison Based Search Let’s take comparison-based searching problem: –Search for a key in a data set consisting of N keys using only comparison of keys –How many possible outcomes? N + 1: N successes, 1 failure –Then, the running time of ANY comparison-based search algorithm is (log 2 N)