Computer Science 112 Fundamentals of Programming II Searching, Sorting, and Complexity Analysis

Things to Desire in a Program Correctness Robustness Maintainability Efficiency

Measuring Efficiency Empirical - use clock to get actual running times for different inputs Problems: –Different machines have different running times –Some running times are so long that it is impractical to check them

Measuring Efficiency Analytical - use pencil and paper to determine the abstract amount of work that a program does for different inputs Advantages: –Machine independent –Can predict running times of programs that are impractical to run

Complexity Analysis Pick an instruction that will run most often in the code Determine the number of times this instruction will be executed as a function of the size of the input data Focus on abstract units of work, not actual running time

Example: Search for the Minimum def ourMin(lyst): minSoFar = lyst[0] for item in lyst: minSoFar = min(minSoFar, item) return minSoFar We focus on the assignment ( = ) inside the loop and ignore the other instructions. for a list of length 1, one assignment for a list of length 2, 2 assignments. for a list of length n, n assignments

Big-O Notation Big-O notation expresses the amount of work a program does as a function of the size of the input O(N) stands for order of magnitude N, or order of N for short Search for the minimum is O(N), where N is the size of the input (a list, a number, etc.)

Common Orders of Magnitude ConstantO(k) LogarithmicO(log 2 n) LinearO(n) QuadraticO(n 2 ) ExponentialO(k n )

Graphs of O(n) and O(n 2 )

Common Orders of Magnitude n O(log 2 n) O(n) O(n 2 )O(2 n ) digits yikes!

Suppose an algorithm requires exactly 3N + 3 steps As N gets very large, the difference between N and N + K becomes negligible (where K is a constant) As N gets very large, the difference between N and N / K or N * K also becomes negligible Use the highest degree term in a polynomial and drop the others (N 2 – N)/2  N 2 Approximations

Example Approximations n O(n) O(n) + 2 O(n 2 ) O(n 2 ) + n

def ourIn(target, lyst): for item in lyst: if item == target: return True # Found target return False # Target not there Example: Sequential Search Which instruction do we pick? How fast is its rate of growth as a function of n? Is there a worst case and a best case? An average case?

Improving Search Assume data are in ascending order Goto midpoint and look there Otherwise, repeat the search to left or to right of midpoint target 3 midpoint leftright

Improving Search Assume data are in ascending order Goto midpoint and look there Otherwise, repeat the search to left or to right of midpoint target 5 midpoint rightleft

Example: Binary Search def ourIn(target, sortedLyst): left = 0 right = len(sortedLyst) - 1 while left <= right: midpoint = (left + right) // 2 if target == sortedLyst[midpoint]: return True elif target < sortedLyst[midpoint]: right = midpoint - 1 else: left = midpoint + 1 return False

Analysis while left <= right: midpoint = (left + right) // 2 if target == sortedLyst[midpoint]: return True elif target < sortedLyst[midpoint]: right = midpoint - 1 else: left = midpoint + 1 How many times will == be executed in the worst case?

Sorting a List sort

Selection Sort For each position i in the list –Select the smallest element from i to n - 1 –Swap it with the ith one

Trace smallest i Step 1: find the smallest element i Step 2: swap with first element i Step 3: advance i and goto step 1

for each i from 0 to n - 1 minIndex = minInRange(lyst, i, n) if minIndex != i swap(lyst, i, minIndex) Design of Selection Sort minInRange returns the index of the smallest element swap exchanges the elements at the specified positions

def selectionSort(lyst): n = len(lyst) for i in range(n): minIndex = minInRange(lyst, i, n) if minIndex != i: swap(lyst, i, minIndex) Implementation

def minInRange(lyst, i, n): minValue = lyst[i] minIndex = i for j in range(i, n): if lyst[j] < minValue: minValue = lyst[j] minIndex = j return minIndex def selectionSort(lyst): n = len(lyst) for i in range(n): minIndex = minInRange(lyst, i, n) if minIndex != i: swap(lyst, i, minIndex)

Implementation def swap(lyst, i, j): lyst[i], lyst[j] = lyst[j], lyst[i] def minInRange(lyst, i, n): minValue = lyst[i] minIndex = i for j in range(i, n): if lyst[j] < minValue: minValue = lyst[j] minIndex = j return minIndex def selectionSort(lyst): n = len(lyst) for i in range(n): minIndex = minInRange(lyst, i, n) if minIndex != i: swap(lyst, i, minIndex)

Analysis of Selection Sort The main loop runs approximately n times Thus, the function minInRange runs n times Within the function minInRange, a loop runs n - i times

Analysis of Selection Sort Overall, the number of comparisons performed in function minInRange is n n n = (n 2 – n) / 2  n 2

For Thursday Finding Faster Algorithms