Elementary Data Structures and Algorithms Algorithm Analysis Chris Kiekintveld CS 2401 (Fall 2010) Elementary Data Structures and Algorithms

Algorithm Analysis There are many different algorithms to solve the same problem Ask 5 programmers to write a non-trivial program, you will get 5 different solutions Which is best? Correctness Efficiency Java Programming: Program Design Including Data Structures

Computational Resources Algorithms require resources to run Time (processor operations) Space (computer memory) Network bandwidth Programmer time Two types of costs Fixed: same every time we run the algorithm Variable: depends on the size of the input Java Programming: Program Design Including Data Structures

Measuring Resource Use How can we compare the resources used by different algorithms? Empirical Code both algorithms Run them an record the resources used You did this in the Fibonacci lab! Java Programming: Program Design Including Data Structures

Empirical Analysis Problems Depends on code quality/implementation Better/worse programmers, not the algorithm itself Depends on computer speed/architecture Depends on language/compiler efficiency Depends on the input E.g. linear search is very fast for some inputs, and very slow for others Java Programming: Program Design Including Data Structures

Analytical Approach Analyze the algorithm itself Abstract away from implementation details How many operations will be executed? How much memory is used? Consider different cases (depending on input) Best Worst Average Java Programming: Program Design Including Data Structures

Counting Operations int i = 2; int j = 2; int k = i + j; System.out.println(k); How many operations are there? Assignment: Addition: Print: Total: Java Programming: Program Design Including Data Structures

Counting Operations int i = 2; int j = 2; int k = i + j; System.out.println(i+j+k); How many operations are there? Assignment: Addition: Print: Total: Java Programming: Program Design Including Data Structures

Counting Operations int i = 0; while (i < 10) { System.out.println(i); i++; } How many operations are there? Assignment: Comparison: Increment: Print: Total: Java Programming: Program Design Including Data Structures

Counting Operations for (int i=0; i < 10; i++) { System.out.println(i); } How many operations are there? Assignment: Comparison: Increment: Print: Total: Java Programming: Program Design Including Data Structures

Counting Operations for (int i=0; i < n; i++) { System.out.println(i); } How many operations are there? Assignment: Comparison: Increment: Print: Total: Java Programming: Program Design Including Data Structures

Counting Operations for (int i=0; i < n; i++) { for (int j=0; j < n; j++) { System.out.println(i); } How many operations are there? Assignment: Comparison: Increment: Print: Total: Java Programming: Program Design Including Data Structures

Counting Operations So far, we have counted every operation This is quite tedious, especially for infrequent operations Focus on the most important operation Most frequent May need to figure out what this is Java Programming: Program Design Including Data Structures

Another Look at Search Algorithms We have discussed two ways to search a list Linear search (unordered data) Binary search (sorted data) Data is sorted by “keys” Unique for each element Well-defined order Java Programming: Program Design Including Data Structures

Linear (Sequential) Search public int seqSearch(T[] list, int length, T searchItem) { int loc; boolean found = false; for (loc = 0; loc < length; loc++) if (list[loc].equals(searchItem)) found = true; break; } if (found) return loc; else return -1; Java Programming: Program Design Including Data Structures

Sequential Search Analysis The statements in the for loop are repeated several times For each iteration of the loop, the search item is compared with an element in the list When analyzing a search algorithm, you count the number of comparisons Suppose that L is a list of length n The number of key comparisons depends on where in the list the search item is located Java Programming: Program Design Including Data Structures

Sequential Search Analysis (continued) Best case The item is the first element of the list You make only one key comparison Worst case The item is the last element of the list You make n key comparisons What is the average case Java Programming: Program Design Including Data Structures

Sequential Search Analysis (continued) To determine the average case Consider all possible cases Find the number of comparisons for each case Add them and divide by the number of cases Average case On average, a successful sequential search searches half the list Java Programming: Program Design Including Data Structures

Binary Search public int binarySearch(T[] list, int length, T searchItem) { int first = 0; int last = length - 1; int mid = -1; boolean found = false; while (first <= last && !found) mid = (first + last) / 2; Comparable<T> compElem = (Comparable<T>) list[mid]; Java Programming: Program Design Including Data Structures

Binary Search (continued) if (compElem.compareTo(searchItem) == 0) found = true; else if (compElem.compareTo(searchItem) > 0) last = mid - 1; first = mid + 1; } if (found) return mid; return -1; }//end binarySearch Java Programming: Program Design Including Data Structures

Binary Search Example Figure 18-1 Sorted list for a binary search Table 18-1 Values of first, last, and middle and the Number of Comparisons for Search Item 89 Java Programming: Program Design Including Data Structures

Performance of Binary Search Suppose that L is a sorted list of size n And n is a power of 2 (n = 2m) After each iteration of the for loop, about half the elements are left to search The maximum number of iteration of the for loop is about m + 1 Also m = log2n Each iteration makes two key comparisons Maximum number of comparisons: 2(m + 1) Java Programming: Program Design Including Data Structures

Comparison: Linear vs Binary Worst case number of comparison List Size Linear Binary 4 6 8 32 12 512 20 1048576 42 Java Programming: Program Design Including Data Structures

Asymptotic Analysis: Motivation So far, we have counted operations exactly We don’t really care about the details Computers execute billions of operations per second A few here or there is negligible Care about overall scalability As the input size grows, does computation grow quickly or slowly? Don’t lose the forest for the trees Java Programming: Program Design Including Data Structures

Asymptotic Analysis Asymptotic means the study of the function f as n becomes larger and larger without bound Consider functions g(n) = n2 and f(n) = n2 + 4n + 20 As n becomes larger and larger, the term 4n + 20 in f(n) becomes insignificant g(1000) = 1,000,000 and f(1000) = 1,004,020 You can predict the behavior of f(n) by looking at the behavior of g(n) Java Programming: Program Design Including Data Structures

Asymptotic Algorithm Analysis Identify a function that describes the growth in runtime as the input gets large An “upper bound” of sorts on the running time Typically worst-case, but occasionally average case Describe the number of operations done using a function Focus only on most important operations Ignore one time initializations, etc. Java Programming: Program Design Including Data Structures

Common Asymptotic Functions Table 18-4 Growth Rate of Various Functions Java Programming: Program Design Including Data Structures

Common Functions, visual Figure 18-9 Growth Rate of Various Functions Java Programming: Program Design Including Data Structures

Java Programming: Program Design Including Data Structures

Asymptotic Notation: Big-O Notation (continued) Table 18-7 Some Big-O Functions That Appear in Algorithm Analysis Java Programming: Program Design Including Data Structures

Big-Oh Notation (Definition) A function f(n) is O(g(n)) if there exist positive constants c and n0 such that: f(n) ≤ cg(n) for all n ≥ n0 Java Programming: Program Design Including Data Structures

Big-Oh Notation Translation: After some point, f(n) is always smaller than g(n) “Some point” refers to increasing problem size The constant c says that we don’t care about multipliers So, 2n and n have the same essential growth rate 2n is O(n) Java Programming: Program Design Including Data Structures

Asymptotic Notation: Big-O Notation (continued) Table 18-8 Number of Comparisons for a List of Length n Java Programming: Program Design Including Data Structures