1. Elementary Data Structures and Algorithms
Algorithm Analysis
Chris Kiekintveld
CS 2401 (Fall 2010)
Elementary Data Structures and Algorithms

2. Algorithm Analysis
There are many different algorithms to solve the same problem
Ask 5 programmers to write a non-trivial program, you will get 5 different solutions
Which is best?
Correctness
Efficiency
Java Programming: Program Design Including Data Structures

3. Computational Resources
Algorithms require resources to run
Time (processor operations)
Space (computer memory)
Network bandwidth
Programmer time
Two types of costs
Fixed: same every time we run the algorithm
Variable: depends on the size of the input
Java Programming: Program Design Including Data Structures

4. Measuring Resource Use
How can we compare the resources used by different algorithms?
Empirical
Code both algorithms
Run them an record the resources used
You did this in the Fibonacci lab!
Java Programming: Program Design Including Data Structures

5. Empirical Analysis Problems
Depends on code quality/implementation
Better/worse programmers, not the algorithm itself
Depends on computer speed/architecture
Depends on language/compiler efficiency
Depends on the input
E.g. linear search is very fast for some inputs, and very slow for others
Java Programming: Program Design Including Data Structures

6. Analytical Approach Analyze the algorithm itself
Abstract away from implementation details
How many operations will be executed?
How much memory is used?
Consider different cases (depending on input)
Best
Worst
Average
Java Programming: Program Design Including Data Structures

7. Counting Operations
int i = 2; int j = 2; int k = i + j; System.out.println(k); How many operations are there? Assignment: Addition: Print: Total:
Java Programming: Program Design Including Data Structures

8. Counting Operations
int i = 2; int j = 2; int k = i + j; System.out.println(i+j+k); How many operations are there? Assignment: Addition: Print: Total:
Java Programming: Program Design Including Data Structures

9. Counting Operations
int i = 0; while (i < 10) { System.out.println(i); i++; } How many operations are there? Assignment: Comparison: Increment: Print: Total:
Java Programming: Program Design Including Data Structures

10. Counting Operations
for (int i=0; i < 10; i++) { System.out.println(i); } How many operations are there? Assignment: Comparison: Increment: Print: Total:
Java Programming: Program Design Including Data Structures

11. Counting Operations
for (int i=0; i < n; i++) { System.out.println(i); } How many operations are there? Assignment: Comparison: Increment: Print: Total:
Java Programming: Program Design Including Data Structures

12. Counting Operations
for (int i=0; i < n; i++) { for (int j=0; j < n; j++) { System.out.println(i); } How many operations are there? Assignment: Comparison: Increment: Print: Total:
Java Programming: Program Design Including Data Structures

13. Counting Operations So far, we have counted every operation
This is quite tedious, especially for infrequent operations
Focus on the most important operation
Most frequent
May need to figure out what this is
Java Programming: Program Design Including Data Structures

14. Another Look at Search Algorithms
We have discussed two ways to search a list
Linear search (unordered data)
Binary search (sorted data)
Data is sorted by “keys”
Unique for each element
Well-defined order
Java Programming: Program Design Including Data Structures

15. Linear (Sequential) Search
public int seqSearch(T[] list, int length, T searchItem) {
int loc;
boolean found = false;
for (loc = 0; loc < length; loc++)
if (list[loc].equals(searchItem))
found = true;
break; }
if (found)
return loc;
else
return -1;
Java Programming: Program Design Including Data Structures

16. Sequential Search Analysis
The statements in the for loop are repeated several times
For each iteration of the loop, the search item is compared with an element in the list
When analyzing a search algorithm, you count the number of comparisons
Suppose that L is a list of length n
The number of key comparisons depends on where in the list the search item is located
Java Programming: Program Design Including Data Structures

17. Sequential Search Analysis (continued)
Best case
The item is the first element of the list
You make only one key comparison
Worst case
The item is the last element of the list
You make n key comparisons
What is the average case
Java Programming: Program Design Including Data Structures

18. Sequential Search Analysis (continued)
To determine the average case
Consider all possible cases
Find the number of comparisons for each case
Add them and divide by the number of cases
Average case
On average, a successful sequential search searches half the list
Java Programming: Program Design Including Data Structures

19. Binary Search
public int binarySearch(T[] list, int length, T searchItem) {
int first = 0;
int last = length - 1;
int mid = -1;
boolean found = false;
while (first <= last && !found)
mid = (first + last) / 2;
Comparable<T> compElem = (Comparable<T>) list[mid];
Java Programming: Program Design Including Data Structures

20. Binary Search (continued)
if (compElem.compareTo(searchItem) == 0)
found = true;
else
if (compElem.compareTo(searchItem) > 0)
last = mid - 1;
first = mid + 1; }
if (found)
return mid;
return -1;
}//end binarySearch
Java Programming: Program Design Including Data Structures

21. Binary Search Example Figure 18-1 Sorted list for a binary search
Table 18-1 Values of first, last, and middle and the Number of
Comparisons for Search Item 89
Java Programming: Program Design Including Data Structures

22. Performance of Binary Search
Suppose that L is a sorted list of size n
And n is a power of 2 (n = 2m)
After each iteration of the for loop, about half the elements are left to search
The maximum number of iteration of the for loop is about m + 1
Also m = log2n
Each iteration makes two key comparisons
Maximum number of comparisons: 2(m + 1)
Java Programming: Program Design Including Data Structures

23. Comparison: Linear vs Binary Worst case number of comparison
List Size
Linear
Binary 4 6 8 32 12
512 20 42
Java Programming: Program Design Including Data Structures

24. Asymptotic Analysis: Motivation
So far, we have counted operations exactly
We don’t really care about the details
Computers execute billions of operations per second
A few here or there is negligible
Care about overall scalability
As the input size grows, does computation grow quickly or slowly?
Don’t lose the forest for the trees
Java Programming: Program Design Including Data Structures

25. Asymptotic Analysis
Asymptotic means the study of the function f as n becomes larger and larger without bound
Consider functions g(n) = n2 and f(n) = n2 + 4n + 20
As n becomes larger and larger, the term 4n + 20 in f(n) becomes insignificant
g(1000) = 1,000,000 and f(1000) = 1,004,020
You can predict the behavior of f(n) by looking at the behavior of g(n)
Java Programming: Program Design Including Data Structures

26. Asymptotic Algorithm Analysis
Identify a function that describes the growth in runtime as the input gets large
An “upper bound” of sorts on the running time
Typically worst-case, but occasionally average case
Describe the number of operations done using a function
Focus only on most important operations
Ignore one time initializations, etc.
Java Programming: Program Design Including Data Structures

27. Common Asymptotic Functions
Table 18-4 Growth Rate of Various Functions
Java Programming: Program Design Including Data Structures

28. Common Functions, visual
Figure 18-9 Growth Rate of Various Functions
Java Programming: Program Design Including Data Structures

29. Java Programming: Program Design Including Data Structures

30. Asymptotic Notation: Big-O Notation (continued)
Table 18-7 Some Big-O Functions That Appear in Algorithm Analysis
Java Programming: Program Design Including Data Structures

31. Big-Oh Notation (Definition)
A function f(n) is O(g(n)) if there exist positive constants c and n0 such that:
f(n) ≤ cg(n) for all n ≥ n0
Java Programming: Program Design Including Data Structures

32. Big-Oh Notation
Translation: After some point, f(n) is always smaller than g(n)
“Some point” refers to increasing problem size
The constant c says that we don’t care about multipliers
So, 2n and n have the same essential growth rate
2n is O(n)
Java Programming: Program Design Including Data Structures

33. Asymptotic Notation: Big-O Notation (continued)
Table 18-8 Number of Comparisons for a List of Length n
Java Programming: Program Design Including Data Structures