8. Comparison of Algorithms Yan Shi CS/SE 2630 Lecture Notes Part of this note is from “C++ Plus Data Structure” textbook slides

Map to Joe’s Diner

Which Cost More to Feed?

How efficient is an algorithm? Efficiency concerns: CPU usage (time complexity) memory usage (space complexity) disk usage network usage We discuss time complexity in this course.

Order of Magnitude of a Function The order of magnitude, or Big-O notation, of a function expresses the computing time of a problem as the term in a function that increases most rapidly relative to the size of a problem. Big-O is used to express time complexity of an algorithm

Example How many time do we add to sum? int sum = 0, n = 100; for(int i = 0; i < n; ++i) for(int j = 0; j < i; ++j) sum += i * j; How many time do we add to sum? outer loop repeat n times each iteration add i times to sum 0+1+2+…+n-1 = n(n-1)/2 The time required by an algorithm is proportional to the number of “basic operations” that it performs! Big-O: measure growth O( 0.5n2 – 0.5n ) = O( n2 – n ) = O(n2) Note that the big-O expressions do not have constants or low-order terms. This is because, when N gets large enough, constants and low-order terms don't matter.

Comparison of Two Algorithms sum of consecutive integers 1 to 20 (n) O(n) O(1)

Types of Complexity Best case complexity: Worst case complexity: related to minimum # of steps required by an algorithm, given an ideal set of inputs Worst case complexity: related to maximum # of steps required by an algorithm, given the worst possible set of inputs Average case complexity: related to the average number of steps required by an algorithm, calculated across all possible sets of inputs this is the one we usually use.

Big-O Formal Definition A function T(N) is O(F(N)) if for some constant c and for all values of N greater than some value n0: T(N) <= c * F(N) Example: T(N) = 4*N2+7  it is O(N2) why? when c = 5, n0 = 2, T(N) <= c * F(N)

Finding “John Smith” Best case: O(1) How about algorithm 2? Worst case: O(N) Average case: O(N) average # of steps: (1+2+…+N)/N = (N+1)/2

Binary Search Example Look for 1 in sorted list 1-64. [1-32]  [1-16][1-8][1-4][1-2][1] we need to look 6 times: 26 = 64 this is the worst case. In the worst case, look for an item in a sorted list of size N, we need to look k times 2k = N  k = log2N  O(log2N)!

Names of Orders of Magnitude O(1) bounded (by a constant) time O(log2N) logarithmic time O(N) linear time O(N*log2N) N*log2N time O(N2) quadratic time O( 2N ) exponential time

Comparison of Orders N log2N N*log2N N2 2N 1 0 0 1 2 2 1 2 4 4 1 0 0 1 2 2 1 2 4 4 4 2 8 16 16 8 3 24 64 256 16 4 64 256 65,536 32 5 160 1024 4,294,967,296 64 6 384 4096 128 7 896 16,384

Comparison of Orders http://www.objc.io/issue-7/collections.html

How to Determine Complexities Sequence of statements: total time = time(statement 1) + time(statement 2) + ... + time(statement k) statement 1; statement 2; ... statement k;

How to Determine Complexities Branch: total time = max(time(sequence 1), time(sequence 2)) if (condition) { sequence of statements 1 } else { sequence of statements 2 }

How to Determine Complexities Loops: total time = N*time(sequence) How about nested loops? for (i = 0; i < N; i++) { sequence of statements }

Nested loop example for (i = 0; i < N; i++) for ( j = 0; j < i; j++ ) sequential statements i j total#of inner iteration 0 none 0 1 0 1 2 0,1 2 … N-1 0,1,…N-2 N-1 total time = (0+1+2+…+N-1)*time(sequence) = N(N-1)/2 * time(sequence)

Nested loop exercise i j total#of inner iteration for (i = -1; i < N+3; i++) for ( j = N+4; j >= i; j-- ) sequential statements i j total#of inner iteration

Nested loop exercise i j total#of inner iteration for (i = n+8; i > -2; i--) for ( j = 0; j < i+6; j++ ) sequential statements i j total#of inner iteration

Exercise: what is the Big-O? for ( i = 0 ; i < n ; i++ ) { subtotal = 0; if ( flag == true ) { for( j=i ; j > 0; j--) subtotal += j; tot += subtotal; } else subtotal = -1;

Exercise Array based sorted list: Sorted linked list: find(linear search): insert: delete: Sorted linked list: find: O(n) O(n)+O(n) = O(n) O(n)+O(n) = O(n) O(n) O(n)+O(1) = O(n) O(n)+O(1) = O(n) How about unsorted list?