Computer Science and Software Engineering University of Wisconsin - Platteville 8. Comparison of Algorithms Yan Shi CS/SE 2630 Lecture Notes Part of this note is from “C++ Plus Data Structure” textbook slides

Map to Joe’s Diner

Which Cost More to Feed?

How efficient is an algorithm?  Efficiency concerns: —CPU usage (time complexity) —memory usage (space complexity) —disk usage —network usage We discuss time complexity in this course.

Order of Magnitude of a Function  The order of magnitude, or Big-O notation, of a function expresses the computing time of a problem as the term in a function that increases most rapidly relative to the size of a problem.  Big-O is used to express time complexity of an algorithm

Example  How many time do we add to sum? —outer loop repeat n times —each iteration add i times to sum —0+1+2+…+n-1 = n(n-1)/2  The time required by an algorithm is proportional to the number of “basic operations” that it performs!  Big-O: measure growth —O( 0.5n 2 – 0.5n ) = O( n 2 – n ) = O(n 2 ) int sum = 0, n = 100; for(int i = 0; i < n; ++i) for(int j = 0; j < i; ++j) sum += i * j;

Comparison of Two Algorithms sum of consecutive integers 1 to 20 (n) O(n)O(1)

Types of Complexity  Best case complexity: —related to minimum # of steps required by an algorithm, given an ideal set of inputs  Worst case complexity: —related to maximum # of steps required by an algorithm, given the worst possible set of inputs  Average case complexity: —related to the average number of steps required by an algorithm, calculated across all possible sets of inputs this is the one we usually use.

Big-O Formal Definition  A function T(N) is O(F(N)) if for some constant c and for all values of N greater than some value n 0 : T(N) <= c * F(N)  Example: T(N) = 4*N 2 +7  it is O(N 2 ) why? when c = 5, n 0 = 2, T(N) <= c * F(N)

Finding “John Smith” Best case: O(1) Worst case: O(N) Average case: O(N) average # of steps: (1+2+…+N)/N = (N+1)/2 How about algorithm 2?

Binary Search Example  Look for 1 in sorted list [1-32]  [1-16]  [1-8]  [1-4]  [1-2]  [1] —we need to look 6 times: 2 6 = 64 —this is the worst case.  In the worst case, look for an item in a sorted list of size N, we need to look k times —2 k = N  k = log 2 N  O(log 2 N)!

Names of Orders of Magnitude O(1) bounded (by a constant) time O(log 2 N) logarithmic time O(N) linear time O(N*log 2 N) N*log 2 N time O(N 2 ) quadratic time O( 2 N ) exponential time

N log 2 N N*log 2 N N 2 2 N , ,294,967, ,384 Comparison of Orders

How to Determine Complexities  Sequence of statements: total time = time(statement 1) + time(statement 2) time(statement k) statement 1; statement 2;... statement k;

How to Determine Complexities  Branch: total time = max(time(sequence 1), time(sequence 2)) if (condition) { sequence of statements 1 } else { sequence of statements 2 }

How to Determine Complexities  Loops: total time = N*time(sequence) How about nested loops? for (i = 0; i < N; i++) { sequence of statements }

Nested loop example for (i = 0; i < N; i++) for ( j = 0; j < i; j++ ) sequential statements ijtotal#of inner iteration 0none ,12 … N-10,1,…N-2N-1 total time= (0+1+2+…+N-1)*time(sequence) = N(N-1)/2 * time(sequence)

Exercise: what is the Big-O? for ( i = 0 ; i < n ; i++ ) { subtotal = 0; if ( flag == true ) { for( j=i ; j > 0; j++) subtotal += j; tot += subtotal; } else subtotal = -1; } total time = n * ( 1 + max( ( ( n-1 ) + 1 ), 1 ) )  O(n 2 )

Exercise  Array based sorted list: —find(linear search): —insert: —delete:  Sorted linked list: —find: —insert: —delete: O(n) O(n)+O(n) = O(n) O(n) O(n)+O(1) = O(n) How about unsorted list?