1. Computer Science and Software Engineering University of Wisconsin - Platteville 8. Comparison of Algorithms Yan Shi CS/SE 2630 Lecture Notes Part of this note is from “C++ Plus Data Structure” textbook slides

2. Map to Joe’s Diner

3. Which Cost More to Feed?

4. How efficient is an algorithm?  Efficiency concerns: —CPU usage (time complexity) —memory usage (space complexity) —disk usage —network usage We discuss time complexity in this course.

5. Order of Magnitude of a Function  The order of magnitude, or Big-O notation, of a function expresses the computing time of a problem as the term in a function that increases most rapidly relative to the size of a problem.  Big-O is used to express time complexity of an algorithm

6. Example  How many time do we add to sum? —outer loop repeat n times —each iteration add i times to sum —0+1+2+…+n-1 = n(n-1)/2  The time required by an algorithm is proportional to the number of “basic operations” that it performs!  Big-O: measure growth —O( 0.5n 2 – 0.5n ) = O( n 2 – n ) = O(n 2 ) int sum = 0, n = 100; for(int i = 0; i < n; ++i) for(int j = 0; j < i; ++j) sum += i * j;

7. Comparison of Two Algorithms sum of consecutive integers 1 to 20 (n) O(n)O(1)

8. Types of Complexity  Best case complexity: —related to minimum # of steps required by an algorithm, given an ideal set of inputs  Worst case complexity: —related to maximum # of steps required by an algorithm, given the worst possible set of inputs  Average case complexity: —related to the average number of steps required by an algorithm, calculated across all possible sets of inputs this is the one we usually use.

9. Big-O Formal Definition  A function T(N) is O(F(N)) if for some constant c and for all values of N greater than some value n 0 : T(N) <= c * F(N)  Example: T(N) = 4*N 2 +7  it is O(N 2 ) why? when c = 5, n 0 = 2, T(N) <= c * F(N)

10. Finding “John Smith” Best case: O(1) Worst case: O(N) Average case: O(N) average # of steps: (1+2+…+N)/N = (N+1)/2 How about algorithm 2?

11. Binary Search Example  Look for 1 in sorted list 1-64. [1-32]  [1-16]  [1-8]  [1-4]  [1-2]  [1] —we need to look 6 times: 2 6 = 64 —this is the worst case.  In the worst case, look for an item in a sorted list of size N, we need to look k times —2 k = N  k = log 2 N  O(log 2 N)!

12. Names of Orders of Magnitude O(1) bounded (by a constant) time O(log 2 N) logarithmic time O(N) linear time O(N*log 2 N) N*log 2 N time O(N 2 ) quadratic time O( 2 N ) exponential time

13. N log 2 N N*log 2 N N 2 2 N 1 001 2 2 124 4 4 28 16 16 8 324 64 256 16 4 64 256 65,536 32 5160 1024 4,294,967,296 64 6384 4096 128 7896 16,384 Comparison of Orders

14. http://www.objc.io/issue-7/collections.html

15. How to Determine Complexities  Sequence of statements: total time = time(statement 1) + time(statement 2) +... + time(statement k) statement 1; statement 2;... statement k;

16. How to Determine Complexities  Branch: total time = max(time(sequence 1), time(sequence 2)) if (condition) { sequence of statements 1 } else { sequence of statements 2 }

17. How to Determine Complexities  Loops: total time = N*time(sequence) How about nested loops? for (i = 0; i < N; i++) { sequence of statements }

18. Nested loop example for (i = 0; i < N; i++) for ( j = 0; j < i; j++ ) sequential statements ijtotal#of inner iteration 0none0 101 20,12 … N-10,1,…N-2N-1 total time= (0+1+2+…+N-1)*time(sequence) = N(N-1)/2 * time(sequence)

19. Exercise: what is the Big-O? for ( i = 0 ; i < n ; i++ ) { subtotal = 0; if ( flag == true ) { for( j=i ; j > 0; j++) subtotal += j; tot += subtotal; } else subtotal = -1; } total time = n * ( 1 + max( ( ( 1+2+...+n-1 ) + 1 ), 1 ) )  O(n 2 )

20. Exercise  Array based sorted list: —find(linear search): —insert: —delete:  Sorted linked list: —find: —insert: —delete: O(n) O(n)+O(n) = O(n) O(n) O(n)+O(1) = O(n) How about unsorted list?