1. 8. Comparison of Algorithms
Yan Shi
CS/SE 2630 Lecture Notes
Part of this note is from “C++ Plus Data Structure” textbook slides

2. Map to Joe’s Diner

3. Which Cost More to Feed?

4. How efficient is an algorithm?
Efficiency concerns:
CPU usage (time complexity)
memory usage (space complexity)
disk usage
network usage
We discuss time complexity in this course.

5. Order of Magnitude of a Function
The order of magnitude, or Big-O notation, of a function expresses the computing time of a problem as the term in a function that increases most rapidly relative to the size of a problem.
Big-O is used to express time complexity of an algorithm

6. Example How many time do we add to sum?
int sum = 0, n = 100;
for(int i = 0; i < n; ++i)
for(int j = 0; j < i; ++j)
sum += i * j;
How many time do we add to sum?
outer loop repeat n times
each iteration add i times to sum
0+1+2+…+n-1 = n(n-1)/2
The time required by an algorithm is proportional to the number of “basic operations” that it performs!
Big-O: measure growth
O( 0.5n2 – 0.5n ) = O( n2 – n ) = O(n2)
Note that the big-O expressions do not have constants or low-order terms. This is because, when N gets large enough, constants and low-order terms don't matter.

7. Comparison of Two Algorithms
sum of consecutive integers 1 to 20 (n)
O(n)
O(1)

8. Types of Complexity Best case complexity: Worst case complexity:
related to minimum # of steps required by an algorithm, given an ideal set of inputs
Worst case complexity:
related to maximum # of steps required by an algorithm, given the worst possible set of inputs
Average case complexity:
related to the average number of steps required by an algorithm, calculated across all possible sets of inputs
this is the one we usually use.

9. Big-O Formal Definition
A function T(N) is O(F(N)) if for some constant c and for all values of N greater than some value n0:
T(N) <= c * F(N)
Example:
T(N) = 4*N2+7  it is O(N2) why?
when c = 5, n0 = 2, T(N) <= c * F(N)

10. Finding “John Smith” Best case: O(1) How about algorithm 2?
Worst case: O(N)
Average case: O(N)
average # of steps:
(1+2+…+N)/N = (N+1)/2

11. Binary Search Example Look for 1 in sorted list 1-64.
[1-32]  [1-16][1-8][1-4][1-2][1]
we need to look 6 times: 26 = 64
this is the worst case.
In the worst case, look for an item in a sorted list of size N, we need to look k times
2k = N  k = log2N  O(log2N)!

12. Names of Orders of Magnitude
O(1) bounded (by a constant) time
O(log2N) logarithmic time
O(N) linear time
O(N*log2N) N*log2N time
O(N2) quadratic time
O( 2N ) exponential time

13. Comparison of Orders N log2N N*log2N N2 2N 1 0 0 1 2 2 1 2 4 4
,536
,294,967,296
,384

14. Comparison of Orders

15. How to Determine Complexities
Sequence of statements:
total time = time(statement 1) + time(statement 2)
time(statement k)
statement 1;
statement 2;
...
statement k;

16. How to Determine Complexities
Branch:
total time = max(time(sequence 1), time(sequence 2))
if (condition)
{ sequence of statements 1 }
else
{ sequence of statements 2 }

17. How to Determine Complexities
Loops:
total time = N*time(sequence)
How about nested loops?
for (i = 0; i < N; i++) {
sequence of statements }

18. Nested loop example
for (i = 0; i < N; i++)
for ( j = 0; j < i; j++ )
sequential statements
i j total#of inner iteration 0 none ,1 2 … N-1 0,1,…N-2 N-1 total time = (0+1+2+…+N-1)*time(sequence) = N(N-1)/2 * time(sequence)

19. Nested loop exercise i j total#of inner iteration
for (i = -1; i < N+3; i++)
for ( j = N+4; j >= i; j-- )
sequential statements
i j total#of inner iteration

20. Nested loop exercise i j total#of inner iteration
for (i = n+8; i > -2; i--)
for ( j = 0; j < i+6; j++ )
sequential statements
i j total#of inner iteration

21. Exercise: what is the Big-O?
for ( i = 0 ; i < n ; i++ ) {
subtotal = 0;
if ( flag == true ) {
for( j=i ; j > 0; j--)
subtotal += j;
tot += subtotal; }
else
subtotal = -1;

22. Exercise Array based sorted list: Sorted linked list:
find(linear search):
insert:
delete:
Sorted linked list:
find:
O(n)
O(n)+O(n) = O(n)
O(n)+O(n) = O(n)
O(n)
O(n)+O(1) = O(n)
O(n)+O(1) = O(n)
How about unsorted list?