1. Algorithm Efficiency and Sorting
Chapter 9

2. Chapter 9 -- Algorithm Efficiency and Sorting
This chapter will show you how to analyze the efficiency of algorithms.
The basic mathematical techniques for analyzing algorithms are central to more advanced topics in Computer Science.
As examples we will see analysis of some algorithms that you have studied before
In addition, this chapter examines the important topic of sorting data.
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

3. Measuring the efficiency of Algorithms
Measuring an algorithms efficiency is quite important because your choice of algorithm for a given application often has a great impact.
Suppose two algorithms perform the same task, such as searching.
What does it mean to compare the algorithms and conclude that one is better?
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

4. Chapter 9 -- Algorithm Efficiency and Sorting
The analysis of algorithms is the area of Computer Science that provides the tools for contrasting the efficiency of different methods of solution.
How do you compare them?
Implement them both in C++ and run them?
There are a few difficulties with this
How are the algorithms coded?
What computer should you use?
What data should the programs use?
To overcome these (and other) difficulties, we will use some mathematical techniques to the algorithms independently of coding, computers, and data.
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

5. Chapter 9 -- Algorithm Efficiency and Sorting
The Execution Time of Algorithms
We compared array based lists and linked list based list:
access of the nth element
Insertion and Deletion
An algorithm’s execution time is related to the number of operations it requires.
Counting these operations (if possible) is a way to assess its efficiency
Consider a few examples:
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

6. Chapter 9 -- Algorithm Efficiency and Sorting
Traversal of a Linked List
Code for traversal:
Node *curr = head;
while (curr != NULL){
cout << curr-> item << endl;
curr = curr->next }
Assuming a list of n nodes, these statements require
n+1 assignments,
n+1 comparisons, and
n write operations.
So the total time is A(n+1) + C(n+1) + W(n)
Which is proportional to n
Therefore a list of 100 takes longer than a list of 10
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

7. Chapter 9 -- Algorithm Efficiency and Sorting
The Towers of Hanoi
Chapter 5 proved recursively that the solution to the towers of Hanoi problem with n disks requires 2n-1 moves
If each move requires the same time, m, the solution requires (2n-1)*m time units.
As you can see, this time requirement increases rapidly as the number of disks increases
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

8. Chapter 9 -- Algorithm Efficiency and Sorting
Nested Loops
Consider an algorithm that contains nested loops of the following form:
for (i=1 to n)
for (j=1 to i)
for (k=1 to 5)
taks T
If task T requires t time units,
the innermost loop (on k) requires 5*t time units,
the loop on j requres 5*t*I time units, and
the outermost loop on i requires
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

9. Chapter 9 -- Algorithm Efficiency and Sorting
Algorithm Growth Rates
As you can see, the previous examples derive an algorithm’s time requirement as a function of the problem size.
The way to measure a problem’s size depends on the application
Thus we reached conclusions such as
Algorithm A requires n2/5 time units for a problem of size n
Algorithm B requires 5*n time units for a problem of size n
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

10. Chapter 9 -- Algorithm Efficiency and Sorting
The most important thing to learn is how quickly the algorithm’s time requirement grows as a function of the problem size.
Algorithm A requires time proportional to n2
Algorithm B requires time proportional to n
This is called the growth rate
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

11. Chapter 9 -- Algorithm Efficiency and Sorting
Order of Magnitude Analysis and Big O Notation
If Algorithm A requires time proportional to f(n) Algorithm A is said to be of order f(n)
Denoted: O(f(n))
Because the notation uses the O to denote order it is called Big O notation.
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

12. Chapter 9 -- Algorithm Efficiency and Sorting
Definition of the Order of an Algorithm
Algorithm A is order f(n) – deonted O(f(n)) – if constants k and n0 exist such that A requires no more than k*f(n) time units to solve a problem of size n (n >= n0)
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

13. Chapter 9 -- Algorithm Efficiency and Sorting
Example 1:
Example 2:
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

14. Chapter 9 -- Algorithm Efficiency and Sorting
Example: Table Form
Graphical form
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

15. Chapter 9 -- Algorithm Efficiency and Sorting
Properties of Growth Rate functions:
You can ignore low order terms in an algorithm’s growth rate function:
O(n3+4n2+3n) is O(n3)
You can ignore a multiplicative constant in the high-order term of an algorithm’s growth rate function
O(5n3) is O(n3)
You can combine growth rate functions
O(f(n)) + O(g(n)) = O(f(n)+g(n))
O(n2) + O(n) = O(n2+n) = O(n2)
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

16. Chapter 9 -- Algorithm Efficiency and Sorting
Worst-case and average-case analysis
A particular algorithm might require different times to solve different problems of the same size.
Worst-case analysis concludes that the algorithms is O(f(n)) if, in the worst case, A requires no more time than k*f(n)
Average-case analysis attempts to determine the average amount of time that an algorithm requires to solve problems of size n
This is more difficult than worst-case analysis.
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

17. Chapter 9 -- Algorithm Efficiency and Sorting
Keeping your perspective
When comparing you are interested only in significant differences in efficiency
List retrieval
Implementation
Array implementation
Linked List implementation
List size
> 100 10
Big O focuses on large problems.
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

18. Chapter 9 -- Algorithm Efficiency and Sorting
The Efficiency of Searching Algorithms
Lets look at two algorithms sequential search and binary search
Sequential Search
Start at the beginning and look until you find it or run out of data.
Best case:
Worst case:
Average case:
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

19. Chapter 9 -- Algorithm Efficiency and Sorting
Binary Search
Assumes a sorted array.
The algorithm determines which half to look at and ignores the other half.
If n = 2k, there are k divisions
Therefore k = log2n
Thus the algorithm is O(log2n) in the worst case
So, is a binary search better than a sequential search?
If n = 1,000,000
log2 1,000,000 = 19
Therefore at most 20 compares as opposed to at most 1,000,000 compares.
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

20. Sorting Algorithms and Their Efficiency
What is sorting?
You can organize sorts into two categories:
Internal sorts and external sorts.
We will be looking at internal sorts
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

21. Chapter 9 -- Algorithm Efficiency and Sorting
Selection Sort
Select the largest and put it in its place
Repeat until done.
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

22. Chapter 9 -- Algorithm Efficiency and Sorting
Code:
Big O Analysis:
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

23. Chapter 9 -- Algorithm Efficiency and Sorting
Bubble Sort
Compare adjacent elements and swap if out of order.
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

24. Chapter 9 -- Algorithm Efficiency and Sorting
Code:
Big O Analysis:
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

25. Chapter 9 -- Algorithm Efficiency and Sorting
Insertion Sort
Two parts to a list: sorted and unsorted
(sorted starts at size 1, unsorted at size n-1)
1 element is sorted)
Go through and take next element out of unsorted list and insert them into the correct place in the sorted list.
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

26. Chapter 9 -- Algorithm Efficiency and Sorting
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

27. Chapter 9 -- Algorithm Efficiency and Sorting
Code:
Big O Analysis:
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

28. Chapter 9 -- Algorithm Efficiency and Sorting
Mergesort
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

29. Chapter 9 -- Algorithm Efficiency and Sorting
Mergesort (cont)
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

30. Chapter 9 -- Algorithm Efficiency and Sorting
Code:
Big O Analysis:
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

31. Chapter 9 -- Algorithm Efficiency and Sorting
Quicksort
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

32. Chapter 9 -- Algorithm Efficiency and Sorting
Quicksort (cont)
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

33. Chapter 9 -- Algorithm Efficiency and Sorting
Quicksort (cont)
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

34. Chapter 9 -- Algorithm Efficiency and Sorting
Quicksort (cont)
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

35. Chapter 9 -- Algorithm Efficiency and Sorting
Code:
Big O Analysis:
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

36. Chapter 9 -- Algorithm Efficiency and Sorting
Radixsort
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

37. Chapter 9 -- Algorithm Efficiency and Sorting
Code:
Big O Analysis:
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

38. Chapter 9 -- Algorithm Efficiency and Sorting
A Comparison of Sorting Algorithms
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting

39. Chapter 9 -- Algorithm Efficiency and Sorting
CS 308
Chapter 9 -- Algorithm Efficiency and Sorting