1. Lecture 6 Efficiency of Algorithms (2) (S&G, ch.3)
12/4/2018 8:29 AM
Lecture 6
Efficiency of Algorithms (2)
(S&G, ch.3)
12/4/2018
CS Lecture 6
CS 100

2. Read S&G chapter 4
Warning: There are printing errors in ch. 4 in formulas!
We will send out corrections by
12/4/2018
CS Lecture 6

3. slide courtesy of S. Levy
Lecture 6
12/4/2018 8:29 AM
Sorting
One of the most common activities of a computer is sorting data.
Arranging data into numerical or alphabetical order for purposes of
Reports by category
Summarizing data
Searching data
12/4/2018
CS Lecture 6
slide courtesy of S. Levy
CS 100

4. slide courtesy of S. Levy
Lecture 6
12/4/2018 8:29 AM
Sorting (2)
Given: n, N1, N2, …, Nn
Want: Rearrangement M1, M2, …, Mn where M1 …  Mn according to the appropriate understanding of .
There are many well-known algorithms for doing this. Preference may be due to
list size
degree of order already present
ease of programming
program available
12/4/2018
CS Lecture 6
slide courtesy of S. Levy
CS 100

5. Sorting - Selection Sort
Lecture 6
12/4/2018 8:29 AM
Sorting - Selection Sort
Strategy:
find the largest element in list
swap it with last element in list
find next largest
swap it with next to last element in list
etc
Variables:
U - Marker for unsorted section
L - Position of largest so far
C - Current position
12/4/2018
CS Lecture 6
slide courtesy of S. Levy
CS 100

6. Selection Sort - The Algorithm: High Level
Lecture 6
12/4/2018 8:29 AM
Selection Sort - The Algorithm: High Level
Set U to n
Repeat until U = 1
Search For Largest of N [1] to N [U] and put its location in L
Exchange N [L] and N [U]
Set U to U – 1
Stop
The inner loop finds the largest in N[1] … N[U]
12/4/2018
CS Lecture 6
CS 100

7. Selection Sort - The Algorithm
Lecture 6
12/4/2018 8:29 AM
Selection Sort - The Algorithm
Set U to n
Repeat until U = 1
Set L to 1
Set C to 2
Repeat until C > U
If N [C] > N [L] then
Set L to C
Set C to C + 1
Exchange N [L] and N [U]
Set U to U – 1
Stop
Search For Largest of N [1], …, N [U] and put location in L
The inner loop finds the largest in N[1] … N[U]
12/4/2018
CS Lecture 6
slide courtesy of S. Levy
CS 100

8. Selection Sort - Example (Pass 1)
Lecture 6
Selection Sort - Example (Pass 1)
12/4/2018 8:29 AM 6 5 19 9 12 U L C 6 5 19 9 12 U L C 6 5 19 9 12 U L C 6 5 19 9 12 U L C 6 5 19 9 12 U L C 6 5 12 9 19 U L C 6 5 19 9 U L C 12
12/4/2018
CS Lecture 6
slide courtesy of S. Levy
CS 100

9. Selection Sort - Example (Pass 2)
Lecture 6
Selection Sort - Example (Pass 2)
12/4/2018 8:29 AM 6 5 12 9 U L C 19 6 12 9 19 U L C 5 6 5 9 19 U L C 12 6 5 12 19 U L C 9 6 5 12 9 19 U L C 6 12 9 U L C 19 5
12/4/2018
CS Lecture 6
slide courtesy of S. Levy
CS 100

10. Selection Sort - Example (Pass 3)
Lecture 6
Selection Sort - Example (Pass 3)
12/4/2018 8:29 AM 6 12 U L C 19 9 5 6 5 9 12 U L C 19 6 5 12 U L C 19 9 6 5 19 LU C 12 9 6 5 9 12 U L C 19
12/4/2018
CS Lecture 6
slide courtesy of S. Levy
CS 100

11. Selection Sort - Example (Pass 4)
Lecture 6
Selection Sort - Example (Pass 4)
12/4/2018 8:29 AM 6 5 12 U L C 19 9 5 6 12 U 19 9 6 5 12 U L C 19 9 5 9 12 19 LU C 6
12/4/2018
CS Lecture 6
slide courtesy of S. Levy
CS 100

12. Efficiency Analysis of Selection Sort
Lecture 6
12/4/2018 8:29 AM
Efficiency Analysis of Selection Sort
Makes n – 1 passes through the list
Each pass uses Search For Largest, on the unsorted part of the list, to find the largest element
If the unsorted part has length U, Search For Largest takes U – 1 comparisons
12/4/2018
CS Lecture 6
CS 100

13. Efficiency of Selection Sort (2)
Lecture 6
12/4/2018 8:29 AM
Efficiency of Selection Sort (2)
To sort n element list:
n – 1 comparisons to find largest
n – 2 comparisons to find second largest …
2 comparisons to find largest of last three
1 comparisons to find largest of last two
Total number of comparisons:
12/4/2018
CS Lecture 6
CS 100

14. A Useful Formula 12/4/2018 CS 100 - Lecture 6 Lecture 6
12/4/2018 8:29 AM
A Useful Formula
12/4/2018
CS Lecture 6
CS 100

15. A Useful Formula (2)
12/4/2018
CS Lecture 6

16. Best, Worst, & Average 12/4/2018 CS 100 - Lecture 6 Lecture 6
12/4/2018 8:29 AM
Best, Worst, & Average
Note that for some common (and useful) algorithms, the worst case is quite bad,
but the (observed) average case is quite good.
12/4/2018
CS Lecture 6
CS 100

17. Motivation for Asymptotic Complexity
Lecture 6
12/4/2018 8:29 AM
Motivation for Asymptotic Complexity
We are interested in comparing the efficiency of various algorithms, independent of computer running them
Therefore, we choose to ignore:
the absolute time taken by the instructions
constant initialization overhead
time taken by the less frequently executed instructions
12/4/2018
CS Lecture 6
CS 100

18. Motivation for Asymptotic Complexity (2)
Lecture 6
12/4/2018 8:29 AM
Motivation for Asymptotic Complexity (2)
Not concerned about efficiency for small problems (they’re all quick enough)
Usually interested in using on large problems
We investigate how resource usage varies on progressively larger problems
Which is asymptotically better, i.e., for sufficiently large problems
12/4/2018
CS Lecture 6
CS 100

19. Quadratic vs. Linear Growth
Lecture 6
12/4/2018 8:29 AM
Quadratic vs. Linear Growth
Variable curve is y = n x2 (n in [0, 10])
On big enough problem, quadratic algorithm takes more time than linear algorithm
12/4/2018
CS Lecture 6
CS 100

20. Quadratic vs. Linear Growth (2)
Lecture 6
12/4/2018 8:29 AM
Quadratic vs. Linear Growth (2)
Note the linear algorithm has higher fixed overhead and a quicker rate of growth
Quadratic behavior eventually dominates quadratic algorithm
12/4/2018
CS Lecture 6
CS 100

21. Equivalent Complexity
For our purposes: An algorithm taking time 10n sec. is as good as one taking 2n sec.
because absolute time for individual instructions doesn’t matter
An algorithm taking 2n + 10 sec. is as good as one taking 2n sec.
because fixed overhead doesn’t matter
An algorithm taking n2 + 2n sec. is as good as one taking n2 sec.
less frequently executed instructions don’t matter
12/4/2018
CS Lecture 6

22. Complexity Classes No growth: (1) Linear growth: (n)
Lecture 6
12/4/2018 8:29 AM
Complexity Classes
No growth: (1)
Linear growth: (n)
Quadratic growth: (n2)
Cubic growth: (n3)
Polynomial growth: (nk) for any k
Exponential growth: (kn) for any k
 (theta) is called the “exact order” of a function
Call Q the “exact order” of a function
12/4/2018
CS Lecture 6
CS 100

23. Comparison of Growth Orders
Lecture 6
12/4/2018 8:29 AM
Comparison of Growth Orders
grey: constant
yellow: logarithmic
blue: linear
light blue: quadratic
green: cubic
purple: exponential (2n)
red: exponential (3n)
12/4/2018
CS Lecture 6
CS 100

24. Combinatorial Explosion
Consider a game with k possible moves on each play
Therefore:
k possible first moves
k2 possible replies
k3 possible replies to the replies, etc.
In general, must consider kn possibilities to look n moves ahead (exponential algorithm)
“Brute force” solutions to many problems lead to “combinatorial explosion”
12/4/2018
CS Lecture 6

25. Intractable Problems
For some problems, all known algorithms are exponential-time
No known polynomial-time algorithms — not even (n100)!
Sometimes there are efficient approximation algorithms
give good, but not perfect answer
may be good enough for practical purposes
12/4/2018
CS Lecture 6

26. Limitations of Asymptotic Complexity
Note that 1,000,000 n is (n), but n2 is (n2)
Paradox: 1,000,000 n hrs >> n2 hrs (for n << 1012), but (n) is “more efficient” than (n2)
Not useful if concerned with absolute real-time constraints (e.g., must respond in 2 sec.)
Not useful if concerned with fixed size or bounded problems (e.g., n < 1,000,000)
Sometimes, “the constants matter”
12/4/2018
CS Lecture 6