1. CS 312: Algorithm Analysis
This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.
CS 312: Algorithm Analysis
Lecture #2: Asymptotic Notation
Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick

2. Quiz What is the reward for submitting project reports early?
How many penalty-free late days are in your budget for the semester?
How many penalty-free late days can you use on any one project?
What is the penalty for late submission of project reports (after you’ve used your penalty-free late days)?
T/F: You may submit late regular homework assignments.
Syllabus policies

3. Announcements Your initial experiences with Visual Studio?
My office hours updated: Tu & Th 2-3 and by appointment
TA office hours are on the course wiki
Different kinds of TAs
Project #1
Instructions are linked from the online schedule
Help session with TA
Thursday at 4pm
In the Windows help lab (1066 TMCB)
Focus on intro. to C# and Visual Studio
We’ll cover the mathematical ideas in class
Purpose of help sessions

4. Objectives Revisit orders of growth
Formally introduce asymptotic notation: O, W, Q
Classify functions in asymptotic orders of growth

5. Orders of Growth
Efficiency: how cost grows with the size/ difficulty n of a given problem instance.
C(n): the cost (e.g., number of steps) required by an algorithm on an input of size / difficulty n.
Order of growth: the functional form of C(n) up to a constant multiple as n goes to infinity.

6. Orders of Growth n log2n nlog2n n2 n3 10 3.3 3.3*10 102 103 6.6
6.6*102
104
106
1.0*104
109 13
1.3*105
108
1012
105 17
1.7*106
1010
1015 20
2.0*107
1018

7. Orders of Growth Efficient n log2n nlog2n n2 n3 2n n! 10 3.3 3.3*10
102
103
3.6*106
6.6
6.6*102
104
106
1.3*1030
9*10157
1.0*104
109 … 13
1.3*105
108
1012
105 17
1.7*106
1010
1015 20
2.0*107
1018
Efficient

8. Kinds of Efficiency
Need to decide which instance of a given size to use as the representative for that class:
Algorithm Domain
Worst Case
Best Case
Instances
Average Case (over all possible instances)
Instance size

9. Asymptotic Notation
Definition: given an asymptotically non-negative fn. g(n),
Translation: An asymptotically non-negative function f(n) belongs to the set Q(g(n)) iff there exist positive constants c1 and c2 such that f(n) can be “sandwiched” between c1g(n) and c2g(n), scaled versions of g(n), for sufficiently large n.
Convention:

10. Asymptotic Notation
Definitions: given an asymptotically non-negative fn. g(n),

11. Asymptotic Notation f(n) c g(n) n0 f(n) c g(n) n0 n0 c1g(n) c2g(n)

12. Distinguishing 𝑂 from Θ
Can you draw a function that is in 𝑂 𝑛 2 but not Θ 𝑛 2 ?

13. Distinguishing 𝑂 from Θ
Can you draw a function that is in 𝑂 𝑛 2 but not Θ 𝑛 2 ?

14. Example Show that Must find positive constants c1, c2, n0 such that
Divide through by n2 to get
Hint: consider one side of the inequality at a time:
for n0=7 we have c1 1/14
for n0=6 we have c2 >1/2; works for larger n0=7 as well
This proof is constructive

15. Another Example Show that 6 𝑛 3 ∉𝛩 𝑛 2
Use proof by contradiction: assume that 6 𝑛 3 ∈Θ 𝑛 2
Suppose positive constants c2, n0 exist such that 6 𝑛 3 ≤ 𝑐 2 𝑛 2 ∀𝑛≥ 𝑛 0
But this implies that 𝑛≤ 𝑐 ∀𝑛≥ 𝑛 0
i.e., 𝑛 is bounded by 𝑐 2 6 , a constant
But 𝑛 is unbounded; hence we have a contradiction.
Thus, our assumption was false; hence we have shown that 6 𝑛 3 ∉𝛩 𝑛 2

16. Other proof types? Induction Deduction Others!
Optional example on HW #1
Deduction
Others!

17. Duality

18. Duality
g(n) in O(f(n)) means all instances of size n are solvable within c1f(n) time.
f(n) in Ω(g(n)) means at least one instance of size n is solvable in c2g(n) time.
… for worst case performance.

19. Duality Suppose t(n) models worst case behavior of an algorithm.
Then t(n) = O(n!) means every instance of size n is solvable in factorial time.
Then t(n) = (n!) means at least one instance of size n requires factorial time.
But infinitely many size-n instances may require n2 time!
Using our definition of , a  bound on the worst case isn’t useful.
1. Change the definition. But then duality fails.
2. Live with it. That’s what we’ll do.
Similarly, using duality we also conclude that a O bound on a best case isn’t useful!

20. The Limit Rule

21. The Limit Rule

22. Example

23. Example

24. Useful Identity: L’Hopital’s Rule
Applicable when lim f(n) = lim g(n) = 0
and when lim f(n) = lim g(n) = 

25. Review: Log Identities

26. Review: Log Identities

27. Review: More Logarithms

28. Review: More Logarithms

29. Assignment Read: Section 1.3 HW #1: 0.1 (a-e, g, m) in the textbook
Optional: 0.3 (a)
Remember:
Don’t spend more than 2 focused hours on the homework exercises.
If you reach 2 hours of concentrated effort, stop and make a note to that effect on your paper (unless you’re having fun and want to continue).
The TAs will take that into consideration when grading.

30. The following slides are extra

31. Addition

32. Addition

33. Multiplication

34. Multiplication

35. Classic Multiplication
American Style
English Style
5001
x 502
10002
5001
x 502
25005
O(n2) for 2 n-digit numbers

36. Multiplication a la Francais / Russe

37. Multiplication a la Francais / Russe

38. Mulitplication a la Francais / Russe
function multiply(x,y)
Input: Two n-bit integers x and y, where y  0
Output: Their product
if y = 0: return 0
if y = 1: return x
z = multiply(x, floor(y/2))
if y is even:
return 2z
else:
return x+2z

39. Mulitplication a la Francais / Russe
function multiply(x,y)
Input: Two n-bit integers x and y, where y  0
Output: Their product
if y = 0: return 0
if y = 1: return x
z = multiply(x, floor(y/2))
if y is even:
return 2z
else:
return x+2z

40. Mulitplication a la Francais / Russe
function multiply(x,y)
Input: Two n-bit integers x and y, where y>=0
Output: Their product
if y = 0: return 0
if y = 1: return x
z = multiply(x, floor(y/2))
if y is even:
return 2z
else:
return x+2z

41. Multiplication a la russe
Don’t need to know
multiplication
tables
Just need to know
how to double,
halve, and add!

42. Multiplication a la russe
101
10100
= 2510
1012 x2 1012
= 2510
do this example in binary because it’s a little easier to see how it works.
O(n2)

43. Arabic Multiplication 1 2 3 4

44. Division function divide(x,y)
Input: Two n-bit integers x and y, where y  1
Output: The quotient and remainder of x divided by y
if x=0: return (q, r) = (0,0)
(q, r) = divide(floor(x/2),y)
q=2*q, r=2*r
if x is odd: r=r+1
if r  y: r=r-y, q=q+1
return (q, r)

45. Division function divide(x,y)
Input: Two n-bit integers x and y, where y  1
Output: The quotient and remainder of x divided by y
if x=0: return (q, r) = (0,0)
(q, r) = divide(floor(x/2),y)
q=2*q, r=2*r
if x is odd: r=r+1
if r  y: r=r-y, q=q+1
return (q, r)