1. Introduction to Algorithms: Asymptotic Notation

2. CS 421 - Analysis of Algorithms
Asymptotic Notation
O-notation (upper bounds):
that 0 f(n) cg(n) for all n n .
CS Analysis of Algorithms

3. CS 421 - Analysis of Algorithms
Asymptotic Notation
O-notation (upper bounds):
that 0 f(n) cg(n) for all n n .
where c = 1, n0 = 2
CS Analysis of Algorithms

4. CS 421 - Analysis of Algorithms
Asymptotic Notation
O-notation (upper bounds):
that 0 f(n) cg(n) for all n n .
where c = 1, n0 = 2
functions, not values
funny, one-way equality
CS Analysis of Algorithms

5. Set Definition of Ο-notation
Ο(𝑔 𝑛 ) is the set of all functions with a SMALLER or the SAME order of growth as 𝑔 𝑛 .
CS Analysis of Algorithms

6. Set Definition of Ο-notation
Ο(𝑔 𝑛 ) is the set of all functions with a SMALLER or the SAME order of growth as 𝑓(𝑛).
EXAMPLE:
Ο( 𝑛 3 )= 5𝑛 3 , 10𝑛 , log 𝑛, …
Or put another way:
5 𝑛 3 ∈ Ο( 𝑛 3 )
CS Analysis of Algorithms

7. -notation (lower bounds)

8. CS 421 - Analysis of Algorithms
Asymptotic Notation
Ω-notation (lower bounds):
that 0 f(n) cg(n) for all n n .
CS Analysis of Algorithms

9. CS 421 - Analysis of Algorithms
Asymptotic Notation
Ω-notation (lower bounds):
that 0 f(n) cg(n) for all n n .
where c = 1, n0 = 16
CS Analysis of Algorithms

10. Set Definition of Ω-notation
Ω(𝑔 𝑛 ) is the set of all functions with a LARGER or the SAME order of growth as 𝑔 𝑛 .
EXAMPLE:
Ω( 𝑛 2 )= 5𝑛 2 , 10𝑛 , 𝑛 2 ∗log 𝑛, …
Or put another way:
5 𝑛 2 ∈Ω( 𝑛 2 )
CS Analysis of Algorithms

11. -notation (tight bounds)
Combine definitions of Ο and Ω:
𝑊𝑒 𝑤𝑟𝑖𝑡𝑒:
𝑓 𝑛 = Θ 𝑔 𝑛
𝑖𝑓 𝑡ℎ𝑒𝑟𝑒 𝑒𝑥𝑖𝑠𝑡 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠
𝑐1 > 0, 𝑐2>0 𝑎𝑛𝑑 𝑛0>0
𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡
c1 𝑔(𝑛) < 𝑓(𝑛) < 𝑐2𝑔(𝑛)
CS Analysis of Algorithms

12. -notation (tight bounds)
CS Analysis of Algorithms

13. -notation (tight bounds)
EXAMPLE:
CS Analysis of Algorithms

14. CS 421 - Analysis of Algorithms
-notation
O-notation like .
o-notation like <.
𝑊𝑒 𝑤𝑟𝑖𝑡𝑒: 𝑓 𝑛 = 𝜊 𝑔 𝑛
𝑓𝑜𝑟 𝑎𝑛𝑦 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠
𝑐 > 0,
𝑡ℎ𝑒𝑟𝑒 𝑖𝑠 𝑎 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑛 0 > 0
𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡
0 < 𝑓(𝑛) < 𝑐𝑔(𝑛)
CS Analysis of Algorithms

15. CS 421 - Analysis of Algorithms
-notation
O-notation like .
o-notation like <.
𝑊𝑒 𝑤𝑟𝑖𝑡𝑒: 𝑓 𝑛 = 𝜊 𝑔 𝑛
𝑓𝑜𝑟 𝑎𝑛𝑦 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠
𝑐 > 0,
𝑡ℎ𝑒𝑟𝑒 𝑖𝑠 𝑎 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑛 0 > 0
𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡
0 < 𝑓(𝑛) < 𝑐𝑔(𝑛)
EXAMPLE:
2n2 = o(n3),
n0 = 2/c
CS Analysis of Algorithms

16. Set Definition of ο-notation
ο(𝑔 𝑛 ) is the set of all functions with a strictly SMALLER order of growth as 𝑔 𝑛 .
CS Analysis of Algorithms

17. Set Definition of ο-notation
ο(𝑔 𝑛 ) is the set of all functions with a SMALLER order of growth as 𝑔 𝑛 .
EXAMPLE:
ο( 𝑛 3 )= 10𝑛 , log 𝑛, …
Or put another way:
10 𝑛 ∈ ο( 𝑛 3 )
CS Analysis of Algorithms

18. CS 421 - Analysis of Algorithms
-notation
-notation is like .
-notation is like >.
𝑊𝑒 𝑤𝑟𝑖𝑡𝑒: 𝑓 𝑛 = ω 𝑔 𝑛
𝑓𝑜𝑟 𝑎𝑛𝑦 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠
𝑐 > 0,
𝑡ℎ𝑒𝑟𝑒 𝑖𝑠 𝑎 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑛 0 > 0
𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡
0 <𝑐𝑔(𝑛) <𝑓(𝑛)
CS Analysis of Algorithms

19. CS 421 - Analysis of Algorithms
-notation
-notation is like .
-notation is like >.
𝑊𝑒 𝑤𝑟𝑖𝑡𝑒: 𝑓 𝑛 = ω 𝑔 𝑛
𝑓𝑜𝑟 𝑎𝑛𝑦 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠
𝑐 > 0,
𝑡ℎ𝑒𝑟𝑒 𝑖𝑠 𝑎 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑛 0 > 0
𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡
0 <𝑐𝑔(𝑛) <𝑓(𝑛)
n  (lg n),
n0 = 1+1/c
EXAMPLE:
CS Analysis of Algorithms

20. Set Definition of ω-notation
ω(𝑓(𝑛)) is the set of all functions with a strictly LARGER order of growth as 𝑓(𝑛).
CS Analysis of Algorithms

21. Set Definition of ω-notation
ω(𝑓(𝑛)) is the set of all functions with a strictly LARGER order of growth as 𝑓(𝑛).
EXAMPLE:
ω( 𝑛 2 )= 10𝑛 , 𝑛 2 ∗log 𝑛, …
Or put another way:
10 𝑛 3 ∈ω( 𝑛 2 )
CS Analysis of Algorithms

22. Properties: Asymptotic Notation
Transitivity
Reflexivity
Symmetry
Transposition
CS Analysis of Algorithms

23. CS 421 - Analysis of Algorithms
Transitivity
Assuming f(n) and g(n) are asymptotically positive: 𝑓(𝑛) = Θ(𝑔(𝑛)) and 𝑔(𝑛) = Θ(ℎ(𝑛)) implies 𝑓(𝑛) = Θ(ℎ(𝑛)) Holds for Ο, Ω, ο, and ω relations as well.
CS Analysis of Algorithms

24. CS 421 - Analysis of Algorithms
Reflexivity
Assuming f(n) is asymptotically positive: 𝑓(𝑛) = Ο(𝑓(𝑛)) and 𝑓(𝑛) = Ω(𝑓(𝑛)) 𝑓(𝑛) = Θ(𝑓(𝑛)) DOES NOT hold for ο and ω relations.
CS Analysis of Algorithms

25. CS 421 - Analysis of Algorithms
Symmetry
Assuming f(n) and g(n) are asymptotically positive: 𝑓(𝑛) = Θ(𝑔(𝑛)) iff (if, and only if,) 𝑔(𝑛) = Θ(𝑓(𝑛))
CS Analysis of Algorithms

26. CS 421 - Analysis of Algorithms
Transpose
Assuming f(n) and g(n) are asymptotically positive: 𝑓(𝑛) = Ο(𝑔(𝑛)) iff 𝑔(𝑛) = Ω(𝑓(𝑛)) and 𝑓(𝑛) = ο(𝑔(𝑛)) iff 𝑔(𝑛) = ω(𝑓(𝑛))
CS Analysis of Algorithms

27. Using Limits to Compare Growth Rates
Though using Ο, Ω, ο, and ω indispensable for comparing growth rates of functions in the abstract, when comparing actual functions, convenient to
CS Analysis of Algorithms

28. Using Limits to Compare Growth Rates
0 - f(n) has smaller growth rate than g(n)
c - f(n) has same growth rate as g(n)
∞ - f(n) has larger growth rate than g(n)
first two cases ⟹ 𝑓 𝑛 ∈ Ο 𝑔 𝑛
last two cases ⟹ 𝑓 𝑛 ∈ Ω(𝑔 𝑛 )
last case ⟹ 𝑓 𝑛 ∈ Θ(𝑔 𝑛 )
CS Analysis of Algorithms

29. CS 421 - Analysis of Algorithms
Why Should We Care? n
lg n
n lg n n2 n3 2n n! 10
3.3
101
3.3*101
102
103
3.6*106
6.6
6.6*102
104
106
1.3*1030
9.3*10157
10*103
109 - 13
13*104
108
1012
105 17
17*105
1010
1015 20
20*106
1018
CS Analysis of Algorithms

30. Most Common Growth Rates
Class
Name
Examples 1
Constant
Only used in best-case efficiencies.
log n
logarithmic
Result of cutting problem size by a constant factor, like Binary Search n
linear
Algorithms that scan a list of size n, like Sequential, or Linear, Search
n log n
n-log-n
Divide-and-Conquer algorithms, like Merge Sort and Quick Sort n2
quadratic
Efficiencies with two embedded loops, Bubble Sort and Insertion Sort n3
cubic
Efficiencies with three embedded loops, like many linear algebra algorithms 2n
exponential
Algorithms that generate all sub-sets of an n-element set n!
factorial
Algorithms that generate all permutations of an n-element set
CS Analysis of Algorithms

31. CS 421 - Analysis of Algorithms

32. Macro Substitution
Convention: A set in a formula represents an anonymous function in the set.

33. CS 421 - Analysis of Algorithms
Macro substitution
Convention: A set in a formula represents an anonymous function in the set.
EXAMPLE:
f(n) = n3 + O(n2)
means
f(n) = n3 + h(n)
for some
h(n)  O(n2)
CS Analysis of Algorithms

34. CS 421 - Analysis of Algorithms
Macro substitution
Convention: A set in a formula represents an anonymous function in the set.
EXAMPLE:
CS Analysis of Algorithms