1. 한양대학교 정보보호 및 알고리즘 연구실 2008. 2. 12 이재준 담당교수님 : 박희진 교수님
Chapter 3. Growth of function
Chapter 4. Recurrences
한양대학교 정보보호 및 알고리즘 연구실
이재준
담당교수님 : 박희진 교수님

2. Contents of Table A. Growth of function B. Recurrence
2. Notation of asymptotically larger/smaller
o-notation
ω-notation
3. Relationship between this notations
B. Recurrence
1. Technicalities for abbreviation 2. Recurrence solution methods The substitution method

3. f(n) is asymptotically smaller than g(n)
2. Asymptotically larger/smaller notation
o-notation
Asymptotically smaller
- Definition
for all n, n ≥ n0 , any positive constant c, c>0, 0 ≤ f(n)< cg(n), the function f(n) is smaller to g(n) to within constant factor.
f(n) is asymptotically smaller than g(n)
f(n)
o(g(n)) n0

4. o-notation 2. Asymptotically larger/smaller notation - Definition
o(g(n)) = { f(n) : for any positive constant c>0, there exist a constant
n0 >0 such that 0 ≤ f(n) < cg(n) for all n ≥ n0 }
this limit shows the function f(n) becomes insignificant relative to g(n) as n approaches infinity

5. o-notation Example 1. Asymptotic notation
Use the definition of o-notation to prove the following property.
f(n) = o(n2)

6. o-notation Solution to example
2. Notation of asymptotically larger/smaller
o-notation Solution to example
2n = o(n2)
If f(n) = o((g(n)) then

7. f(n) is asymptotically larger than g(n)
2. Notation of asymptotically larger/smaller
ω -notation
Asymptotically larger
- Definition
for all n, n ≥ n0 , any positive constant c, c>0, 0 ≤ cg(n) < f(n), the function f(n) is larger to g(n) to within constant factor.
f(n) is asymptotically larger than g(n)
f(n)
ω(g(n))

8. ω-notation 2. Notation of asymptotically larger/smaller - Definition
ω(g(n)) = { f(n) : for any positive constant c>0, there exist a constant
n0 >0 such that 0 ≤ cg(n) < f(n) for all n ≥ n0 }
if the limit exists. That is, f(n) becomes arbitrarily large relative to g(n) as n approaches infinity.

9. ω -notation 2. Notation of asymptotically larger/smaller
- We use ω-notation to denote a lower bound that is not asymptotically tight.
- We use o-notation to denote an upper bound that is not asymptotically tight.
- f(n) ∈ ω(g(n)) if and only if g(n) ∈ o(f(n)).
Arbitrarily : 독단적으로, 제멋대로, 마음대로

10. ω -notation Example 2. Notation of asymptotically larger/smaller
Use the definition of o-notation to prove the following property.
f(n) = ω(n)

11. ω -notation Solution to example
2. Notation of asymptotically larger/smaller
ω -notation Solution to example
=ω(n)
If f(n) = ω((g(n)) then

12. 3. Relationship between this notations
f(n)
O(g(n))
o(g(n))
ω(g(n))
Ω(g(n))
θ(g(n))
O(g(n))
Ω(g(n))
ω(g(n))
o(g(n))

13. Transitivity Transpose symmetry 3. Relationship between this notations
f(n) = Θ(g(n)) and g(n) = Θ(h(n)) imply f(n) = Θ(h(n)) ,
f(n) = O(g(n)) and g(n) = O(h(n)) imply f(n) = O(h(n)) ,
f(n) = Ω(g(n)) and g(n) = Ω(h(n)) imply f(n) = Ω(h(n)) ,
f(n) = o(g(n)) and g(n) = o(h(n)) imply f(n) = o(h(n)) ,
f(n) = ω(g(n)) and g(n) = ω(h(n)) imply f(n) = ω(h(n)).
f(n)
O(g(n))
o(g(n))
ω(g(n))
Ω(g(n))
f(n) = O(g(n)) if and only if g(n) = Ω(f(n)),
f(n) = o(g(n)) if and only if g(n) = ω(f(n)).

14. Reflexivity Symmetry f(n) = Θ(f(n)) f(n) = O(f(n)) f(n) = Ω(f(n))
3. Relationship between this notations
Reflexivity
f(n) = Θ(f(n))
f(n) = O(f(n))
f(n) = Ω(f(n))
Symmetry
f(n) = Θ(g(n)) if and only if g(n) = Θ(f(n)).
θ(g(n))
O(g(n))
Ω(g(n))

15. When should we use the recurrence?
Recurrences
Recurrence
When should we use the recurrence?
When algorithm contains a recursive call to itself,
Its running time can often be described by a recurrence.

16. Recurrences Recurrence - Definition
Equation or inequality that describes a function in terms of its
value on smaller inputs.

17. Recurrences Recurrence Example if n=1 if n>1
Worst-case running time of T(n) could be described by the recurrence
Merge-Sort procedure
Show that the solution of T(n) = Θ(n log n)
if n=1
if n>1
범위에 대한 가정을 하고 그 가정이 맞는지 수학적 귀납법을 이용하여 증명하는것입니다.
Recurrence를 트리형태로 바꾸고 각 트리레벨에서의 cost를 구한후 이들의 총합을 구함으로써 전체 cost를 구하는 방법입니다.
Recurrence 의 기본적인 Form에서 주어진 세가지 경우에 따라 값을 구하는 방법입니다.

18. Recurrences Recurrence Solution method to example
- The substitution method
guess a bound and prove our guess correct for small input.
- The recursion-tree method
convert the recurrence into a tree whose nodes represent the
costs incurred at various level of recursion
- The master method
determining asymptotic bounds for many simple recurrences of
the form
범위에 대한 가정을 하고 그 가정이 맞는지 수학적 귀납법을 이용하여 증명하는것입니다.
Recurrence를 트리형태로 바꾸고 각 트리레벨에서의 cost를 구한후 이들의 총합을 구함으로써 전체 cost를 구하는 방법입니다.
Recurrence 의 기본적인 Form에서 주어진 세가지 경우에 따라 값을 구하는 방법입니다.
( a ≥ 1, b ≥ 1, and f(n) is given function )

19. 1. Technicalities for abbreviation
- assumption of integer
The running time T(n) of algorithm is only defined when n
is an integer
- floors / ceilings
We often omit floors and ceilings
- boundary condition
The running time of an algorithm on a constant-sized input is
a constant that we typically ignore.

20. 1. Technicalities for abbreviation
Technicalities Example
The recurrence describing the worst-case running time of Merge-Sort
if n=1
if n>1
Omit assumption of integer
Omit floors and ceilings
생략되는 Technical details은 recurrence solution method를 보면서 보게 될 것이다.
Floor : 같거나 더 큰 수치 중에서 가장 작은 정수
Ceiling : 같거나 더 작은 수치 중에서 가장 큰 정수
Omit boundary condition

21. 2. Recurrence solution methods
Substitution Method
- Definition
When recurrence is easy to guess form of the answer with
the inductive hypothesis is applied to smaller values,
substitution of the guessed answer for the function .
생략되는 Technical details은 recurrence solution method를 보면서 보게 될것이다.
Floors와 ceilings의 경우 몇몇 특수한 경우를 제외하고는 걱정할 필요가 없습니다.
- It can be used to establish either upper or lower bounds on a recurrence.

22. 2. Recurrence solution methods
Substitution Method Steps
1. Guess the form of the solution
2. Use mathematical induction to find the constant and show that the solution works.

23. 2. Recurrence solution methods
Substitution Method
- Making a good guess
prove loose upper and lower bounds on the recurrence
and then reduce the range of uncertainty.
- Guessing a solution takes :
experience, occasionally, creativity
- we can use recursion tree

24. 2. Recurrence solution methods
Substitution Method
- Subtleties
we can prove something stronger for given value by
assuming something stronger for a smaller values.
We guess that the solution is O(n), and show that T(n) ≤ cn
Overcome difficulty by subtracting a lower-order term from our previous guess
( b≥1, c must be chosen large enough )

25. 2. Recurrence solution methods
Substitution Method
- Avoid pitfalls
In recurrence we can falsely prove
T(n)= O(n) by guessing T(n) ≤ cn

26. 2. Recurrence solution methods
Substitution Method Example
Let us determine an upper bound on the recurrence

27. 2. Recurrence solution methods
Substitution Method Solution to example
1. guess that the solution is
2. prove that ( c > 0 )
We need to find constant c and n0
- Assume that this bound holds for ⌊n/2⌋, that is, T (⌊n/2⌋) ≤ c ⌊n/2⌋ lg(⌊n/2⌋).
T(n) ≤ 2(c ⌊n/2⌋lg(⌊n/2⌋)) + n
≤ cn lg(n/2) + n ( because, ⌊n/2⌋ < n/2 )
= cn lg n - cn lg 2 + n
= cn lg n - cn + n
≤ cn lg n (as long as c ≥ 1)

28. 2. Recurrence solution methods
Substitution Method Solution to example
- Find the constant n0
- we can take advantage of asymptotic notation,
we can replace base case
T(1) ≤ cn lg n
T(1) = 1 but, c1 lg1 = 0
- So, we only have to prove T (n) = cn lg n for n ≥ n0 for n (n0 =2)
Replace T(1) by T(2) and T(3) as the base cases by letting n0 =2.

29. 2. Recurrence solution methods
Substitution Method Solution to example
- Find the constant c
- choosing c large enough
T (2) = 4 and T (3) = 5.
T (2) = 4 ≤ c ( 2 lg 2 )
T (3) = 5 ≤ c ( 3 lg 3 ).
Any choice of c ≥ 2 suffices for base case of n=2 and n =3

30. 2. Recurrence solution methods
Substitution Method Example
Let us determine an upper bound on the recurrence

31. 2. Recurrence solution methods
Substitution Method Solution to example
- Simplify this recurrence by changing variable
Renaming m = lg n
S(m) = T(2m)
This new recurrence has same solution : S(m) = O(m log m)

32. 2. Recurrence solution methods
Substitution Method Solution to example
Simplify this recurrence by changing variable
This new recurrence has same solution : S(m) = O(m log m)
Changing back from S(m) to T(n)
T(n)
= T(2m)
= S(m)
= O(m lg m)
= O( lg n lg(lg n) )