Analysis of Algorithms1 O-notation (upper bound) Asymptotic running times of algorithms are usually defined by functions whose domain are N={0, 1, 2, …} (natural numbers) Formal Definition of O-notation f(n) = O(g(n)) if  positive constants c, n 0 such that 0 ≤ f(n) ≤ cg(n),  n ≥ n 0 Ex. 2n 2 = O(n 3 ) 2n 2 ≤ cn 3  cn ≥ 2  c = 1 & n 0 = 2 or c = 2 & n 0 = 1 2n 3 = O(n 3 ) 2n 3 ≤ cn 3  c ≥ 2  c = 2 & n 0 = 1

Analysis of Algorithms2 O-notation (upper bound) “=” is funny; “one-way” equality O-notation is sloppy, but convenient. –We must understand what O(n) really means O(g(n)) is actually a set of functions. O(g(n)) = { f(n) :  positive constants c, n 0 such that 0 ≤ f(n) ≤ cg(n),  n ≥ n 0 } 2n 2 = O(n 3 ) means that 2n 2  O(n 3 )

Analysis of Algorithms3 O-notation (upper bound) O-notation is an upper-bound notation It makes no sense to say “running time of an algorithm is at least O(n 2 )”. let running time be T(n) T(n) ≥ O(n 2 ) means T(n) ≥ h(n) for some h(n)  O(n 2 ) however, this is true for any T(n) since h(n) = 0  O(n 2 ), & running time > 0, so stmt tells nothing about running time

Analysis of Algorithms4 O-notation (upper bound)

Analysis of Algorithms5  -notation (lower bound) Formal Definition of  -notation f(n) =  (g(n)) if  positive constants c, n 0 such that 0 ≤ cg(n) ≤ f(n),  n ≥ n 0  (g(n)) { f(n) :  positive constants c, n 0 such that 0 ≤ cg(n) ≤ f(n),  n ≥ n 0 }

Analysis of Algorithms6  -notation (lower bound)

Analysis of Algorithms7 Θ-notation (tight bound) Formal Definition of Θ -notation f(n) = Θ (g(n)) if  positive constants c 1, c 2, n 0 such that 0 ≤ c 1 g(n) ≤ f(n) ≤ c 2 g(n),  n ≥ n 0 Θ (g(n)) { f(n) :  positive constants c 1, c 2, n 0 such that 0 ≤ c 1 g(n) ≤ f(n) ≤ c 2 g(n),  n ≥ n 0 }

Analysis of Algorithms8 Θ-notation (tight bound) - Example

Analysis of Algorithms9 Θ-notation (tight bound) - 0 < c 1 ≤ h(n) ≤ c 2

Analysis of Algorithms10 Θ-notation (tight bound) - 0 < c 1 ≤ h(n) ≤ c 2

Analysis of Algorithms11 Θ-notation (tight bound) Θ (g(n)) { f(n) :  positive constants c 1, c 2, n 0 such that 0 ≤ c 1 g(n) ≤ f(n) ≤ c 2 g(n),  n ≥ n 0 }

Analysis of Algorithms12 Θ-notation

Analysis of Algorithms13 Θ-notation

Analysis of Algorithms14 Using O-notation for Describing Running Times O-notation is used to bound worst-case running times –bounds running time on arbitrary inputs as well O(n 2 ) bound on worst-case running time of insertion sort also applies to its running time on every input What we really mean by “running time of insertion sort is O(n 2 )” –worst-case running time of insertion sort is O(n 2 ) or equivalently –no matter what particular input of size n is chosen (for each value of) running time on that set of inputs is O(n 2 )

Analysis of Algorithms15 Using Ω-notation for Describing Running Times Ω(n) is used to bound the best-case running times –bounds the running time on arbitrary inputs as well e.g., Ω(n) bound on best-case running time of insertion sort –running time of insertion sort is Ω(n) “running time of an algorithm is Ω(g(n))” means –no matter what particular input of size n is chosen (for any n), running time on that set of inputs is at least a constant times g(n), for sufficiently large n –however, it is not contradictory to say “worst-case running time of insertion sort is Ω(n 2 )” since there exists an input that causes algorithm to take Ω(n 2 ) time

Analysis of Algorithms16 Using Θ-notation for Describing Running Times Case 1. used to bound worst-case & best-case running times of an algorithm if they are not asymptotically equal Case 2. used to bound running time of an algorithm if its worst & best case running times are asymptotically equal

Analysis of Algorithms17 Using Θ-notation for Describing Running Times Case (1) a Θ-bound on worst-/best-case running time does not apply to its running time on arbitrary inputs e.g., Θ(n 2 ) bound on worst-case running time of insertion sort does not imply a Θ(n 2 ) bound on running time of insertion sort on every input since T(n) = O(n 2 ) & T(n) = Ω(n) for insertion sort

Analysis of Algorithms18 Using Θ-notation for Describing Running Times Case (2) implies a Θ-bound on every input e.g., merge sort T(n) = O(nlgn) T(n) = Ω(nlgn)  T(n) = Θ(nlgn)

Analysis of Algorithms19 Asymptotic Notation In Equations

Analysis of Algorithms20 Asymptotic notation appears on LHS of an equation

Analysis of Algorithms21 Other asymptotic notations – o-notation (small o)

Analysis of Algorithms22 o-notation

Analysis of Algorithms23 ω-notation (small omega notation)

Analysis of Algorithms24 Asymptotic comparison of functions

Analysis of Algorithms25 Analogy to the comparison of two real numbers

Analysis of Algorithms26 Analogy to the comparison of two real numbers