CS 3343: Analysis of Algorithms Correctness Proof, Order of Growth, Asymptotic Notations 11/16/2018

What is an algorithm? Algorithms are the ideas behind computer programs. An algorithm is the thing that stays the same regardless of programming language and the computing hardware 11/16/2018

What is an algorithm? (cont’) An algorithm is a precise and unambiguous specification of a sequence of steps that can be carried out to solve a given problem or to achieve a given condition. An algorithm accepts some value or set of values as input and produces a value or set of values as output. Algorithms are closely intertwined with the nature of the data structure of the input and output values 11/16/2018

How to express algorithms? Nature language (e.g. English) Pseudocode Real programming languages Increasing precision Ease of expression Describe the ideas of an algorithm in nature language. Use pseudocode to clarify sufficiently tricky details of the algorithm. 11/16/2018

How to express algorithms? Nature language (e.g. English) Pseudocode Real programming languages Increasing precision Ease of expression To understand / describe an algorithm: Get the big idea first. Use pseudocode to clarify sufficiently tricky details 11/16/2018

Example: sorting Input: A sequence of N numbers a1…an Output: the permutation (reordering) of the input sequence such that a1 ≤ a2 … ≤ an. Possible algorithms you’ve learned so far Insertion, selection, bubble, quick, merge, … More in this course We seek algorithms that are both correct and efficient 11/16/2018

Insertion Sort InsertionSort(A, n) { for j = 2 to n { } ▷ Pre condition: A[1..j-1] is sorted 1. Find position i in A[1..j-1] such that A[i] ≤ A[j] < A[i+1] 2. Insert A[j] between A[i] and A[i+1] ▷ Post condition: A[1..j] is sorted 1 j sorted 11/16/2018

Insertion Sort InsertionSort(A, n) { for j = 2 to n { key = A[j]; i = j - 1; while (i > 0) and (A[i] > key) { A[i+1] = A[i]; i = i – 1; } A[i+1] = key } } 1 i j Key sorted 11/16/2018

Correctness What makes a sorting algorithm correct? In the output sequence, the elements are ordered non-decreasingly Each element in the input sequence has a unique appearance in the output sequence [2 3 1] => [1 2 2] X [2 2 3 1] => [1 1 2 3] X 11/16/2018

Correctness For any algorithm, we must prove that it always returns the desired output for all legal instances of the problem. For sorting, this means even if (1) the input is already sorted, or (2) it contains repeated elements. Algorithm correctness is NOT obvious in some problems (e.g., optimization) 11/16/2018

How to prove correctness? Given a concrete input, eg. <4,2,6,1,7> trace it and prove that it works. Given an abstract input, eg. <a1, … an> trace it and prove that it works. Sometimes it is easier to find a counterexample to show that an algorithm does NOT work. Think about all small examples Think about examples with extremes of big and small Think about examples with ties Failure to find a counterexample does NOT mean that the algorithm is correct 11/16/2018

An Example: Insertion Sort InsertionSort(A, n) { for j = 2 to n { key = A[j]; i = j - 1; ▷Insert A[j] into the sorted sequence A[1..j-1] while (i > 0) and (A[i] > key) { A[i+1] = A[i]; i = i – 1; } A[i+1] = key } } 1 i j Key sorted 11/16/2018

Example of insertion sort 5 2 4 6 1 3 2 5 4 6 1 3 2 4 5 6 1 3 2 4 5 6 1 3 1 2 4 5 6 3 1 2 3 4 5 6 Done! 11/16/2018

Use loop invariants to prove the correctness of loops A loop invariant (LI) is a formal statement about the variables in your program which holds true throughout the loop Claim: at the start of each iteration of the for loop, the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order. Proof by induction Initialization: the LI is true prior to the 1st iteration Maintenance: if the LI is true before the jth iteration, it remains true before the (j+1)th iteration Termination: when the loop terminates, the LI gives us a useful property to show that the algorithm is correct 11/16/2018

Loop invariants and correctness of insertion sort Claim: at the start of each iteration of the for loop, the subarray consists of the elements originally in A[1..j-1] but in sorted order. Proof: by induction 11/16/2018

Prove correctness using loop invariants InsertionSort(A, n) { for j = 2 to n { key = A[j]; i = j - 1; ▷Insert A[j] into the sorted sequence A[1..j-1] while (i > 0) and (A[i] > key) { A[i+1] = A[i]; i = i – 1; } A[i+1] = key } } Loop invariant: at the start of each iteration of the for loop, the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order. 11/16/2018

Initialization InsertionSort(A, n) { for j = 2 to n { key = A[j]; i = j - 1; ▷Insert A[j] into the sorted sequence A[1..j-1] while (i > 0) and (A[i] > key) { A[i+1] = A[i]; i = i – 1; } A[i+1] = key } } Subarray A[1] is sorted. So loop invariant is true before the loop starts. Loop invariant: at the start of each iteration of the for loop, the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order. 11/16/2018

Loop invariant: at the start of each iteration of the for loop, the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order. Maintenance InsertionSort(A, n) { for j = 2 to n { key = A[j]; i = j - 1; ▷Insert A[j] into the sorted sequence A[1..j-1] while (i > 0) and (A[i] > key) { A[i+1] = A[i]; i = i – 1; } A[i+1] = key } } Assume loop variant is true prior to iteration j Loop variant will be true before iteration j +1 1 i j Key sorted 11/16/2018

The algorithm is correct! Loop invariant: at the start of each iteration of the for loop, the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order. Termination InsertionSort(A, n) { for j = 2 to n { key = A[j]; i = j - 1; ▷Insert A[j] into the sorted sequence A[1..j-1] while (i > 0) and (A[i] > key) { A[i+1] = A[i]; i = i – 1; } A[i+1] = key } } The algorithm is correct! Upon termination, A[1..n] contains all the original elements of A in sorted order. 1 n j=n+1 Sorted 11/16/2018

Efficiency Correctness alone is not sufficient Brute-force algorithms exist for most problems To sort n numbers, we can enumerate all permutations of these numbers and test which permutation has the correct order Why cannot we do this? Too slow! By what standard? 11/16/2018

How to measure complexity? Accurate running time is not a good measure It depends on input It depends on the machine you used and who implemented the algorithm It depends on the weather, maybe  We would like to have an analysis that does not depend on those factors 11/16/2018

Machine-independent A generic uniprocessor random-access machine (RAM) model No concurrent operations Each simple operation (e.g. +, -, =, *, if, for) takes 1 step. Loops and subroutine calls are not simple operations. All memory equally expensive to access Constant word size Unless we are explicitly manipulating bits 11/16/2018

Running Time Number of primitive steps that are executed Except for time of executing a function call most statements roughly require the same amount of time y = m * x + b c = 5 / 9 * (t - 32 ) z = f(x) + g(x) We can be more exact if need be 11/16/2018

Running time of insertion sort The running time depends on the input: an already sorted sequence is easier to sort. Parameterize the running time by the size of the input, since short sequences are easier to sort than long ones. Generally, we seek upper bounds on the running time, because everybody likes a guarantee. 11/16/2018

Kinds of analyses Worst case Best case – not very useful Average case Provides an upper bound on running time An absolute guarantee Best case – not very useful Average case Provides the expected running time Very useful, but treat with care: what is “average”? Random (equally likely) inputs Real-life inputs 11/16/2018

Example: analysis of insertion Sort InsertionSort(A, n) { for j = 2 to n { key = A[j] i = j - 1; while (i > 0) and (A[i] > key) { A[i+1] = A[i] i = i - 1 } A[i+1] = key } } How many times will this line execute? 11/16/2018

Example: analysis of insertion Sort InsertionSort(A, n) { for j = 2 to n { key = A[j] i = j - 1; while (i > 0) and (A[i] > key) { A[i+1] = A[i] i = i - 1 } A[i+1] = key } } How many times will this line execute? 11/16/2018

Analysis of insertion Sort: exact Statement cost time__ InsertionSort(A, n) { for j = 2 to n { c1 n key = A[j] c2 (n-1) i = j - 1; c3 (n-1) while (i > 0) and (A[i] > key) { c4 S A[i+1] = A[i] c5 (S-(n-1)) i = i - 1 c6 (S-(n-1)) } 0 A[i+1] = key c7 (n-1) } 0 } S = t2 + t3 + … + tn where tj is number of while expression evaluations for the jth for loop iteration 11/16/2018

Analyzing Insertion Sort : exact T(n) = c1n + c2(n-1) + c3(n-1) + c4S + c5(S - (n-1)) + c6(S - (n-1)) + c7(n-1) = c8S + c9n + c10 What can S be? Best case -- inner loop body never executed tj = 1  S = n - 1 T(n) = an + b is a linear function Worst case -- inner loop body executed for all previous elements tj = j  S = 2 + 3 + … + n = n(n+1)/2 - 1 T(n) = an2 + bn + c is a quadratic function Average case Can assume that on average, we have to insert A[j] into the middle of A[1..j-1], so tj = j/2 S ≈ n(n+1)/4 T(n) is still a quadratic function Θ (n) Θ (n2) Θ (n2) 11/16/2018

Asymptotic Analysis Running time depends on the size of the input Larger array takes more time to sort T(n): the time taken on input with size n Look at growth of T(n) as n→∞. “Asymptotic Analysis” Size of input is generally defined as the number of input elements In some cases may be tricky 11/16/2018

Asymptotic Analysis Ignore actual and abstract statement costs Order of growth is the interesting measure: Highest-order term is what counts As the input size grows larger it is the high order term that dominates 11/16/2018

Comparison of functions log2n n nlog2n n2 n3 2n n! 10 3.3 33 102 103 106 6.6 660 104 1030 10158 109 13 105 108 1012 17 1010 1015 20 107 1018 For a super computer that does 1 trillion operations per second, it will be longer than 1 billion years 11/16/2018

Order of growth 1 << log2n << n << nlog2n << n2 << n3 << 2n << n! (We are slightly abusing of the “<<“ sign. It means a smaller order of growth). 11/16/2018

Asymptotic notations We say InsertionSort’s worst-case running time is Θ(n2) Properly we should say running time is in Θ(n2) It is also in O(n2 ) What’s the relationship between Θ and O? Formal definition soon 11/16/2018

Analysis of insertion Sort: Asymptotic Statement cost time__ InsertionSort(A, n) { for j = 2 to n { c1 n key = A[j] c2 (n-1) i = j - 1; c3 (n-1) while (i > 0) and (A[i] > key) { c4 S A[i+1] = A[i] c5 (S-(n-1)) i = i - 1 c6 (S-(n-1)) } 0 A[i+1] = key c7 (n-1) } 0 } What are the basic operations (most executed lines)? 11/16/2018

Analysis of insertion Sort: Asymptotic Statement cost time__ InsertionSort(A, n) { for j = 2 to n { c1 n key = A[j] c2 (n-1) i = j - 1; c3 (n-1) while (i > 0) and (A[i] > key) { c4 S A[i+1] = A[i] c5 (S-(n-1)) i = i - 1 c6 (S-(n-1)) } 0 A[i+1] = key c7 (n-1) } 0 } 11/16/2018

Analysis of insertion Sort: Asymptotic Statement cost time__ InsertionSort(A, n) { for j = 2 to n { c1 n key = A[j] c2 (n-1) i = j - 1; c3 (n-1) while (i > 0) and (A[i] > key) { c4 S A[i+1] = A[i] c5 (S-(n-1)) i = i - 1 c6 (S-(n-1)) } 0 A[i+1] = key c7 (n-1) } 0 } 11/16/2018

What can S be? S =  j=2..n tj Best case: Worst case: Average case: 1 Inner loop stops when A[i] <= key, or i = 0 1 i j Key sorted S =  j=2..n tj Best case: Worst case: Average case: 11/16/2018

Best case Array already sorted S =  j=2..n tj tj = 1 for all j Inner loop stops when A[i] <= key, or i = 0 1 i j Key sorted Array already sorted S =  j=2..n tj tj = 1 for all j S = n-1 T(n) = Θ (n) 11/16/2018

Worst case Array originally in reversely sorted order S =  j=2..n tj Inner loop stops when A[i] <= key 1 i j Key sorted Array originally in reversely sorted order S =  j=2..n tj tj = j S =  j=2..n j = 2 + 3 + … + n = (n-1) (n+2) / 2 = Θ (n2) 11/16/2018

Average case Array in random order S =  j=2..n tj Inner loop stops when A[i] <= key 1 i j Key sorted Array in random order S =  j=2..n tj tj = j / 2 on average S =  j=2..n j/2 = ½  j=2..n j = (n-1) (n+2) / 4 = Θ (n2) What if we use binary search? Answer: still Θ(n2) 11/16/2018

Exact analysis is hard and unnecessary! Worst-case and average-case are difficult to deal with precisely, because the details can be very complicated It may be easier to talk about upper and lower bounds of the function. 11/16/2018

Asymptotic notations O: Big-Oh Ω: Big-Omega Θ: Theta o: Small-oh ω: Small-omega 11/16/2018

Big O Informally, O (g(n)) is the set of all functions with a smaller or same order of growth as g(n), within a constant multiple If we say f(n) is in O(g(n)), it means that g(n) is an asymptotic upper bound of f(n) Intuitively, it is like f(n) ≤ g(n) What is O(n2)? The set of all functions that grow slower than or in the same order as n2 Abuse of notation (for convenience): f(n) = O(g(n)) actually means f(n)  O(g(n)) 11/16/2018

So: But: n  O(n2) n2  O(n2) 1000n  O(n2) n2 + n  O(n2) Intuitively, O is like ≤ 11/16/2018

(optional) small o Informally, o (g(n)) is the set of all functions with a strictly smaller growth as g(n), within a constant multiple What is o(n2)? The set of all functions that grow slower than n2 So: 1000n  o(n2) But: n2  o(n2) Intuitively, o is like < 11/16/2018

Big Ω Informally, Ω (g(n)) is the set of all functions with a larger or same order of growth as g(n), within a constant multiple f(n)  Ω(g(n)) means g(n) is an asymptotic lower bound of f(n) Intuitively, it is like g(n) ≤ f(n) So: n2  Ω(n) 1/1000 n2  Ω(n) But: 1000 n  Ω(n2) Intuitively, Ω is like ≥ Abuse of notation (for convenience): f(n) = Ω(g(n)) actually means f(n)  Ω(g(n)) 11/16/2018

(optional) small ω Informally, ω (g(n)) is the set of all functions with a larger order of growth as g(n), within a constant multiple So: n2  ω(n) 1/1000 n2  ω(n) n2  ω(n2) Intuitively, ω is like > 11/16/2018

Theta (Θ) Informally, Θ (g(n)) is the set of all functions with the same order of growth as g(n), within a constant multiple f(n)  Θ(g(n)) means g(n) is an asymptotically tight bound of f(n) Intuitively, it is like f(n) = g(n) What is Θ(n2)? The set of all functions that grow in the same order as n2 Abuse of notation (for convenience): f(n) = Θ(g(n)) actually means f(n)  Θ(g(n)) Θ(1) means constant time. 11/16/2018

So: But: n2  Θ(n2) n2 + n  Θ(n2) 100n2 + n  Θ(n2) 100n2 + log2n  Θ(n2) But: nlog2n  Θ(n2) 1000n  Θ(n2) 1/1000 n3  Θ(n2) Intuitively, Θ is like = 11/16/2018

Tricky cases How about sqrt(n) and log2 n? How about log2 n and log10 n How about 2n and 3n How about 3n and n!? 11/16/2018

Mathematical definitions (more discussions later) O(g(n)) = {f(n):  positive constants c and n0 such that 0 ≤ f(n) ≤ cg(n)  n≥n0} Ω(g(n)) = {f(n):  positive constants c and n0 such that 0 ≤ cg(n) ≤ f(n)  n≥n0} Θ(g(n)) = {f(n):  positive constants c1, c2, and n0 such that 0  c1 g(n)  f(n)  c2 g(n)  n  n0} 11/16/2018

O, Ω, and Θ The definitions imply a constant n0 beyond which they are satisfied. We do not care about small values of n. 11/16/2018

Using limits to compare functions for order of growth lim f(n) / g(n) = c > 0 ∞ f(n)  o(g(n)) f(n)  O(g(n)) f(n)  Θ (g(n)) n→∞ f(n)  Ω(g(n)) f(n)  ω (g(n)) 11/16/2018

logarithms compare log2n and log10n logab = logcb / logca log2n = log10n / log102 ~ 3.3 log10n Therefore lim(log2n / log10 n) = 3.3 log2n = Θ (log10n) 11/16/2018

Therefore, 2n  o(3n), and 3n  ω(2n) How about 2n and 2n+1? Compare 2n and 3n lim 2n / 3n = lim(2/3)n = 0 Therefore, 2n  o(3n), and 3n  ω(2n) How about 2n and 2n+1? 2n / 2n+1 = ½, therefore 2n = Θ (2n+1) n→∞ n→∞ 11/16/2018

L’ Hopital’s rule lim f(n) / g(n) = lim f(n)’ / g(n)’ You can apply this transformation as many times as you want, as long as the condition holds lim f(n) / g(n) = lim f(n)’ / g(n)’ Condition: If both lim f(n) and lim g(n) are  or 0 n→∞ n→∞ 11/16/2018

In fact, log n  o(nε), for any ε > 0 Compare n0.5 and log n lim n0.5 / log n = ? (n0.5)’ = 0.5 n-0.5 (log n)’ = 1 / n lim (n-0.5 / 1/n) = lim(n0.5) = ∞ Therefore, log n  o(n0.5) In fact, log n  o(nε), for any ε > 0 n→∞ 11/16/2018

Stirling’s formula (constant) 11/16/2018

Compare 2n and n! Therefore, 2n = o(n!) Compare nn and n! Therefore, nn = ω(n!) How about log (n!)? Θ (n log n) 11/16/2018

About exponential and logarithm functions Textbook page 55-56 It is important to understand what logarithms are and where they come from. A logarithm is simply an inverse exponential function. Saying bx = y is equivalent to saying that x = logb y. Logarithms reflect how many times we can double something until we get to n, or halve something until we get to 1. log21 = ? log22 = ? 11/16/2018

Binary Search In binary search we throw away half the possible number of keys after each comparison. How many times can we halve n before getting to 1? Answer: ceiling (lg n) 11/16/2018

Logarithms and Trees How tall a binary tree do we need until we have n leaves? The number of potential leaves doubles with each level. How many times can we double 1 until we get to n? Answer: ceiling (lg n) 11/16/2018

Logarithms and Bits How many numbers can you represent with k bits? Each bit you add doubles the possible number of bit patterns You can represent from 0 to 2k – 1 with k bits. A total of 2k numbers. How many bits do you need to represent the numbers from 0 to n? ceiling (lg (n+1)) 11/16/2018

logarithms lg n = log2 n ln n = loge n, e ≈ 2.718 lgkn = (lg n)k lg lg n = lg (lg n) = lg(2)n lg(k) n = lg lg lg … lg n lg24 = ? lg(2)4 = ? Compare lgkn vs lg(k)n? 11/16/2018

Useful rules for logarithms For all a > 0, b > 0, c > 0, the following rules hold logba = logca / logcb = lg a / lg b logban = n logba blogba = a log (ab) = log a + log b lg (2n) = ? log (a/b) = log (a) – log(b) lg (n/2) = ? lg (1/n) = ? logba = 1 / logab 11/16/2018

Useful rules for exponentials For all a > 0, b > 0, c > 0, the following rules hold a0 = 1 (00 = ?) a1 = a a-1 = 1/a (am)n = amn (am)n = (an)m aman = am+n 11/16/2018

More advanced dominance ranking 11/16/2018

Definition & Proof by definition 11/16/2018

Big-Oh There exist For all Definition: O(g(n)) = {f(n):  positive constants c and n0 such that 0 ≤ f(n) ≤ cg(n)  n≥n0} 11/16/2018

Big-Oh Claim: f(n) = 3n2 + 10n + 5  O(n2) Prove by definition (Hint: to prove this claim by definition, we need to find some positive constants c and n0 such that f(n) <= cn2 for all n ≥ n0.) (Note: you just need to find one concrete example of c and n0 satisfying the condition, but it needs to be correct for all n ≥ n0. So do not try to plug in a concrete value of n and show the inequality holds.) Proof: 3n2 + 10n + 5  3n2 + 10n2 + 5,  n ≥ 1  3n2 + 10n2 + 5n2, n ≥ 1  18 n2,  n ≥ 1 If we let c = 18 and n0 = 1, we have f(n)  c n2,  n ≥ n0. Therefore by definition, f(n)  O(n2). 11/16/2018

Big-Omega Definition: Ω(g(n)) = {f(n):  positive constants c and n0 such that 0 ≤ cg(n) ≤ f(n)  n≥n0} 11/16/2018

Big-Omega Claim: f(n) = n2 / 10 = Ω(n) Prove by definition: f(n) = n2 / 10, g(n) = n Need to find a c and a no to satisfy the definition of f(n)  Ω(g(n)), i.e., f(n) ≥ cg(n) for n ≥ n0 Proof: n ≤ n2 / 10 when n ≥ 10 If we let c = 1 and n0 = 10, we have f(n) ≥ cn,  n ≥ n0. Therefore, by definition, n2 / 10 = Ω(n). 11/16/2018

Theta Definition: Alternatively: f(n) = O(g(n)) and f(n) = Ω(g(n)) Θ(g(n)) = {f(n):  positive constants c1, c2, and n0 such that 0  c1 g(n)  f(n)  c2 g(n),  n  n0 } Alternatively: f(n) = O(g(n)) and f(n) = Ω(g(n)) 11/16/2018

Theta Claim: f(n) = 2n2 + n = Θ (n2) Prove by definition: Need to find the three constants c1, c2, and n0 such that c1n2 ≤ 2n2+n ≤ c2n2 for all n ≥ n0 A simple solution is c1 = 2, c2 = 3, and n0 = 1 11/16/2018

More Examples Prove n2 + 3n + lg n is in O(n2) Need to find c and n0 such that n2 + 3n + lg n <= cn2 for n ≥ n0 Proof: n2 + 3n + lg n <= n2 + 3n2 + n for n ≥ 1 <= n2 + 3n2 + n2 for n ≥ 1 <= 5n2 for n ≥ 1 Therefore, by definition n2 + 3n + lg n  O(n2). (Alternatively: n2 + 3n + lg n <= n2 + n2 + n2 for n ≥ 10 <= 3n2 for n ≥ 10) 11/16/2018

More Examples Prove n2 + 3n + lg n is in Ω(n2) Want to find c and n0 such that n2 + 3n + lg n >= cn2 for n ≥ n0 n2 + 3n + lg n >= n2 for n ≥ 1 n2 + 3n + lg n = O(n2) and n2 + 3n + lg n = Ω (n2) => n2 + 3n + lg n = Θ(n2) 11/16/2018

(FYI: How to prove logn < n) Let f(n) = 1 + log n, g(n) = n. Then f(n)’ = 1/n g(n)’ = 1. f(1) = g(1) = 1. Because f(n)’ ≤ g(n)’  n ≥ 1, by the racetrack principle, we have f(n) ≤ g(n)  n ≥ 1, i.e., 1 + log n ≤ n. Therefore, log n < 1 + log n ≤ n for all n ≥ 1. From now on, we will use that fact that log n < n  n ≥ 1 without proof. 11/16/2018

Proof for abstract functions Example: for any two asymptomatically positive functions f(n) and g(n) that satisfy f(n)  O(g(n)), prove that g(n)  Θ(f(n) + g(n)) Proof: Since f(n) is asymptotically positive, we have g(n)  f(n) + g(n) for sufficiently large n. Therefore, by definition g(n)  O(f(n) + g(n)). We now need to show g(n)  Ω(f(n) + g(n)). 11/16/2018

Proof for abstract functions (cont’d) Given: f(n)  O(g(n)) Need to show: g(n)  Ω(f(n) + g(n)) f(n)  O(g(n)) implies that f(n)  cg(n) for sufficiently large n => f(n) + g(n)  cg(n) + g(n) => f(n) + g(n)  (c+1)g(n) => (f(n) + g(n)) / (c+1)  g(n) => g(n)  Ω(f(n) + g(n)). g(n)O(f(n) + g(n)) and g(n)  Ω(f(n) + g(n)) => g(n)  Θ(f(n) + g(n)) 11/16/2018

Properties of asymptotic notations Textbook page 51 Transitivity f(n) = (g(n)) and g(n) = (h(n)) => f(n) = (h(n)) (holds true for o, O, , and  as well). Symmetry f(n) = (g(n)) if and only if g(n) = (f(n)) Transpose symmetry f(n) = O(g(n)) if and only if g(n) = (f(n)) f(n) = o(g(n)) if and only if g(n) = (f(n)) 11/16/2018

Asymptotic notations: Summary O: Big-Oh Ω: Big-Omega Θ: Theta o: Small-oh ω: Small-omega Intuitively: O is like  o is like <  is like   is like >  is like = 11/16/2018

Mathematical definitions O(g(n)) = {f(n):  positive constants c and n0 such that 0 ≤ f(n) ≤ cg(n)  n≥n0} Ω(g(n)) = {f(n):  positive constants c and n0 such that 0 ≤ cg(n) ≤ f(n)  n≥n0} Θ(g(n)) = {f(n):  positive constants c1, c2, and n0 such that 0  c1 g(n)  f(n)  c2 g(n)  n  n0} 11/16/2018