CS 312: Algorithm Analysis This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License. CS 312: Algorithm Analysis Lecture #2: Asymptotic Notation Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick

Quiz What is the reward for submitting project reports early? How many penalty-free late days are in your budget for the semester? How many penalty-free late days can you use on any one project? What is the penalty for late submission of project reports (after you’ve used your penalty-free late days)? T/F: You may submit late regular homework assignments. Syllabus policies

Announcements Your initial experiences with Visual Studio? My office hours updated: Tu & Th 2-3 and by appointment TA office hours are on the course wiki Different kinds of TAs Project #1 Instructions are linked from the online schedule Help session with TA Thursday at 4pm In the Windows help lab (1066 TMCB) Focus on intro. to C# and Visual Studio We’ll cover the mathematical ideas in class Purpose of help sessions

Objectives Revisit orders of growth Formally introduce asymptotic notation: O, W, Q Classify functions in asymptotic orders of growth

Orders of Growth Efficiency: how cost grows with the size/ difficulty n of a given problem instance. C(n): the cost (e.g., number of steps) required by an algorithm on an input of size / difficulty n. Order of growth: the functional form of C(n) up to a constant multiple as n goes to infinity.

Orders of Growth n log2n nlog2n n2 n3 10 3.3 3.3*10 102 103 6.6 6.6*102 104 106 1.0*104 109 13 1.3*105 108 1012 105 17 1.7*106 1010 1015 20 2.0*107 1018

Orders of Growth Efficient n log2n nlog2n n2 n3 2n n! 10 3.3 3.3*10 102 103 3.6*106 6.6 6.6*102 104 106 1.3*1030 9*10157 1.0*104 109 … 13 1.3*105 108 1012 105 17 1.7*106 1010 1015 20 2.0*107 1018 Efficient

Kinds of Efficiency Need to decide which instance of a given size to use as the representative for that class: Algorithm Domain Worst Case Best Case Instances Average Case (over all possible instances) 1 2 3 4 5 Instance size

Asymptotic Notation Definition: given an asymptotically non-negative fn. g(n), Translation: An asymptotically non-negative function f(n) belongs to the set Q(g(n)) iff there exist positive constants c1 and c2 such that f(n) can be “sandwiched” between c1g(n) and c2g(n), scaled versions of g(n), for sufficiently large n. Convention:

Asymptotic Notation Definitions: given an asymptotically non-negative fn. g(n),

Asymptotic Notation f(n) c g(n) n0 f(n) c g(n) n0 n0 c1g(n) c2g(n)

Distinguishing 𝑂 from Θ Can you draw a function that is in 𝑂 𝑛 2 but not Θ 𝑛 2 ?

Distinguishing 𝑂 from Θ Can you draw a function that is in 𝑂 𝑛 2 but not Θ 𝑛 2 ?

Example Show that Must find positive constants c1, c2, n0 such that Divide through by n2 to get Hint: consider one side of the inequality at a time: for n0=7 we have c1 1/14 for n0=6 we have c2 >1/2; works for larger n0=7 as well This proof is constructive

Another Example Show that 6 𝑛 3 ∉𝛩 𝑛 2 Use proof by contradiction: assume that 6 𝑛 3 ∈Θ 𝑛 2 Suppose positive constants c2, n0 exist such that 6 𝑛 3 ≤ 𝑐 2 𝑛 2 ∀𝑛≥ 𝑛 0 But this implies that 𝑛≤ 𝑐 2 6 ∀𝑛≥ 𝑛 0 i.e., 𝑛 is bounded by 𝑐 2 6 , a constant But 𝑛 is unbounded; hence we have a contradiction. Thus, our assumption was false; hence we have shown that 6 𝑛 3 ∉𝛩 𝑛 2

Other proof types? Induction Deduction Others! Optional example on HW #1 Deduction Others!

Duality

Duality g(n) in O(f(n)) means all instances of size n are solvable within c1f(n) time. f(n) in Ω(g(n)) means at least one instance of size n is solvable in c2g(n) time. … for worst case performance.

Duality Suppose t(n) models worst case behavior of an algorithm. Then t(n) = O(n!) means every instance of size n is solvable in factorial time. Then t(n) = (n!) means at least one instance of size n requires factorial time. But infinitely many size-n instances may require n2 time! Using our definition of , a  bound on the worst case isn’t useful. 1. Change the definition. But then duality fails. 2. Live with it. That’s what we’ll do. Similarly, using duality we also conclude that a O bound on a best case isn’t useful!

The Limit Rule

The Limit Rule

Example

Example

Useful Identity: L’Hopital’s Rule Applicable when lim f(n) = lim g(n) = 0 and when lim f(n) = lim g(n) = 

Review: Log Identities

Review: Log Identities

Review: More Logarithms

Review: More Logarithms

Assignment Read: Section 1.3 HW #1: 0.1 (a-e, g, m) in the textbook Optional: 0.3 (a) Remember: Don’t spend more than 2 focused hours on the homework exercises. If you reach 2 hours of concentrated effort, stop and make a note to that effect on your paper (unless you’re having fun and want to continue). The TAs will take that into consideration when grading.

The following slides are extra

Addition

Addition

Multiplication

Multiplication

Classic Multiplication American Style English Style 5001 x 502 10002 + 25005 2510502 5001 x 502 25005 + 10002 2510502 O(n2) for 2 n-digit numbers

Multiplication a la Francais / Russe

Multiplication a la Francais / Russe

Mulitplication a la Francais / Russe function multiply(x,y) Input: Two n-bit integers x and y, where y  0 Output: Their product if y = 0: return 0 if y = 1: return x z = multiply(x, floor(y/2)) if y is even: return 2z else: return x+2z

Mulitplication a la Francais / Russe function multiply(x,y) Input: Two n-bit integers x and y, where y  0 Output: Their product if y = 0: return 0 if y = 1: return x z = multiply(x, floor(y/2)) if y is even: return 2z else: return x+2z

Mulitplication a la Francais / Russe function multiply(x,y) Input: Two n-bit integers x and y, where y>=0 Output: Their product if y = 0: return 0 if y = 1: return x z = multiply(x, floor(y/2)) if y is even: return 2z else: return x+2z

Multiplication a la russe 981 1234 1234 490 2468 245 4936 4936 122 9872 61 19744 19744 30 39488 15 78976 78976 7 157952 157952 3 315904 315904 1 631808 + 631808 1210554. Don’t need to know multiplication tables Just need to know how to double, halve, and add!

Multiplication a la russe 101 10100 110012 = 2510 1012 x2 1012 101 101 101 10 1010 0 1 10100 10100 110012 = 2510 do this example in binary because it’s a little easier to see how it works. O(n2)

Arabic Multiplication 9 8 1 1 2 3 4 0 0 0 1 1 0 2 2 0 3 3 0 9 8 1 8 6 2 7 4 3 6 2 4 5 5 4

Division function divide(x,y) Input: Two n-bit integers x and y, where y  1 Output: The quotient and remainder of x divided by y if x=0: return (q, r) = (0,0) (q, r) = divide(floor(x/2),y) q=2*q, r=2*r if x is odd: r=r+1 if r  y: r=r-y, q=q+1 return (q, r)

Division function divide(x,y) Input: Two n-bit integers x and y, where y  1 Output: The quotient and remainder of x divided by y if x=0: return (q, r) = (0,0) (q, r) = divide(floor(x/2),y) q=2*q, r=2*r if x is odd: r=r+1 if r  y: r=r-y, q=q+1 return (q, r)