1. O -Notation April 23, 2003 Prepared by Doug Hogan CSE 260

2. O -notation: The Idea Big-O notation is a way of ranking about how much time it takes for an algorithm to execute How many operations will be done when the program is executed? Find a bound on the running time, i.e. functions that are on the same order. We care about what happens for large amounts of data  asymptotic order.

3. O -notation: The Idea Use mathematical tools to find asymptotic order. Real functions to approximate integer functions. Depends on some variable, like n or X, which is usually the size of an array or how much data is going to be processed

4. O -notation The complicated math behind it all… Given f and g, real functions of variable x … First form: g provides an upper bound for f ≡ graph of f lies closer to the x axis than g More general form: g provides an upper bound for f ≡ graph of f lies closer to the x axis than some positive multiple ( M ) of g after some minimum value of x ( x 0 ).

5. O -notation: A graphical view

6. So what does “closer to the x- axis” MEAN? -M ∙ g(x) ≤ f(x) ≤ M ∙ g(x) But that’s absolute value… |f(x)| ≤ M ∙ |g(x)|

7. Another graphical view x0x0 g M ∙g f x y Before x 0, nothing claimed about f’s growth After x 0, |f(x)| ≤ M ∙ |g(x)|

8. Formal Definition Let f and g be real-valued functions defined on the same set of reals. f is of order g, written f ( x ) = O ( g(x) ), iff there exists a positive real number M (multiple) a real number x 0 (starting point) such that for all x in the domain of f and g, |f(x)| ≤ M ∙ |g(x)| when x > x 0

9. Example Use the definition of O -notation to express |17 x 6 – 3 x 3 + 2 x + 8| ≤ 30| x 6 | for all x > 1 M = 30 x 0 = 1 17 x 6 – 3 x 3 + 2 x + 8 is O( x 6 )

10. Graphically… 17 x 6 – 3 x 3 + 2 x + 8 is O( x 6 ); M = 30; x 0 = 1 30x 6 x6x6 17x6 – 3x3 + 2x + 8

11. Problem Use the definition of O -notation to express for all x > 6 M = 45 x 0 = 6

12. Graphically… M = 45; x 0 = 6

13. Another graphical example f ( x ) = 7 x 3 - 2 x + 3 12x 3 x3x3 M = 12, x 0 = 1 7 x 3 - 2 x + 3 is O ( x 3 )

14. Using O -notation… Order of Power Functions: For any rational numbers r and s, if r < s, x r is O ( x s ) Order of Polynomial Functions: If a 0, a 1,…, a n are real numbers and a n ≠ 0 a n x n + a n -1 x n -1 +… + a 1 x + a 0 is O ( x m ) for all m ≥ n

15. Examples Example: Find an order for f(x) = 7x 5 + 5x 3 – x + 4 (all reals x) O(x 5 ) Is that the only answer? No… But it’s the “best”

16. Showing that a function is NOT Big-O of another… Show that x 2 is not O(x). [Arguing by contradiction.] Suppose not, that x 2 is O(x). By definition of O(…), then there exist a positive real number M a real number x 0 such that | x 2 | ≤ M ∙ | x| for all x > x 0 (1)

17. Showing that a function is NOT Big-O of another…, ctd. Let x be a positive real number greater than both M and x 0, i.e. x > M and x > x 0. Then by multiplying both sides of x > M by x, x∙x > M∙x. Since x is positive, | x 2 | > M∙|x|. So there is a real number x > x 0 s.t. | x 2 | > M∙|x|. This contradicts (1) above. So, the supposition is false and thus x 2 is not O( x ). □

18. Generalization If a 0, a 1,…, a n are real numbers and a n ≠ 0 a n x n + a n -1 x n -1 +… + a 1 x + a 0 is NOT O ( x m ) for all m < n

19. Best approximation Definition Suppose S is a set of functions from a subset of R to R and f is a R->R function. Function g is a best big-O approximation for f in S iff f ( x ) is O ( g ( x )) for any h in S, if f(x) is O(h(x)), then g(x) is O ( h ( x )).

20. Problem Find best big-O approx for f(x) = 5x 3 – 2x + 1 By thm. on polynomial orders, f ( x ) is O ( x n ) for all n ≥ 3 By previous property, f ( x ) is NOT O ( x m ) for all m < 3 So O ( x 3 ) is the best approximation.

21. O -Arithmetic Let f and g be functions and k be a constant. 1. O( k * f ) = O( f ) 2. O( f * g ) = O( f ) * O(g) 3. O( f / g ) = O( f ) / O(g) 4. O( f ) ≥ O( g) iff f dominates* g 5. O( f + g ) = Max[O( f ), O( g )] (Headington 546)

22. Dominance Let f and g be functions and a,b,m,n be constants. 1. x x dominates x ! 2. x ! dominates a x 3. a x dominates b x if a > b 4. a x dominates x n if n > m 5. x dominates log a x if a > 1 6. log a x dominates log b x if b > a > 1 7. log a x dominates 1if a > 1 (Headington 547)

23. Works Cited Epp, Susanna. Discrete Mathematics with Applications. 2nd Ed. Belmont, CA: Brooks, 1995. Headington, Mark A., and David Riley. Data Abstraction and Structures using C++. Lexington, MA: Heath, 1994.