Copyright © 2014 Curt Hill Growth of Functions Analysis of Algorithms and its Notation

Introduction We are interested how fast a function grows Usually this is related to the running time of an algorithm compared to the size of the input –We may also be interested in space required by an algorithm We may wish to estimate the running time of a program that implements an algorithm Copyright © 2014 Curt Hill

Speed of a program There are many factors of a program that have nothing to do with the underlying algorithm Clock speed of the computer Instruction set of the computer Language in which program is written Coding details We may change the running speed by changing any of the above Instead we are interested in those intrinsic factors of the algorithm itself Copyright © 2014 Curt Hill

Big O Notation The O stands for Order of This notation captures the most elementary function that describes how the running time varies with the input size We typically eliminate everything except the most rapidly growing function We also eliminate all constant factors as well Copyright © 2014 Curt Hill

Example Suppose that the running time of a program is given by the following function: f(x) = 1.8x x –Where x is the number of input items –Result is in some unit of time, eg milliseconds We would then say the program is O(x 2 ) Changing any number of things would change the constants but not the O Copyright © 2014 Curt Hill

Again What does this really mean? When we say our program with running time 1.8x x is O(x 2 ) what we really mean is the following: –There is a constant k –When x gets large then f(x) < kx 2 Since the square term will ultimately dominate all the lesser terms this will continue as x gets larger Copyright © 2014 Curt Hill

Big O Rules Again If f(x) is the sum of several functions the one with the fastest rate is retained and others discarded If there are constants that are multiplied times a function they are also ignored Copyright © 2014 Curt Hill

Common Os Constant – O(1) or O(c) Logarithmic – O(log 2 n) Linear – O(n) NlogN – O(nlog 2 n) Quadratic – O(n 2 ) Cubic – O(n 3 ) Polynomial – O(n c ), where c is a constant, i ncludes quadratic and cubic Exponential – O(2 n ) Factorial – O(n!) Copyright © 2014 Curt Hill

Picture Copyright © 2014 Curt Hill

Notation Again Sometimes we say f(x) is O(g(x)) –Eg. f(x) is O(n log n) This is sometimes written f(x) = O(g(x)) –This is strange, but accepted, use of = Thus O(x) = O(x 2 ) is not symmetric with O(x 2 ) = O(x) Instead what we are saying in O(x) = O(x 2 ) is that linear time is a subset of quadratic time –Eg. Has an upper bound Copyright © 2014 Curt Hill

Commentary Big O is all about the upper bound of a computation Algorithms with exponents large than four are unusual Polynomial and slower growing are tractable Exponential, factorial and faster growing algorithms are generally called intractable Copyright © 2014 Curt Hill

Tractable and Intractable Copyright © 2014 Curt Hill O(n) O(n log n) ,4699,966 n2n2 1002,50010,00090,0001 x 10 7 n3n ,0001 x x x n2n 1024~10 16 ~ ~ Big n!3.6 x 10 7 ~10 65 ~ ~ Big

Other Notations Big O is all about the upper bound of a computation Occasionally we want other bounds Big Omega (big  ) is used for lower bounds –There is k factor that will always be less than the running time Big Theta (big  ) is used for both upper and lower bounds –There are two constants that bracket upper and lower bounds Copyright © 2014 Curt Hill

Example Suppose that: f(x) = 2x 4 +42x x x What is big  ? Copyright © 2014 Curt Hill

Exercises 3.2 –3, 7, 19, 23, 27 Copyright © 2014 Curt Hill