CS2336: Computer Science II Today’s Agenda Asymptotic Analysis of algorithm A good formal mathematical understanding of algorithms running times and Big O notation We measure algorithms and data structures against another using Asymptotic Analysis CS2336: Computer Science II

CS2336: Computer Science II Example Suppose you write a program that manages inventory, for a nation wide chain of stores every night you have to process all of the changes that happened in inventory in every store across the country that day! 10,000 ms to read inventory from the disk 10ms to process each transactions for n, when n is a large number, the total time takes: (10,000+10n) ms for example there are 1 million transactions a day! CS2336: Computer Science II

CS2336: Computer Science II Asymptotic Analysis Study of how algorithms behave as size of the data you processing grows very large! As it goes to infinity! Big O notation , represent upper bound in a function. Has a very formal mathematical meaning! CS2336: Computer Science II

CS2336: Computer Science II Big O notation Let n be size of the program’s input size can be number of bits that our program has to read from disk for the inventory example! it can be numbers of items, how many items I want to insert to my list! for Asymptotic analysis we don’t care how exactly n is measured! Let T(n) be a real world function, that describe some actual process in real. for example an actual running time of your inventory program in ms on an input of n items. T(n) = (10,000 + 10n) Let F(n) be another function-preferably simple function! (we want to use it as a upper bound) CS2336: Computer Science II

CS2336: Computer Science II Informal Big O Complexity of T(n) is O( F(n) ) if and only if T(n) <= c F(n) whenever when n is big and for some large constant c. How big is big? choose n big enough to make T(n) function to fit under F(n) How large the c have to be? choose c big enough to make T(n) function to fit under F(n) CS2336: Computer Science II

Example T(n) = 10,000+10n lets try f(n) = n, c=20 cf(n) = 20n T(n) 30000 how they behave when n go to infinity! T(n)<=cf(n) for any n>=1000 SO T(n) complexity is O(n) 20000 10000 1000 CS2336: Computer Science II

CS2336: Computer Science II Formal Big O O(f(n)) is an infinite set of all functions T(n) that satisfy : There exist positive constants c and N such that, for all n >= N , T(n) <= c f(n) N is 1000 in our example CS2336: Computer Science II

CS2336: Computer Science II Example 1,000,000n  O(n) Big O doesn’t care about the constant factors! n3+n2+1  O(n3) Big O is usually used to pick the fastest growing term. CS2336: Computer Science II

Table of important big O sets Function common name O(1) constant O(log n) logarithmic O(log 2 n ) log- squared O(n1/2 ) root n O(n) linear O(n log n) O(n2) quadratic O(n3) cubic O(n4) quartic O(2n) exponential O(en) exponential (but very more so!) O(n!) < O(nn) Asymptotically en is much more faster than 2n CS2336: Computer Science II

CS2336: Computer Science II Logarithms If you don’t know: logab is equal to logcb / logca you need to review logarithms again! CS2336: Computer Science II

binary search : recursion tree T(n) = T(n/2) + 1 = T(n/22) + 2 = T(n/23) + 3 = T(n/24) + 4 = …….. = T(n/2k) + k = T(1) + k = 1 + log2n O(log2n) if n=1 => T(1) = 1 n/2k = 1 => k = log2 n https://math.dartmouth.edu/archive/m19w03/public_html/Section5-1.pdf CS2336: Computer Science II

A finished recursion tree CS2336: Computer Science II

CS2336: Computer Science II The Master Theorem n is the size of the problem. a is the number of sub-problems in the recursion. n/b is the size of each sub-problem. f (n) is the cost of the work done outside the recursive calls, which includes the cost of dividing the problem and the cost of merging the solutions to the sub-problems. CS2336: Computer Science II

The Master Theorem The master theorem concerns recurrence relations of the form: f(n) = nc 1. if logb a < c, T(n) = O(nc), 2. if logb a = c, T(n) = O(nc log n), 3. if logb a > c, T(n) = O(nlogb a). if n = 1 if n > 1 https://math.dartmouth.edu/archive/m19w03/public_html/Section5-2.pdf

Example T(n) = 8T(n/2) + 1000n2 T(n) = T(n/2) + 1 T(n) = 9T(n/3) +

Example T(n) = 8T(n/2) + 1000n2 a = 8 , b = 2, c = 2 logba = log28= 3 > c  case 3  T(n) = O(n3) T(n) = T(n/2) + 1 a = 1 , b = 2, c = 0 logba = log21= 0 = c  case 2  T(n) = O(n0logn) = O(logn) T(n) = 9T(n/3) + n a = 9, b = 3, c = 1 logba = log39= 2 > c  case 3  T(n) = O(n2) T(n) = 2T(n/2) + n a=2,b=2,c=1 logba = log22= 1 = c  case 2  T(n) = O(n1logn) = O(n logn) T(n) = 3T(n/3) + n2 a=3,b=3,c=2 logba = log33= 1 < c  case 1  T(n) = O(n2)