Download presentation
Presentation is loading. Please wait.
Published byAldous Jessie Rose Modified over 9 years ago
1
Asymptotic Growth Rates Themes Analyzing the cost of programs Ignoring constants and Big-Oh Recurrence Relations & Sums Divide and Conquer Examples Sort Computing powers Euclidean algorithm (computing gcds) Integer Multiplication
2
Asymptotic Growth Rates – “Big-O” (upper bound) f(n) = O(g(n)) [f grows at the same rate or slower than g] iff: There exists positive constants c and n 0 such that f(n) c g(n) for all n n 0 f is bound above by g Note: Big-O does not imply a tight bound Ignore constants and low order terms
3
Big-O, Examples E.G. 1: 5n 2 = O(n 3 ) c = 1, n 0 = 5: 5n 2 n n 2 = n 3 E.G. 2: 100n 2 = O(n 2 ) c = 100, n 0 = 1 E.G. 3: n 3 = O(2 n ) c = 1, n 0 = 12 n 3 (2 n/3 ) 3, n 2 n/3 for n 12 [use induction]
4
Little-o Loose upper bound f(n) = o(g(n)) [f grows strictly slower than g] f(n) = O(g(n)) and g(n) O(f(n)) lim n f(n)/g(n) = 0 “f is bound above by g, but not tightly”
5
Little-o, restatement lim n f(n)/g(n) = 0 f(n) = o(g(n)) >0, n 0 s.t. n n 0, f(n)/g(n) <
6
Equivalence - Theta f(n) = (g(n)) [grows at the same rate] f(n) = O(g(n)) and g(n) = O(f(n)) g(n) = (f(n)) lim n f(n)/g(n) = c, c ≠ 0 f(n) = (g(n)) “f is bound above by g, and below by g”
7
Common Results [j < k] lim n n j /n k = lim n 1/n (k-j) = 0 n j = o(n k ), if j<k [c < d] lim n c n /d n = lim n (c/d) n = 0 c n = o(d n ), if c<d lim n ln(n)/n = / lim n ln(n)/n = lim n (1/n)/1 = 0 [L’Hopital’s Rule] ln(n) = o(n) [ > 0] ln(n) = o(n ) [similar calculation]
8
Common Results [c > 1, k an integer] lim n n k /c n = / lim n kn k-1 / c n ln(c) lim n k(k-1)n k-2 / c n ln(c) 2 … lim n k(k-1)…(k-1)/c n ln(c) k = 0 n k = o(c n )
9
Asymptotic Growth Rates (log(n)) – logarithmic [log(2n)/log(n) = 1 + log(2)/log(n)] (n) – linear [double input double output] (n 2 ) – quadratic [double input quadruple output] (n 3 ) – cubit [double input output increases by factor of 8] (n k ) – polynomial of degree k (c n ) – exponential [double input square output]
10
Asymptotic Manipulation (cf(n)) = (f(n)) (f(n) + g(n)) = (f(n)) if g(n) = O(f(n))
11
Computing Time Functions Computing time function is the time to execute a program as a function of its inputs Typically the inputs are parameterized by their size [e.g. number of elements in an array, length of list, size of string,…] Worst case = max runtime over all possible inputs of a given size Best case = min runtime over all possible inputs of a given size Average = avg. runtime over specified distribution of inputs
12
Analysis of Running Time We can only know the cost up to constants through analysis of code [number of instructions depends on compiler, flags, architecture, etc.] Assume basic statements are O(1) Sum over loops Cost of function call depends on arguments Recursive functions lead to recurrence relations
13
Loops and Sums for (i=0;i<n;i++) for (j=i;j<n;j++) S; // assume cost of S is O(1)
14
Merge Sort and Insertion Sort Insertion Sort T I (n) = T I (n-1) + O(n) = (n 2 ) [worst case] T I (n) = T I (n-1) + O(1) = (n) [best case] Merge Sort T M (n) = 2T M (n/2) + O(n) = (nlogn) [worst case] T M (n) = 2T M (n/2) + O(n) = (nlogn) [best case]
15
Karatsuba’s Algorithm Using the classical pen and paper algorithm two n digit integers can be multiplied in O(n 2 ) operations. Karatsuba came up with a faster algorithm. Let A and B be two integers with A = A 1 10 k + A 0, A 0 < 10 k B = B 1 10 k + B 0, B 0 < 10 k C = A*B = (A 1 10 k + A 0 )(B 1 10 k + B 0 ) = A 1 B 1 10 2k + (A 1 B 0 + A 0 B 1 )10 k + A 0 B 0 Instead this can be computed with 3 multiplications T 0 = A 0 B 0 T 1 = (A 1 + A 0 )(B 1 + B 0 ) T 2 = A 1 B 1 C = T 2 10 2k + (T 1 - T 0 - T 2 )10 k + T 0
16
Complexity of Karatsuba’s Algorithm Let T(n) be the time to compute the product of two n-digit numbers using Karatsuba’s algorithm. Assume n = 2 k. T(n) = (n lg(3) ), lg(3) 1.58 T(n) 3T(n/2) + cn 3(3T(n/4) + c(n/2)) + cn = 3 2 T(n/2 2 ) + cn(3/2 + 1) 3 2 (3T(n/2 3 ) + c(n/4)) + cn(3/2 + 1) = 3 3 T(n/2 3 ) + cn(3 2 /2 2 + 3/2 + 1) … 3 i T(n/2 i ) + cn(3 i-1 /2 i-1 + … + 3/2 + 1)... cn[((3/2) k - 1)/(3/2 -1)] --- Assuming T(1) c 2c(3 k - 2 k ) 2c3 lg(n) = 2cn lg(3)
17
Divide & Conquer Recurrence Assume T(n) = aT(n/b) + (n) T(n) = (n) [a < b] T(n) = (nlog(n)) [a = b] T(n) = (n log b (a) ) [a > b]
18
(blank for notes)
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.