The Efficiency of Algorithms Chapter 4 Adapted from Pearson Education, Inc.

Contents Motivation Measuring an Algorithm’s Efficiency Counting Basic Operations Best, Worst, and Average Cases Big Oh Notation The Complexities of Program Constructs Picturing Efficiency The Efficiency of Implementations of the ADT Bag An Array-Based Implementation A Linked Implementation Comparing the Implementations Adapted from Pearson Education, Inc.

Objectives Assess efficiency of given algorithm Compare expected execution times of two methods Given efficiencies of algorithms Adapted from Pearson Education, Inc.

Motivation - Even a simple program can be inefficient Three algorithms for computing ∑ i Trace each with n=5 and time yourself Trace each with n=10 and time yourself What do you notice after doubling n? When doubling n The time remained fixed When doubling n The time was doubled When doubling n The time was quadrupled Adapted from Pearson Education, Inc.

Measuring an Algorithm’s Efficiency Complexity Space requirements Time requirements Other issues for best solution Generality of algorithm Programming effort Problem size – number of items program will handle Growth-rate function Adapted from Pearson Education, Inc.

Counting Basic Operations Linearly related to n Quadratically related to n Independent of n Adapted from Pearson Education, Inc.

Counting Basic Operations Linearly related to n Quadratically related to n Independent of n Adapted from Pearson Education, Inc.

(a) Typical growth-rate functions – (b) Effect of doubling the size of n – (c) An example time to process 106 items @ 1MIPS (b) (c) Adapted from Pearson Education, Inc.

Best, Worst, and Average Cases Some algorithms depend only on size of data set Other algorithms depend on nature of the data Best case search when item at beginning Worst case when item at end Average case somewhere between Adapted from Pearson Education, Inc.

Big Oh Notation A function f(n) is of order at most g(n) that is, We can say f(n) = (n2+n)/2 is of order g(n) = n2 f(n) has a Big Oh of n2  O(n2) This means, with c=1 and N=1 (n2+n)/2 ≤ 1∙ n2 for all n ≥ 1 A function f(n) is of order at most g(n) that is, f(n) is O(g(n)) — if : A positive real number c and positive integer N exist such that f(n) ≤ c ∙ g(n) for all n ≥ N. That is, c ∙ g(n) is an upper bound on f(n) when n is sufficiently large. Adapted from Pearson Education, Inc.

Big Oh Identities O(k g(n)) = O(g(n)) for a constant k O(g1(n)) + O(g2(n)) = O(g1(n) + g2(n)) O(g1(n)) x O(g2(n)) = O(g1(n) x g2(n)) O ( g1(n) + g2(n) + . . . + gm(n) ) = O ( max[ g1(n), g2(n), . . ., gm(n) ] ) = max [ O(g1(n)), O(g2(n)), . . . , O(gm(n)) ] Adapted from Pearson Education, Inc.

Big Oh Identities O(k g(n)) = O(g(n)) for a constant k O(g1(n)) + O(g2(n)) = O(g1(n) + g2(n)) O(g1(n)) x O(g2(n)) = O( g1(n) x g2(n) ) O ( g1(n) + g2(n) + . . . + gm(n) ) = O (max[ g1(n), g2(n), . . ., gm(n) ] ) = max[O(g1(n)), O(g2(n)), . . . , O(gm(n)) ] O(3 g(n)) = O(g(n)) O(n) + O(3n2) = O(n + 3n2) O(n) x O(3n2) = O(n x 3n2) O (n + 3n2 + 5logn+n ) = O ( max[ n, 3n2, 5logn+n ] ) = max [ O(n), O(3n2), O(5logn+n) ] = O(n+n2) = O( max(n, n2)) = O(n2) = O(3n3) = O(n3) = O(2n+3n3+5logn) = max(O(n),O(n3),O(logn)) = O(n3) Adapted from Pearson Education, Inc.

Figure 4-6 An O(n) algorithm Adapted from Pearson Education, Inc.

Figure 4-7 An O(n2) algorithm Adapted from Pearson Education, Inc.

Figure 4-8 Another O(n2) algorithm Adapted from Pearson Education, Inc.

Complexities of program constructs Time Complexity Consecutive segments If that choses between if- part and else-part A loop that iterates m times Add growth-rates  max O(cond) + max(O(if-part),O(else-part)) m * (O(cond) + O(body)) Adapted from Pearson Education, Inc.

Array Based Implementation (fixed size) Adding an entry to a bag, an O(1) method, Adapted from Pearson Education, Inc.

Array Based Implementation (fixed size) Adding an entry to a bag, an O(1) method, O(1) O(1) +O(1) = O(1) O(1) Max(O(1),O(1)) = O(1) O(1) O(1) O(1) O(1) +O(1) +O(1) = O(1) Adapted from Pearson Education, Inc.

Array Based Implementation Searching for an entry, O(1) best case, O(n) worst or average case  O(n) method overall Adapted from Pearson Education, Inc.

Array Based Implementation Searching for an entry, O(1) best case, O(n) worst or average case  O(n) method overall The size of the bag, is the size of the problem  numberOfEntries is our n This loop executes numberOfEntries times, i.e. n times O(1) n * (O(1) + O(1)) = n * O(1) = O(n) O(1) O(1) +O(1) =O(1) O(1) O(1) O(1) O(1) +O(n) +O(1) = O(n) Adapted from Pearson Education, Inc.

A Linked Implementation Adding an entry to a bag, an O(1) method, O(1) +O(1) = O(1) Adapted from Pearson Education, Inc.

A Linked Implementation Searching a bag for a given entry, O(1) best case, O(n) worst case  O(n) overall Adapted from Pearson Education, Inc.

A Linked Implementation Searching a bag for a given entry, O(1) best case, O(n) worst case  O(n) overall The size of the bag, is the size of the problem This loop executes n times O(1) n * (O(1)+O(1)) = n * O(1) = O(n) O(1) O(1) O(1) O(1) +O(n) +O(1) = O(n) Adapted from Pearson Education, Inc.

Time efficiencies of the ADT bag operations Adapted from Pearson Education, Inc.

End Chapter 4 Adapted from Pearson Education, Inc.