Chapter 2 Fundamentals of the Analysis of Algorithm Efficiency Copyright © 2007 Pearson Addison-Wesley. All rights reserved.

Chapter 2 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Topic Objectives: is devoted to analysis of algorithms. learn about time efficiency. learn about big Oh notation. learn how to calculate time efficiency for nonrecursive algorithm

2-2 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 1: Maximum element

2-3 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 1: Maximum element

2-4 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 1: Maximum element

2-5 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 1: Maximum element = (n-1)-1+1

2-6 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 1: Maximum element = (n-1)-1+1=n-1=O(n)

2-7 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 2 (i):

2-8 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 2 (i):

2-9 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 2 (i):

2-10 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 2 (i):

2-11 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 2(ii):

2-12 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 2(ii):

2-13 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 2(ii):

2-14 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 2(iii)

2-15 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 2(iii) Summation formulas  Page:502

2-16 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 2(iii) (n-1)(n-2-0+1) = (n-1)(n-1)

2-17 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 2(iii)

2-18 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 3: Matrix multiplication

2-19 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 3: Matrix multiplication

2-20 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Recursive Algorithms

2-21 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Plan for Analysis of Recursive Algorithms b Decide on a parameter indicating an input’s size. b Identify the algorithm’s basic operation. b Check whether the number of times the basic op. is executed may vary on different inputs of the same size. (If it may, the worst, average, and best cases must be investigated separately.) b Set up a recurrence relation with an appropriate initial condition expressing the number of times the basic op. is executed. b Solve the recurrence (or, at the very least, establish its solution’s order of growth) by backward substitutions or another method.

2-22 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Recurrence Relation:

2-23 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2

2-24 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2

2-25 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2

2-26 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2

2-27 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2

2-28 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 1: Recursive evaluation of n! Definition: n ! = 1  2  …  (n-1)  n for n ≥ 1 and 0! = 1 Recursive definition of n!: F(n) = F(n-1)  n for n ≥ 1 and F(0) = 1 F(0) = 1Size: Basic operation: Recurrence relation:

2-29 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Solving the recurrence for M(n) M(n) = M(n-1) + 1, for n > 0 To compute F(n-1) To multiply F(n-1) by n M(0) = 0 If n=0 ; return 1  no multiplication

2-30 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Solving the recurrence for M(n) M(n) = M(n-1) + 1, for n > 0 for n -1 =M(n-1) = [M(n-1-1) + 1] +1 = [M(n-2) + 2 for n -2 =M(n-2) = [M(n-2-1) + 1] +2 = M(n-3) + 3

2-31 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Solving the recurrence for M(n) M(n) = M(n-1) + 1, for n > 0 for n -3 =M(n-3) = [M(n-3-1) + 1] +3 = [M(n-4) + 4 :::: =M(n) = [M(n-1) + 1= M(n-i)+i = ……M(n-n)+n=n K.M= O(n) K.M= O(n)

2-32 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 2: The Tower of Hanoi Puzzle Recurrence for number of moves:

2-33 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Solving recurrence for number of moves M(1) = 1 M(n) = M(n-1) M(n-1) for n>1 for n -1 M(n-1) = 2[2M(n-1-1) + 1]+1 = 2 2 M(n-2)+2+1 for n -2 M(n-2) = 2 2 [2M(n-2-1)+1]+3 = 2 3 M(n-3)

2-34 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Solving recurrence for number of moves M(1) = 1 M(n) = M(n-1) M(n-1) for n>1 for n -3 M(n-3) = 2 3 [2M(n-3-1)+1)] = 2 4 M(n-4) :::: M(n) = 2 i [2M(n-i)+2 i i-2 + … = 2 i M(n-i)+2 i -1 = = 2 i M(n-i)+2 i -1 =

2-35 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Solving recurrence for number of moves M(n) = 2 i [2M(n-i)+2 i i-2 + … = 2 i M(n-i)+2 i -1 = 2 i M(n-i)+2 i -1 For i=n-1; M(n) = 2 n-1 M(n-(n-1)+2 n-1 -1 = 2 n-1 M(1) +2 n-1 -1 = 2 n-1 M(1) +2 n-1 -1 = 2.2 n-1 -1 = 2.2 n-1 -1 = 2 n = 2 n = 2 n -1 ; K.M= O(2 n ) = 2 n -1 ; K.M= O(2 n )

2-36 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Tree of calls for the Tower of Hanoi Puzzle

2-37 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch Recursive Solution b b moves(n) = 0 when n = 0 b b moves(n) = 2*moves(n-1) + 1 = 2 n -1 when n > 0 A BC 1

2-38 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 3: Counting #bits A(n) = A([n/2]) +1 for n > 1 A(1) = 0 Recurrence and initial condition :

2-39 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Example 3: Counting #bits A(n) = A([n/2]) +1 for n > 1 A(1) = 0 Recurrence and initial condition : End up with  A( 2 k ) = A(1) + k n= 2 k  A(n) = log n = O(log n)

2-40 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2

2-41 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2

2-42 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Fibonacci numbers The Fibonacci numbers: 0, 1, 1, 2, 3, 5, 8, 13, 21, … The Fibonacci recurrence: F(n) = F(n-1) + F(n-2) F(0) = 0 F(1) = 1 General 2 nd order linear homogeneous recurrence with constant coefficients: aX(n) + bX(n-1) + cX(n-2) = 0 aX(n) + bX(n-1) + cX(n-2) = 0

2-43 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Solving aX(n) + bX(n-1) + cX(n-2) = 0 b Set up the characteristic equation (quadratic) ar 2 + br + c = 0 ar 2 + br + c = 0 b Solve to obtain roots r 1 and r 2 b General solution to the recurrence if r 1 and r 2 are two distinct real roots: X(n) = α r 1 n + β r 2 n if r 1 = r 2 = r are two equal real roots: X(n) = α r n + β nr n b Particular solution can be found by using initial conditions

2-44 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Application to the Fibonacci numbers F(n) = F(n-1) + F(n-2) or F(n) - F(n-1) - F(n-2) = 0 Characteristic equation: Characteristic equation: Roots of the characteristic equation: General solution to the recurrence: Particular solution for F(0) =0, F(1)=1:

2-45 Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 2 Computing Fibonacci numbers 1. Definition-based recursive algorithm 2. Nonrecursive definition-based algorithm 3. Explicit formula algorithm 4. Logarithmic algorithm based on formula: F(n-1) F(n) F(n) F(n+1) =n for n≥1, assuming an efficient way of computing matrix powers.