Chapter 2: Fundamentals of the Analysis of Algorithm Efficiency The Design and Analysis of Algorithms Chapter 2: Fundamentals of the Analysis of Algorithm Efficiency Mathematical Analysis of Non-recursive and Recursive Algorithms

Section 2.3. Mathematical Analysis of Non-recursive Algorithms Steps in mathematical analysis of non-recursive algorithms Decide on parameter n indicating input size Identify algorithm’s basic operation Determine worst, average, and best case for input of size n Set up summation for C(n) reflecting algorithm’s loop structure Simplify summation using standard formulas (see Appendix A)

Example: Selection sort 1 Input: An array A[0..n-1] Output: Array A[0..n-1] sorted in ascending order for i  0 to n-2 do min  i for j = i + 1 to n – 1 do if A[j] < A[min] min  j swap A[i] and A[min]

Example: Selection sort 2 Basic operation: comparison Inner loop: n-1 S(i) =  1 = (n-1) – (i + 1) + 1 = n – 1 – i j = i+1 Outer loop: n-2 n-2 n-2 n-2 C(n) =  S(i) =  (n – 1 – i) =  (n – 1) –  i i = 0 i = 0 i = 0 i = 0 n Basic formula:  i = n(n+1) / 2 i = 0 C(n) = (n – 1 )(n -1 ) – (n-2)(n-1)/2 = (n – 1) [2(n – 1) – (n – 2)] / 2 = = (n – 1) n / 2 = O(n2)

Section 2.4. Mathematical Analysis of Recursive Algorithms Steps in mathematical analysis of non-recursive algorithms Decide on parameter n indicating input size Identify algorithm’s basic operation Determine worst, average, and best case for input of size n Set up a recurrence relation and initial condition(s) Solve the recurrence to obtain a closed form or estimate the order of magnitude of the solution (see Appendix B)

Important Recurrence Types One (constant) operation reduces problem size by one. T(n) = T(n-1) + c T(1) = d Solution: T(n) = (n-1)c + d linear A pass through input reduces problem size by one. T(n) = T(n-1) + cn T(1) = d Solution: T(n) = [n(n+1)/2 – 1] c + d quadratic One (constant) operation reduces problem size by half. T(n) = T(n/2) + c T(1) = d Solution: T(n) = c log n + d logarithmic A pass through input reduces problem size by half. T(n) = 2T(n/2) + cn T(1) = d Solution: T(n) = cn log n + d n n log n

Example 1: Factorial n! = n*(n-1)! 0! = 1 Recurrence relation: T(n) = T(n-1) + 1 T(1) = 1 Telescoping: T(n) = T(n-1) + 1 T(n-1) = T(n-2) + 1 T(n-2) = T(n-3) + 1 … T(2) = T(1 ) + 1 Add the equations and cross equal terms on opposite sides: T(n) = T(1) + (n-1) = = n

Example 2: Binary Search Recurrence Relation T(n) = T(n/2) + 1, T(1) = 1 Telescoping T(n/2) = T(n/4) + 1 … T(2) = T(1) + 1 Add the equations and cross equal terms on opposite sides: T(n) = T(1) + log(n) = O(log(n))

Master Theorem: A general divide-and-conquer recurrence T(n) = aT(n/b) + f (n) where f (n) ∈ Θ(nk) a < bk T(n) ∈ Θ(nk) a = bk T(n) ∈ Θ(nk log n ) a > bk T(n) ∈ Θ(n to the power of (logba)) Note: the same results hold with O instead of Θ.