1. Analysis of Recursive Algorithms October 29, 2014

2. Recursive Algorithms
Analysis becomes a bit more complicated for recursive algorithms.
Recurrence relations is helpful for such analyses.
A recursive step can be expressed as a recurrence relation, showing how earlier terms in a sequence are related to later terms.

3. Recurrence Relation
A recurrence relation for a sequence a1, a2, a3, … is a formula that relates each term ak to its certain predecessors ak-1, ak-2, …, ak-i where i is a fixed integer and k is any integer greater than i. The initial conditions for such a recurrence relation specify the values of a1, a2, …, ai-1.

4. Example For the Tower of Hanoi problem: Fibonacci Sequence: F1 = 1
Fn = 2Fn-1 + 1
(It can be shown with a complex proof that 2n-1 is the optimal number of steps.)
Fibonacci Sequence:
F0 = 0
Fn = Fn-1 + Fn-2

5. Class Discussion
Can you set up some recurrence relations to compute f(n) where f(n) is the number of 0-1 bit strings with n bits and with no consecutive 1s, i.e., no “11” patterns?
Can you set up a recurrence relation to count the number of ways to parenthesize x1x2…xn? E.g., when n = 3, there are two ways, ((x1x2)x3) and (x1(x2x3)).
f(n) = f0(n)+f1(n); f0(n) = f(n-1); f1(n) = f0(n-1); f0(1) = f1(1) = 1

6. Repeated Substitution
Solving recurrence relation is the key to analyze recursive algorithms.
One direct method of solving recurrence relations is called repeated substitution.
This is done by repeatedly substitute the formulae for the earlier terms in the recurrence relation.

7. Repeated Substitution
For example:
a0 = 1
ak = ak-1 + 2
We start by:
= ak
= ak ……
= a0 + 2k = 1 + 2k

8. Induction We can also start with a guess of the closed form solution.
Then, prove its correctness with induction.

9. Induction For example: Suppose we want to show that ak=1+2k: a0 = 1
ak = ak-1 + 2
Suppose we want to show that ak=1+2k:
Base case: k = 0, a0 = 1 = 1 + 20
Inductive case: Assume that ak = 1+2k for k  0
ak+1 = ak + 2
= 1 + 2k + 2
= 1 + 2(k+1)
Therefore, we can conclude that ak=1+2k
(Structural Induction)

10. Class Discussion
Give a closed form solution to the following recurrence relation:
a0 = 5
an = nan-1

11. Recursive Algorithms
Solving recurrence relations is the key to analyze recursive algorithms.
A recurrence relation can be used to express the running time of a recursive algorithm, showing how many recursive calls are made and how much work is done at each level.
We can then solve the recurrence relations to determine an upper bound to the worst case running time.

12. Example Consider the following simple example:
1. factorial(n)
2. if (n  1)
3. return(1)
4. else
5. return(n  factorial(n-1)
Let T(n) be the running time of factorial(n):
T(1) = a
T(n) = T(n-1) + b

13. Example Lets try to solve T(n): T(n) = T(n-1) + b = T(n-2) + 2b……
= T(n-(n-1)) + (n-1)b
= a + (n-1)b
= O(n)

14. More Example Consider the following: 1. SelectionSort(A[ ], i, n)
2. if (i < n)
3. small = i
4. for (j = i + 1, j  n, j++)
5. if (A[j] < A[small])
6. small = j
7. t = A[small]
8. A[small] = A[i]
9. A[i] = t
10. SelectionSort(A, i + 1, n)

15. More Example
Let T(m) denote the running time of SelectSort(A,n-m+1,n).
T(1) = 0
T(m) = T(m-1) + O(m)  T(m-1) + Cm
 T(m-2) + C(m-1) + Cm ……
 T(1) + C2 + C3 + … + Cm
 C(m+2)(m-1)/2 = O(m2)
Therefore, the run time of SelectSort(A,1,n) is O(n2)

16. Class Discussion Consider MergeSort(): 1. MergeSort(A[ ])
2. If (length of A[ ] > 1)
3. Split A[ ] into two halves, A1[ ] and A2[ ]
4. MergeSort(A1)
5. MergeSort(A2)
6. Merge A1 and A2 into one sorted list A
What is the running time of MergeSort on an array with n elements?
T(n)  2T(n/2) + Cn  2(2T(n/4) + C(n/2)) + Cn …  2lgnT(1) + lgn  Cn = O(nlgn)

17. Master Theorem
Let f be an increasing function that satisfies the recurrence relation:
f(n) = af(n/b) + cnd
(assume n = bk), where k is a positive integer, a1, b is an integer greater than 1, c is a positive real number and d is a non-negative real number, then:

18. Class Discussion
Use the Master Theorem to find the Big-O of MergeSort()

19. f(n) = af(n-1) + bf(n-2) for n  2
More Theorems
Let f be a function that satisfies the recurrence relation:
f(n) = af(n-1) + bf(n-2) for n  2
and f(0) and f(1) have given values. Let r1 and r2 be the roots of the equation:
x2 = ax + b
Then, if r1  r2, f(n) = c1r1n + c2r2n where n  0 for some constants c1 and c2.
If r1 = r2 = r, f(n) = (c1 + c2n)rn where n  0 for some constants c1 and c2.

20. Class Discussion Give a Big-O of the following recurse():
1. recurse(n)
2. if (n == 0)
3. return 1
4. for (i = 0; i < 4; i++)
5. total += recurse(n-2)
6. return total