1. Fuw-Yi Yang1 演算法概論 Introduction to Algorithms Department of Computer Science and Information Engineering, Chaoyang University of Technology 朝陽科技大學資工系 Speaker: Fuw-Yi Yang 楊伏夷 伏夷非征番, 道德經 察政章 (Chapter 58) 伏 者潛藏也 道紀章 (Chapter 14) 道無形象, 視之不可見者曰 夷

2. Fuw-Yi Yang2 Reference: K. H. Rosen, Discrete Mathematics and Its Applications, 5th ed. J. A. Dossey, A. D. Otto, L. E. Spence and C. V. Eynden, Discrete Mathematics, 4th ed. R.1 Introduction R.2 Recurrence Relations R.3 Modeling with Recurrence Relations R.4 Solving Recurrence Relations R.5 First-Order Linear Difference Equation R.6 Divide-and-Conquer Algorithms and Recurrence Relations R.7 Prune-and-Search

3. Fuw-Yi Yang3 R.1 Introduction The number of a bacteria in a colony doubles every hour. If a colony begins with five bacteria, how many will be present in n hours? To solve this problem, let a n be the number of bacteria at the end of n hours. Since the number of bacteria doubles every hours, the relationship a n = 2a n-1 holds whenever n is a positive integer. This relationship, together with the initial condition a 0 = 5, uniquely determines a n for all nonnegative integers n. Solution: 5, 10, 20, 40, …

4. Fuw-Yi Yang4 R.2 Recurrence Relations Definition R.2.1. A recurrence relation for the sequence {a n } is an equation that expresses a n in terms of one or more of the previous terms of the sequence, namely, a 0, a 1, …, a n-1, for all integers n with n ≧ n 0, where n 0 is a nonnegative integer. A sequence is called a solution of a recurrence relation if its terms satisfy the recurrence relation.

5. Fuw-Yi Yang5 R.2 Recurrence Relations Connection between recursion and recurrence relations Recursive algorithms provide the solution of a problem of size n in terms of the solutions of one or more instances of the same problem of a smaller size. Consequently, when we analyze the complexity of a recursive algorithm, we obtain a recurrence relation that expresses the operations required to solve a problem of size n in terms of the number of operations required to solve the problem for one or more instances of smaller size.

6. Fuw-Yi Yang6 R.2 Recurrence Relations Example R.2.1 Let {a n } be a sequence that satisfies the recurrence relation a n = a n-1 – a n-2 for n = 2, 3, 4,…, and suppose that a 0 = 3 and a 1 = 5. What are a 2 and a 3 ? Solution: a 2 = a 1 – a 0 = 5 – 3 = 2, a 3 = a 2 – a 1 = 2 – 5 = -3

7. Fuw-Yi Yang7 R.2 Recurrence Relations Example R.2.2 Determine whether the sequence {a n } is a solution of the recurrence relation a n = 2a n-1 – a n-2 for n = 2, 3, 4,…, where a n = 3n for every nonnegative integer n. Answer the same question where a n = 2 n and where a n = 5. Solution: suppose that a n = 3n for every nonnegative integer n. Then for n ≧ 2, we have a n = 2a n-1 – a n-2 = 2(3(n - 1)) - 3(n - 2) = 3n. Therefore, the sequence {a n } is a solution of the recurrence relation.

8. Fuw-Yi Yang8 R.2 Recurrence Relations Example R.2.3 Determine whether the sequence {a n } is a solution of the recurrence relation a n = 2a n-1 – a n-2 for n = 2, 3, 4,…, where a n = 3n for every nonnegative integer n. Answer the same question where a n = 2 n and where a n = 5. Solution: suppose that a n =2 n for every nonnegative integer n. Then for n > 2, we have a n = 2*2 n-1 – 2 n-2 = 2 n – 2 n-2 ≠ 2 n. Therefore, the sequence {a n } is not a solution of the recurrence relation.

9. Fuw-Yi Yang9 R.2 Recurrence Relations Example R.2.4 Determine whether the sequence {a n } is a solution of the recurrence relation a n = 2a n-1 – a n-2 for n = 2, 3, 4,…, where a n = 3n for every nonnegative integer n. Answer the same question where a n = 2 n and where a n = 5. Solution: suppose that a n = 5 for every nonnegative integer n. Then for n ≧ 2, we have a n = 2*5 – 5 = 5. Therefore, the sequence {a n } is a solution of the recurrence relation.

10. Fuw-Yi Yang10 R.3 Modeling with Recurrence Relations We can use recurrence relations to model a wide variety of problems, such as finding compound interest, counting rabbits on an island, determining the number of moves in the Tower of Hanoi puzzle, and counting bit strings with certain properties.

11. Fuw-Yi Yang11 R.3 Modeling with Recurrence Relations Example R.3.1 Compound Interest. Suppose that a person deposit $10000 in a saving account at a bank yielding 11% per year with interest compounded annually. How much will be in the account after 30 years? Solution: Let p n denote the amount in the account after n years. Then we see that the sequence {p n } satisfies the recurrence relation

12. Fuw-Yi Yang12 R.3 Modeling with Recurrence Relations Solution: Let p n denote the amount in the account after n years. Then we see that the sequence {p n } satisfies the recurrence relation p n = p n-1 + 0.11p n-1 = (1.11)p n-1. The initial condition is p 0 = 10000. p 1 = (1.11) p 0 p 2 = (1.11) p 1 = (1.11) 2 p 0 … p n = (1.11) p n-1 = (1.11) n p 0. Thus p 30 = (1.11) 30 p 30 = 228922.97

13. Fuw-Yi Yang13 R.3 Modeling with Recurrence Relations Example R.3.2 Rabbits and the Fibonacci Numbers. Consider this problem, which was originally posed by Leonardo di Pisa, also known as Fibonacci, in the thirteenth century in his book Liber abaci. A young pair of rabbits (one of each sex) is placed on an island. A pair of rabbits does not breed until they are 2 months old. After they are 2 months old, each pair of rabbits produces another pair each month. Find a recurrence relation for the number of pairs of rabbits on the island after n months, assuming that no rabbits ever die. Solution: Let f n denote the number of pairs of rabbits after n months.

14. Fuw-Yi Yang14 R.3 Modeling with Recurrence Relations Solution: Let f n denote the number of pairs of rabbits after n months. At the end of the first month, the number of pairs of rabbits on the island is f 1 = 1. At the end of the second month, the number of pairs of rabbits on the island is f 2 = 1. N.t.

15. Fuw-Yi Yang15 R.3 Modeling with Recurrence Relations MonthReproducing pairs f n-2 Young pairs f n-1 Total pairs f n 1011 2011 3112 4123 5235 6358 7585 The sequence {f n } satisfies the recurrence relation f n = f n-1 + f n-2

16. Fuw-Yi Yang16 R.3 Modeling with Recurrence Relations Example R.3.3 Counting bit string of length n with no two consecutive 0s. Find a recurrence relation and give initial conditions for the number of bit strings of length n that do not have two consecutive 0s. How many such bit strings are there of length five? Solution: Let a n denote the number of bit strings of length n that do not have two consecutive 0s.

17. Fuw-Yi Yang17 R.3 Modeling with Recurrence Relations Solution: Let a n denote the number of bit strings of length n that do not have two consecutive 0s. To obtain a recurrence relation for {a n }, note that by the sum rule, the number of bit strings of length n that do not have two consecutive 0s equals the number of such bit strings ending with a 0 plus the number of such bit strings ending with a 1.

18. Fuw-Yi Yang18 R.3 Modeling with Recurrence Relations a n : the number of bit strings of length n that do not have two consecutive 0s equals 1 the number of bit strings of length n that do not have two consecutive 0s ending with a 1 plus 2 the number of bit strings of length n that do not have two consecutive 0s ending with a 0. a n-1 : the number of the first term (1) is precisely the number of bit strings of length n-1 that do not have two consecutive 0s, i.e. a n-1. a n-2 : the number of the second term (2) is precisely the number of bit strings of length n-2 that do not have two consecutive 0s, i.e. a n-2. Therefore, the recurrence relation a n = a n-1 + a n-2 satisfies the sequence {a n }.

19. Fuw-Yi Yang19 R.4 Solving Recurrence Relations Definition R.4.1 A linear homogeneous recurrence relation of degree k with constant coefficients is a recurrence relation of the form a n = c 1 a n-1 + c 2 a n-2 + … + c k a n-k, where c 1, c 2, …, c k are real numbers, and c k ≠0.

20. Fuw-Yi Yang20 R.4 Solving Recurrence Relations Linear: since the right-hand side is a sum of multiples of the previous terms of the sequence. Homogeneous: since no terms occur that are not multiples of the a j s. Constants: the coefficients of the terms of the sequence are all constants, rather than functions that depend on n. Degree: the degree is k because an is expressed in terms of the previous k terms of the sequence.

21. Fuw-Yi Yang21 R.4 Solving Recurrence Relations A sequence satisfying the recurrence relation in the definition is uniquely determined by this recurrence relation and the k initial conditions a 0 = C 0, a 1 = C 1, …,a k-1 = C k-1.

22. Fuw-Yi Yang22 R.4 Solving Recurrence Relations Example R.4.1 some linear homogeneous recurrence relations: degree 1: p n = (1.11) p n-1, degree 2: f n = f n-1 + f n-2, degree 5: a n = a n-5, Example 2. some recurrence relations: a n = a n-1 + (a n-2 ) 2, not linear (square), H n = 2H n-1 + 1, not homogeneous (1), B n = nB n-1, not constant coefficient (n).

23. Fuw-Yi Yang23 R.4 Solving Recurrence Relations Linear homogeneous recurrence relations are studied for two reasons. First, they often occur in modeling of problems. Second, they can be systematically solved.

24. Fuw-Yi Yang24 R.4 Solving Recurrence Relations with Constant Coefficients The basic approach for solving linear homogeneous recurrence relations is to look for solutions of the form a n = r n, where r is a constant. Note that a n = r n is a solution of the recurrence relation a n = c 1 a n-1 + c 2 a n-2 + … + c k a n-k, if and only if : r n = c 1 r n-1 + c 2 r n-2 + … + c k r n-k. Both sides dividing by r n-k, we obtain, r k - c 1 r k-1 - c 2 r k-2 - … - c k r k-k = 0.

25. Fuw-Yi Yang25 R.4 Solving Recurrence Relations with Constant Coefficients Note that a n = r n is a solution of the recurrence relation a n = c 1 a n-1 + c 2 a n-2 + … + c k a n-k, if and only if : r n = c 1 r n-1 + c 2 r n-2 + … + c k r n-k. Both sides dividing by r n-k, we obtain, r k - c 1 r k-1 - c 2 r k-2 - … - c k r k-k = 0. Consequently, the sequence {a n } with a n = r n is a solution if and only if r is a solution of this last equation, which is called the characteristic equation of the recurrence relation. The solutions of this equation are called the characteristic roots of the recurrence relation.

26. Fuw-Yi Yang26 R.4 Solving Recurrence Relations with Constant Coefficients Example R.4.2 What is the solution of the recurrence relation: a n = a n-1 + 2a n-2, with a 0 = 2 and a 1 = 7 ? Solution: The characteristic equation of the recurrence relation is r 2 - r 1 - 2 = 0. The characteristic roots of the recurrence relation are r = 2 and r = -1. Which one is the solution ?

27. Fuw-Yi Yang27 R.4 Solving Recurrence Relations with Constant Coefficients Which one is the solution ? With the same method of solving homogeneous differential equations, we guess the solution may be the combination of both of them. The followings show that. Let the characteristic roots be r 1 and r 2 for a characteristic equation r 2 - c 1 r 1 - c 2 = 0. The corresponding recurrence relation is a n = c 1 a n-1 + c 2 a n-2. Also A and B denote constant coefficients.

28. Fuw-Yi Yang28 R.4 Solving Recurrence Relations with Constant Coefficients The corresponding recurrence relation is a n = c 1 a n-1 + c 2 a n-2. Also A and B denote constant coefficients. If the combination of r 1 and r 2 is solution, then a n = A(r 1 ) n + B(r 2 ) n. That is: c 1 a n-1 + c 2 a n-2 = c 1 (A(r 1 ) n-1 + B(r 2 ) n-1 ) + c 2 (A(r 1 ) n-2 + B(r 2 ) n-2 ) = A(r 1 ) n-2 (c 1 r 1 + c 2 ) + B(r 2 ) n-2 (c 1 r 2 + c 2 ) = A(r 1 ) n-2 (r 1 ) 2 + B(r 2 ) n-2 (r 2 ) 2 = a n.

29. Fuw-Yi Yang29 R.4 Solving Recurrence Relations with Constant Coefficients Hence the sequence {a n } is a solution to the recurrence relation iff a n = A(2) n + B(-1) n, From the initial conditions, it follows that a 0 = 2 = A + B, a 1 = 7 = 2 – B. Thus, A = 3 and B = -1. Hence, the solution to the recurrence relation and initial conditions is the sequence {a n } with a n = 3*2 n - (-1) n.

30. Fuw-Yi Yang30 R.4 Solving Recurrence Relations with Constant Coefficients Theorem R.4.1 Let c 1 and c 2 be real numbers. Suppose that r 2 - c 1 r - c 2 = 0 has two distinct roots r 1 and r 2. Then the sequence {a n } is a solution of the recurrence relation a n = c 1 a n-1 + c 2 a n-2 iff a n = A(r 1 ) n + B(r 2 ) n for n = 0, 1, 2, …, where A and B are constant coefficients.

31. Fuw-Yi Yang31 R.4 Solving Recurrence Relations with Constant Coefficients Theorem R.4.2 Let c 1 and c 2 be real numbers. Suppose that r 2 - c 1 r - c 2 = 0 has only one roots r 0. Then the sequence {a n } is a solution of the recurrence relation a n = c 1 a n-1 + c 2 a n-2 iff a n = A(r 0 ) n + Bn(r 0 ) n for n = 0, 1, 2, …, where A and B are constant coefficients.

32. Fuw-Yi Yang32 R.4 Solving Recurrence Relations with Constant Coefficients Theorem R.4.3 Let c 1, c 2, …, c k be real numbers. Suppose that the characteristic equation r k - c 1 r k-1 - …- c k = 0 has k distinct roots r 1, r 2, …, r k. Then the sequence {a n } is a solution of the recurrence relation a n = c 1 a n-1 + c 2 a n-2 + … + c k a n-k iff a n = A 1 (r 1 ) n + A 2 (r 2 ) n + … +A k (r k ) n for n = 0, 1, 2, …, where A 1, A 2, …, A k are constant coefficients.

33. Fuw-Yi Yang33 R.5 First Order Linear Difference Equation with Constant Coefficient The simplest type of recurrence relation give s n as a function of s n-1 for n ≧ 1. We call an equation of the form s n = as n-1 + b, where a and b are constants and a ≠ 0, a first order linear difference equation with constant coefficients. It is not difficult to derive estimates of the size of functions satisfy such recurrence relations. s n = a s n-1 + b = a (a s n-2 + b) + b = a 2 s n-2 + ab + b = a 3 s n-3 + a 2 b + ab + b =… = a n s 0 + a n-1 b + … + ab + b

34. Fuw-Yi Yang34 R.5 First Order Linear Difference Equation with Constant Coefficient s n = a s n-1 + b = a n s 0 + a n-1 b + … + ab + b IF a = 1, then s n = s 0 + nb If a ≠ 1, s n = s n-1 + b = a n s 0 + a n-1 b + … + ab + b = a n s 0 + b(a n - 1)/ (a - 1) = a n (s 0 + b/(a - 1)) - b/(a - 1)

35. Fuw-Yi Yang35 R.5 First Order Linear Difference Equation with Constant Coefficient Example R.5.1 Sequential Search Algorithm This algorithm searches a list of n items a 1, a 2, …,a n for a given target value t. If t = a k for some index k, then the algorithm gives the first such index k. Otherwise the algorithm gives k = 0. Step 1 j = 1 Step 2 while j ≦ n and a j ≠t j = j + 1 Step 3 if j ≦ n then k = j elsek k = 0

36. Fuw-Yi Yang36 R.5 First Order Linear Difference Equation with Constant Coefficient To search a list of n items, we compare the target value to the first item in the list and then, in the worst case, still have to search a list of n-1 items. Thus we see that s n satisfies the recurrence relation s n = s n-1 + 1 for for n ≧ 1. Also, the initial condition s 0 = 0. s n = s n-1 + 1 = (s n-2 + 1) + 1 = s 0 + n = n + 1 Therefore the complexity of this algorithm is O(n).

37. Fuw-Yi Yang37 R.5 First Order Linear Difference Equation with Constant Coefficient Example R.5.2 Find a formula for s n if s n = 3 s n-1 -1 for n ≧ 1. The initial condition is s 0 = 2. Solution: s n = as n-1 + b = a n (s 0 + b/(a - 1)) - b/(a - 1), where a = 3 and b = -1. Therefore, s n = 3 n (2 - 1/2) + 1/2 =1/2(3 n+1 +1). The time complexity of s n is of O(3 n ).

38. Fuw-Yi Yang38 R.5 First Order Linear Difference Equation Example R.5.3 Bubble Sort This algorithm places the numbers in the list a 1, a 2, …, a n in non- decreasing order. Step 1 for j = 1to n-1 // set beginning of sublist Step 2 for k = n-1 to j by -1 // find the smallest element Step 3 if a k+1 < a k interchange the values of a k+1 and a k Step endfor Step 4 Print a 1, a 2, …, a n

39. Fuw-Yi Yang39 R.5 First Order Linear Difference Equation In performing a bubble sort, the first iteration requires comparing a n-1 to a n, then a n-2 to a n-1, and so forth until a 2 is compared to a 1. After each such comparison, we interchange the values of a k and a k+1, if necessary, so that a k ≦ a k+1. Thus when the first iteration is complete, a 1 is the smallest number in the original list. A second iteration is then performed on the smaller list a 2, a 3,…,a n in order to make a 2 the second-smallest number in the original list. After n-1 such iterations, the original list will be non-decreasing order. Thus the recurrence relation is b n = b n-1 + (n-1) with initial condition b 1 = 0. Find its complexity.

40. Fuw-Yi Yang40 R.5 First Order Linear Difference Equation Solution: The recurrence relation is b n = b n-1 + (n-1), for n ≧ 2 initial condition b 1 = 0. b 1 = 0, b 2 = 0 + 1, b 3 = 0 + 1 + 2, we conjecture that b n = 1 + 1 + 2 + … + (n-1) = (n 2 – n)/2 Thus b n is O(n 2 ).

41. Fuw-Yi Yang41 R.6 Divide-and-Conquer Algorithms and Recurrence Relations Recurrence relations of the form f(n) = af(n/b) + g(n) This form of recurrence relation arise in many differential situations. It implies the concepts of divide and conquer. It is possible to derive estimates of the size of functions satisfy such recurrence relations. Suppose that f satisfies this recurrence relation whenever n is divisible by b. Let n = b k, where k is a positive integer. Then f(n) = a f(n/b) + g(n) = a (af((n/b)/b) + g(n/b)) + g(n) = a 2 f(n/b 2 ) + ag(n/b) + g(n) = a 3 f(n/b 3 ) + a 2 g(n/b 2 ) + ag(n/b) + g(n)

42. Fuw-Yi Yang42 R.6 Divide-and-Conquer Algorithms and Recurrence Relations f(n) = a f(n/b) + g(n) = a (af((n/b)/b) + g(n/b)) + g(n) = a 2 f(n/b 2 ) + ag(n/b) + g(n) = a 3 f(n/b 3 ) + a 2 g(n/b 2 ) + ag(n/b) + g(n) =… = a k f(n/b k ) + a k-1 g(n/b k-1 ) + … + ag(n/b) + g(n) = a k f(1) + a k-1 g(n/b k-1 ) + … + ag(n/b) + g(n)

43. Fuw-Yi Yang43 R.6 Divide-and-Conquer Algorithms and Recurrence Relations Recurrence relations of the form f(n) = af(n/b) + g(n) f(n) = a f(n/b) + g(n) = a k f(n/b k ) + a k-1 g(n/b k-1 ) + … + ag(n/b) + g(n) = a k f(1) + a k-1 g(n/b k-1 ) + … + ag(n/b) + g(n) Since n = b k, thus k = log b n. We can use this equation for f(n) to estimate the size of functions that satisfy divide and conquer relation.

44. Fuw-Yi Yang44 R.6 Divide-and-Conquer Algorithms and Recurrence Relations Theorem R.6.1 Let f be an increasing function that satisfies the recurrence relation f(n) = af(n/b) + c. Whenever n is divisible by b, where a ≧ 1, b is an integer greater than 1, and c is a positive real number. Then f(n) is O(n log b a ) if a > 1, O(log n) if a = 1. Furthermore, when n = b k, where k > 0, f(n) = An log b a + B, where A = f(1) + c/(a - 1) and B = -c/(a - 1). Proof:

45. Fuw-Yi Yang45 R.6 Divide-and-Conquer Algorithms and Recurrence Relations f(n) = af(n/b) + c = a k f(n/b k ) + a k-1 c + … + ac + c = a k f(1) + a k-1 c + … + ac + c ∵ n = b k = a k f(1) + (a k – 1)c / (a - 1) If a = 1, then f(n) = f(1) + kc if n is a power of b, n = b k, f(n) = f(1) + c log b n if n is not a power of b, b k < n < b k+1, f(n) = f(1) + c log b n < f(1) + c(k + 1) = (f(1) + c) + ck < (f(1) + c) + c log b n for both cases, f(n) is O(log n).

46. Fuw-Yi Yang46 R.6 Divide-and-Conquer Algorithms and Recurrence Relations If a > 1, and n is a power of b, n = b k, it follows that f(n) = a k f(1) + c(a k – 1)/(a - 1) = a k (f(1) + c/(a – 1)) - c/(a - 1) = A n log b a + B, ∵ a k = a log b n = n log b a where A = f(1) + c/(a – 1), B = - c/(a - 1). If a > 1, and n is not a power of b, by similar processing we obtain the same result. Thus f(n) is O(n log b a ). End of proof

47. Fuw-Yi Yang47 R.6 Divide-and-Conquer Algorithms and Recurrence Relations Example R.6.1 Binary Search. (In the recurrence relation f(n) = af(n/b) + c, a = 1 implies that the Binary search is a method of Prune-and-Search) procedure binary search (x: integer, a 1, a 2, …, a n : increasing integers) i := 1 {i is left endpoint of search interval} j := n {j is right endpoint of search interval} while i < j begin m := integer ((i + j) /2) if x > a m then i := m + 1 else j : = m end if x = a i then location := i else location := 0

48. Fuw-Yi Yang48 R.6 Divide-and-Conquer Algorithms and Recurrence Relations The binary search algorithm reduces the search for an element in a search sequence of size n to the binary search for this element in a search sequence of size n/2, when n is even. Hence, the problem of size n has been reduced to one problem of size n/2. Two comparisons are needed to implement this reduction, one to determine which half of the list to use and the other to determine whether any terms of the list remain. Hence if f(n) is the number of comparisons required to search for an element in a search sequence of size n, then f(n) = f(n/2) + 2, when n is even. Find its time complexity. Solution:

49. Fuw-Yi Yang49 R.6 Divide-and-Conquer Algorithms and Recurrence Relations Solution: The recurrence relation is f(n) = f(n/2) + 2, i.e. a = 1, c = 2. From algorithm, it can be seen f(1) = 1. If n is a power of b, n = b k, it follows that f(n) = f(1) + 2k = 1 + 2 log b n. Thus f(n) is O(log n).

50. Fuw-Yi Yang50 R.6 Divide-and-Conquer Algorithms and Recurrence Relations ---Merge Sort Example R.6.2 Merge Sort Procedure mergesort (L = a 1, a 2, …, a n ) If n > 1 then m := int(n/2) L 1 := a 1, a 2, …, a m L 2 := a m+1, a m+2, …, a n L = merge(mergesort(L 1 ), mergesort(L 2 )) {L is sorted into elements in nondecreasing order}

51. Fuw-Yi Yang51 R.6 Divide-and-Conquer Algorithms and Recurrence Relations --- Merge Sort Procedure merge (L 1, L 2 : lists) L := empty list While L 1 and L 2 are both nonempty remove smaller of first element of L 1 and L 2 from the list it is in and put it at the left end of L if removal of this element makes one list empty then remove all elements form the other list and append them to L Endwhile {L is the merged list with elements in nondecreasing order} Two sorted lists with m elements and n elements can be merged into a sorted list using no more than m+n+1 comparisons.

52. Fuw-Yi Yang52 R.6 Divide-and-Conquer Algorithms and Recurrence Relations – Merge Sort Thus, the recurrence relation for merge sort algorithm is : T(n) = 2T(n/2) + n. Let n = 2 k, T(n) = 2T(n/2) + n = 2 k T(n/2 k ) + 2 k-1 2 + … + 2(n/2) + n = 2 k T(1) + 2 k-1 2 + … + 2(n/2) + n = nT(1) + n log 2 n ∵ n = 2 k Therefore T(n) is O(n log n).

53. Fuw-Yi Yang53 R.6 Divide-and-Conquer Algorithms and Recurrence Relations --- Quick Sort Example R.6.3 Quick Sort Quick sort applies the divide-and-conquer paradigm. Here is the three-step divide-and-conquer process for sorting a typical subarray A[p...r], i.e. Divide, Conquer, and Combine. Divide: partition the array A[p..r] into two subarrays (possibly empty) A[p...q-1] and A[q+1...r] such that for each element of A[p...q-1] is less than or equal to A[q], which is, in turn, less than or equal to each element of A[q+1...r]. Compute the index q as part of this partitioning procedure.

54. Fuw-Yi Yang54 R.6 Divide-and-Conquer Algorithms and Recurrence Relations --- Quick Sort Divide: partition the array A[p..r] into two subarrays (possibly empty) A[p...q-1] and A[q+1...r] such that for each element of A[p...q-1] is less than or equal to A[q], which is, in turn, less than or equal to each element of A[q+1...r]. Compute the index q as part of this partitioning procedure. Conquer: Sort the two subarrays A[p...q-1] and A[q+1...r] by recursive calls to quicksort. Combine: Because the subarrays are already sorted, no work is needed to combine them: the entire array A[p...r] is now sorted.

55. Fuw-Yi Yang55 R.6 Divide-and-Conquer Algorithms and Recurrence Relations --- Quick Sort Procedure Quicksort(A, p, r) If p < r then q = Partition(A, p, r) Quicksort(A, p, q-1) Quicksort(A, q+1, r)

56. Fuw-Yi Yang56 R.6 Divide-and-Conquer Algorithms and Recurrence Relations --- Quick Sort Procedure Partition(A, p, r) x = A[r] i = p - 1 for j = p to r – 1 if A[j]  x i = i + 1 swap(A[i], A[j]) swap(A[i+1], A[r]) return i + 1

57. Fuw-Yi Yang57 R.6 Divide-and-Conquer Algorithms and Recurrence Relations --- Quick Sort

58. Fuw-Yi Yang58 R.6 Divide-and-Conquer Algorithms and Recurrence Relations --- Quick Sort

59. Fuw-Yi Yang59 R.6 Divide-and-Conquer Algorithms and Recurrence Relations --- Quick Sort Worst-case partitioning The worst-case behavior for quick sort occurs when the partitioning routing produces one subproblem with n-1 elements and one with 0 elements. Let us assume that this unbalanced partitioning arises in each recursive call. The partitioning cost  (n) time. Since the recursive call on an array of size 0 just returns, T(0) =  (1), and the recurrence for the running time is: T(n) = T(n-1) + T(0) +  (n) = T(n-1) +  (n)  O(n 2 ).

60. Fuw-Yi Yang60 R.6 Divide-and-Conquer Algorithms and Recurrence Relations --- Quick Sort Best-case partitioning The best-case behavior for quick sort occurs when the partitioning routing produces two subproblems, each of size no more than n/2, since one is  n/2  and one of size  n/2  -1. The partitioning cost  (n) time and the recurrence for the running time is: T(n) = 2T(n/2) +  (n)  O(n log n).

61. Fuw-Yi Yang61 R.6 Divide-and-Conquer Algorithms and Recurrence Relations --- Quick Sort Procedure Randomized-Partition(A, p, r) i = Random(p, r) swap(A[i], A[r]) return Partition(A, p, r) Procedure Randomized-Quicksort(A, p, r) If p < r then q = Randomized-Partition(A, p, r) Randomized-Quicksort(A, p, q-1) Randomized-Quicksort(A, q+1, r)

62. Fuw-Yi Yang62 R.6 Divide-and-Conquer Algorithms and Recurrence Relations --- Quick Sort Average-case partitioning

63. Fuw-Yi Yang63 R.6 Divide-and-Conquer Algorithms and Recurrence Relations --- Quick Sort Average-case partitioning Prove that T avg (n) ≦ kn log n Basis: n = 2 and k = 2(b+c), Note: T avg (0) ≦ b, T avg (1) ≦ b Assumption: T avg (n) ≦ kn log n for 1 ≦ n < m Inductive step: for m we want to prove that T avg (m) ≦ km log m

64. Fuw-Yi Yang64 R.6 Divide-and-Conquer Algorithms and Recurrence Relations --- Quick Sort Average-case partitioning Inductive step: for m we want to prove that T avg (m) ≦ km log m

65. Fuw-Yi Yang65 R.6 Divide-and-Conquer Algorithms and Recurrence Relations --- Quick Sort Average-case partitioning

66. Fuw-Yi Yang66 R.6 Divide-and-Conquer Algorithms and Recurrence Relations --- Quick Sort Average-case partitioning

67. Fuw-Yi Yang67 R.7 Prune-and-Search The Selection Problem A prune-and-Search algorithm to find the kth smallest element Input: A set S of n elements Output: the kth smallest element of S 1. If |S| ≦ 5, solve the problem by any method 2. Divide S into  n/5  subsets. 3. Sort each subset of elements. 4. Recursively find the element x which is the medians of the median of the  n/5  subsets. 5. Partition S into S 1, S 2, and S 3, which contains the elements less than, equal to, and greater than x, respectively. (next page)

68. Fuw-Yi Yang68 R.7 Prune-and-Search The Selection Problem 5. Partition S into S 1, S 2, and S 3, which contains the elements less than, equal to, and greater than x, respectively. 6. If |S 1 | ≧ k, then discard S 2 and S 3 and solve the problem that selects the kth smallest element from S 1 during the next iteration; else if |S 1 | + |S 2 | ≧ k, then x is the kth smallest element of S; otherwise, let k = k - |S 1 | - |S 2 |. Solve the problem that selects the kth smallest element from S 3 during the next iteration.

69. Fuw-Yi Yang69 R.7 Prune-and-Search The Selection Problem — worst case It follows that the number of elements greater than x is at least 3(  1/2  n/5   - 2) ≧ 3n/10 – 6. Similarly, at least 3n/10 – 6 elements are less than x. Thus, in the worst case, step 6 calls algorithm recursively on at most 7n/10 + 6 elements. sss sss ss x bbb bbbb bbbb

70. Fuw-Yi Yang70 R.7 Prune-and-Search The Selection Problem — worst case 11, 10, 23, 40, 32, 25, 76, 45, 88, 28, 13, 32, 53, 29, 31, 19, 98, 54, 16, 42, 67, 73, 55, 62, 34, 88, 63, 17, 25, 74, 35, 67, 99, 48, 78, 93, 92, 97 1013162534173592 1129192855254893 2331 42 (x) 4562636797 32 5476677478 40539888738899 2363 3167 42 (x)97 45 62

71. Fuw-Yi Yang71 R.7 Prune-and-Search The Selection Problem We make the assumption that any input of fewer than 140 elements requires O(1) time. We can therefore obtain the recurrence T(n) ≦ O(1) if n < 140, T(n) ≦ T(  n/5  ) + T(7n/10 + 6 ) + O(n) if n > 140. T(n) ≦ c(  n/5  ) + c(7n/10 + 6 ) + an ≦ cn/5 + c + 7cn/10 + 6c + an = 9cn/10 + 7c + an = cn + (-cn/10 + 7c + an) Which is at most cn if (-cn/10 + 7c + an) ≦ 0. To satisfy the inequality above, n > 140, c ≧ 20a.

72. Fuw-Yi Yang72 R.7 Prune-and-Search Example R.7.1 Binary Search. procedure binary search (x: integer, a 1, a 2, …, a n : increasing integers) i := 1 {i is left endpoint of search interval} j := n {j is right endpoint of search interval} while i < j begin m := integer ((i + j) /2) if x > a m then i := m + 1 else j : = m end if x = a i then location := i else location := 0

73. Fuw-Yi Yang73 R.7 Prune-and-Search The binary search algorithm reduces the search for an element in a search sequence of size n to the binary search for this element in a search sequence of size n/2, when n is even. Hence, the problem of size n has been reduced to one problem of size n/2. Two comparisons are needed to implement this reduction, one to determine which half of the list to use and the other to determine whether any terms of the list remain. Hence if f(n) is the number of comparisons required to search for an element in a search sequence of size n, then f(n) = f(n/2) + 2, when n is even. Find its time complexity. Solution:

74. Fuw-Yi Yang74 R.7 Prune-and-Search Solution: The recurrence relation is f(n) = f(n/2) + 2, i.e. a = 1, c = 2. From algorithm, it can be seen f(1) = 1. If n is a power of b, n = b k, it follows that f(n) = f(1) + 2k = 1 + 2 log b n. Thus f(n) is O(log n).