2. Recursive Algorithms

3. Introduction Applications to Numeric Computation

4. COSC 3101, PROF. J. ELDER 3 Complex Numbers Remember how to multiply 2 complex numbers? (a+bi)(c+di) = [ac –bd] + [ad + bc] i Input: a,b,c,d Output: ac-bd, ad+bc If a real multiplication costs $1 and an addition cost a penny, what is the cheapest way to obtain the output from the input? Can you do better than $4.02?

5. COSC 3101, PROF. J. ELDER 4 Johann Carl Friedrich Gauss (* 30. April 1777 in Braunschweig, † 23. Februar 1855 in Göttingen) Input: a,b,c,d Output: ac-bd, ad+bc m 1 = ac m 2 = bd A 1 = m 1 – m 2 = ac-bd m 3 = (a+b)(c+d) = ac + ad + bc + bd A 2 = m 3 – m 1 – m 2 = ad+bc Gauss’ Method: Total Cost? $3.05!

6. COSC 3101, PROF. J. ELDER 5 Question The Gauss method saves one multiplication out of four. It requires 25% less work. Could there be a context where performing 3 multiplications for every 4 provides a more dramatic savings? Let’s back up a bit.

7. COSC 3101, PROF. J. ELDER 6 How to add 2 n-bit numbers. ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** +

8. COSC 3101, PROF. J. ELDER 7 How to add 2 n-bit numbers. *** *** ** ** ** ** ** ** ** ** ** ** *** *** ** ** ** ** ** ** ** ** +

9. COSC 3101, PROF. J. ELDER 8 How to add 2 n-bit numbers. *** *** ** ** ** ** ** ** ** ** *** *** **** **** ** ** ** ** ** ** ** ** +

10. COSC 3101, PROF. J. ELDER 9 How to add 2 n-bit numbers. *** *** ** ** ** ** ** ** *** *** **** **** **** **** ** ** ** ** ** ** ** ** +

11. COSC 3101, PROF. J. ELDER 10 How to add 2 n-bit numbers. *** *** ** ** ** ** *** *** **** **** **** **** **** **** ** ** ** ** ** ** ** ** +

12. COSC 3101, PROF. J. ELDER 11 How to add 2 n-bit numbers. *** *** **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** * * * * + * ** *

13. COSC 3101, PROF. J. ELDER 12 Time complexity of grade school addition *** *** **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** **** + * ** * **** **** On any reasonable computer adding 3 bits can be done in constant time.

14. COSC 3101, PROF. J. ELDER 13 Is there a faster way to add? QUESTION: Is there an algorithm to add two n-bit numbers whose time grows sub-linearly in n?

15. COSC 3101, PROF. J. ELDER 14 Any algorithm for addition must read all of the input bits –Suppose there is a mystery algorithm that does not examine each bit –Give the algorithm a pair of numbers. There must be some unexamined bit position i in one of the numbers –If the algorithm returns the wrong answer, we have found a bug –If the algorithm is correct, flip the bit at position i and give the algorithm this new input. –The algorithm must return the same answer, which now is wrong.

16. COSC 3101, PROF. J. ELDER 15 So any algorithm for addition must use time at least linear in the size of the numbers. Grade school addition is essentially as good as it can be.

17. COSC 3101, PROF. J. ELDER 16 How to multiply 2 n-bit numbers. X * * * * * * * * * * * * * * * * * * * * * * * * * * * * n2n2

18. COSC 3101, PROF. J. ELDER 17 I get it! The total time is bounded by cn 2.

19. COSC 3101, PROF. J. ELDER 18 How to multiply 2 n-bit numbers: Kindergarten Algorithm a × b = a + a + a +... + a b T(n) = θ(bn) Fast?

20. COSC 3101, PROF. J. ELDER 19 Grade School Addition: Linear time Grade School Multiplication: Quadratic time Kindergarten Multiplication: Exponential time # of bits in numbers time

21. End of Lecture 6 March 23, 2009

22. COSC 3101, PROF. J. ELDER 21 Neat! We have demonstrated that multiplication is a harder problem than addition. Mathematical confirmation of our common sense.

23. COSC 3101, PROF. J. ELDER 22 Don’t jump to conclusions! We have argued that grade school multiplication uses more time than grade school addition. This is a comparison of the complexity of two algorithms. To argue that multiplication is an inherently harder problem than addition we would have to show that no possible multiplication algorithm runs in linear time.

24. COSC 3101, PROF. J. ELDER 23 Grade School Addition: θ (n) time Grade School Multiplication: θ (n 2 ) time Is there a clever algorithm to multiply two numbers in linear time?

25. COSC 3101, PROF. J. ELDER 24 Despite years of research, no one knows!

26. COSC 3101, PROF. J. ELDER 25 Is there a faster way to multiply two numbers than the way you learned in grade school?

27. COSC 3101, PROF. J. ELDER 26 Good question!

28. COSC 3101, PROF. J. ELDER 27 Recursive Divide And Conquer DIVIDE a problem into smaller subproblems CONQUER them recursively GLUE the answers together so as to obtain the answer to the larger problem

29. COSC 3101, PROF. J. ELDER 28 Multiplication of 2 n-bit numbers X = Y = X = a 2 n/2 + b Y = c 2 n/2 + d XY = ac 2 n + (ad+bc) 2 n/2 + bd ab cd

30. COSC 3101, PROF. J. ELDER 29 Multiplication of 2 n-bit numbers X = Y = XY = ac 2 n + (ad+bc) 2 n/2 + bd ab cd MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d RETURN MULT(a,c) 2 n + (MULT(a,d) + MULT(b,c)) 2 n/2 + MULT(b,d)

31. COSC 3101, PROF. J. ELDER 30 Time required by MULT T(n) = time taken by MULT on two n-bit numbers What is T(n)?

32. COSC 3101, PROF. J. ELDER 31 Recurrence Relation T(1) = k for some constant k T(n) = 4 T(n/2) + k’ n + k’’ for some constants k’ and k’’ MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d RETURN MULT(a,c) 2 n + (MULT(a,d) + MULT(b,c)) 2 n/2 + MULT(b,d)

33. COSC 3101, PROF. J. ELDER 32 For example T(1) = 1 T(n) = 4 T(n/2) + n How do we unravel T(n) so that we can determine its growth rate?

34. COSC 3101, PROF. J. ELDER 33 Technique 1: (Substitution) Recurrence: Guess: Proof:

35. COSC 3101, PROF. J. ELDER 34 T(n) Technique 2: Recursion Tree T(n) = n + 4 T(n/2) T(1) = 1 n (n/2) = n T(n/2) = T(1) 1 =

36. COSC 3101, PROF. J. ELDER 35 n T(n/2) T(n) =

37. COSC 3101, PROF. J. ELDER 36 n T(n/2) T(n) = n/2 T(n/4)

38. COSC 3101, PROF. J. ELDER 37 n T(n) = n/2 T(n/4) n/2 T(n/4) n/2 T(n/4) n/2 T(n/4)

39. COSC 3101, PROF. J. ELDER 38 n T(n) = n/2 11111111111111111111111111111111...... 111111111111111111111111111111111 n/4

40. COSC 3101, PROF. J. ELDER 39 Level i is the sum of 4 i copies of n/2 i 1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1 +1+1+1............. n/2 + n/2 + n/2 + n/2 n n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 0 1 2 i

41. COSC 3101, PROF. J. ELDER 40 =1  n = 4  n/2 = 16  n/4 = 4 i  n/2 i = 4 logn  n/2 logn =2 2logn  1=n 2

42. COSC 3101, PROF. J. ELDER 41 =1  n = 4  n/2 = 16  n/4 = 4 i  n/2 i = 4 logn  n/2 logn =2 2logn  1=n 2

43. COSC 3101, PROF. J. ELDER 42 Divide and Conquer MULT: θ (n 2 ) time Grade School Multiplication: θ (n 2 ) time All that work for nothing!

44. COSC 3101, PROF. J. ELDER 43 MULT revisited MULT calls itself 4 times. Can you see a way to reduce the number of calls? MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d RETURN MULT(a,c) 2 n + (MULT(a,d) + MULT(b,c)) 2 n/2 + MULT(b,d)

45. COSC 3101, PROF. J. ELDER 44 Gauss’ Idea: Input: a,b,c,d Output: ac, ad+bc, bd A 1 = ac A 3 = bd m 3 = (a+b)(c+d) = ac + ad + bc + bd A 2 = m 3 – A 1 - A 3 = ad + bc

46. COSC 3101, PROF. J. ELDER 45 Gaussified MULT (Karatsuba 1962) T(n) = 3 T(n/2) + n (More precisely: T(n) = 2 T(n/2) + T(n/2 + 1) + kn) MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e2 n + (MULT(a+b, c+d) – e - f) 2 n/2 + f

47. COSC 3101, PROF. J. ELDER 46 n T(n) = n/2 T(n/4) n/2 T(n/4) n/2 T(n/4) n/2 T(n/4)

48. COSC 3101, PROF. J. ELDER 47 n T(n/2) T(n) =

49. COSC 3101, PROF. J. ELDER 48 n T(n/2) T(n) = n/2 T(n/4)

50. COSC 3101, PROF. J. ELDER 49 n T(n) = n/2 T(n/4) n/2 T(n/4) n/2 T(n/4)

51. COSC 3101, PROF. J. ELDER 50 Level i is the sum of 3 i copies of n/2 i 1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+ 1+1+1............. n/2 + n/2 + n/2 n n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 0 1 2 i

52. COSC 3101, PROF. J. ELDER 51 =1  n = 3  n/2 = 9  n/4 = 3 i  n/2 i = 3 logn  n/2 logn =n log3  1

53. COSC 3101, PROF. J. ELDER 52 Dramatic improvement for large n Not just a 25% savings! θ( n 2 ) vs θ( n 1.58.. )

54. COSC 3101, PROF. J. ELDER 53 Grade-School Multiplication X * * * * * * * * * * * * * * * * * * * * * * * * * * * * n2n2

55. COSC 3101, PROF. J. ELDER 54 Gaussified MULT (Karatsuba 1962) T(n) ≈ 3 T(n/2) + kn What is k? MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e2 n + (MULT(a+b, c+d) – e - f) 2 n/2 + f e.g., k=8

56. COSC 3101, PROF. J. ELDER 55 Dramatic improvement for large n Not just a 25% savings! θ(2 n 2 ) vs θ(8 n 1.58.. ) Example: A networking simulation requires 10 million multiplications of 16-bit integers. Suppose that each bit operation takes 4 picosec on your machine (realistic). Grade School Multiplication Time = 2 days 9 hours (do it over the weekend!) Karatsuba Multiplication Time = 5.4 minutes (just enough time to grab a coffee!) MATLAB takes 0.07 seconds on my machine (don’t blink!)

57. COSC 3101, PROF. J. ELDER 56 Multiplication Algorithms Kindergartenn2 n Grade Schooln2n2 Karatsuban 1.58… Fastest Known (Schönhage-Strassen algorithm, 1971) n logn loglogn

58. COSC 3101, PROF. J. ELDER 57 What a difference a single recursive call makes! What are the underlying principles here? How can we systematically predict which recursive algorithms are going to save time, and which are not?

59. Recurrence Relations T(1) = 1 T(n) = a T(n/b) + f(n)

60. COSC 3101, PROF. J. ELDER 59 Recurrence Relations  Time of Recursive Program procedure Eg(int n) if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a Eg(n/b) Recurrence relations arise from the timing of recursive programs. Let T(n) be the # of “Hi”s on an input of “size” n.

61. COSC 3101, PROF. J. ELDER 60 Recurrence Relations  Time of Recursive Program Given size 1, the program outputs T(1)=1 Hi’s. Given size n, the stackframe outputs f(n) Hi’s. Recursing on a instances of size n/b generates aT(n/b) “Hi”s. For a total of T(1) = 1; T(n) = a·T(n/b) + f(n) “Hi”s. procedure Eg(int n) if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a Eg(n/b)

62. COSC 3101, PROF. J. ELDER 61 Technique 1: (Substitution) Recurrence: Guess: Proof:

63. COSC 3101, PROF. J. ELDER 62 More Generally, and Formally

64. COSC 3101, PROF. J. ELDER 63 To Summarize

65. COSC 3101, PROF. J. ELDER 64 Base Case

66. COSC 3101, PROF. J. ELDER 65 Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: T(n) = an 2 + bn + c Verify: Left Hand SideRight Hand Side T(1) = a+b+c T(n) = an 2 +bn+c 1 4T(n/2) + n = 4 [a ( n / 2 ) 2 + b ( n / 2 ) +c] + n = an 2 +(2b+1)n + 4c Solving Technique 2 Guess Form and Calculate Coefficients

67. COSC 3101, PROF. J. ELDER 66 Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: T(n) = an 2 + bn + c Verify: Left Hand SideRight Hand Side T(1) = a+b+c T(n) = an 2 +bn+c 1 4T(n/2) + n = 4 [a ( n / 2 ) 2 + b ( n / 2 ) +c] + n = an 2 +(2b+1)n + 4c Solving Technique 2 Guess Form and Calculate Coefficients c=4c  c=0

68. COSC 3101, PROF. J. ELDER 67 Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: T(n) = an 2 +bn + 0 Verify: Left Hand SideRight Hand Side T(1) = a+b+c T(n) = an 2 +bn+c 1 4T(n/2) + n = 4 [a ( n / 2 ) 2 + b ( n / 2 ) +c] + n = an 2 +(2b+1)n + 4c Solving Technique 2 Guess Form and Calculate Coefficients b = 2b+1  b = -1

69. COSC 3101, PROF. J. ELDER 68 Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: T(n) = an 2 - 1n + 0 Verify: Left Hand SideRight Hand Side T(1) = a+b+c T(n) = an 2 +bn+c 1 4T(n/2) + n = 4 [a ( n / 2 ) 2 + b ( n / 2 ) +c] + n = an 2 +(2b+1)n + 4c Solving Technique 2 Guess Form and Calculate Coefficients a=a

70. COSC 3101, PROF. J. ELDER 69 Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: T(n) = an 2 - 1n + 0 Verify: Left Hand SideRight Hand Side T(1) = a+b+c T(n) = an 2 +bn+c 1 4T(n/2) + n = 4 [a ( n / 2 ) 2 + b ( n / 2 ) +c] + n = an 2 +(2b+1)n + 4c Solving Technique 2 Guess Form and Calculate Coefficients a+b+c=1 a-1+0=1  a=2

71. COSC 3101, PROF. J. ELDER 70 Recurrence Relation: T(1) = 1 & T(n) = aT(n/b) + f(n) Solving Technique 3 Approximate Form and Calculate Exponent which is bigger? Guess

72. COSC 3101, PROF. J. ELDER 71 Recurrence Relation: T(1) = 1 & T(n) = aT(n/b) + f(n) Guess: aT(n/b) << f(n) Simplify: T(n)  f(n) Solving Technique 3 Calculate Exponent In this case, the answer is easy. T(n) =  (f(n))

73. COSC 3101, PROF. J. ELDER 72 Recurrence Relation: T(1) = 1 & T(n) = aT(n/b) + f(n) Guess: aT(n/b) >> f(n) Simplify: T(n)  aT(n/b) Solving Technique 3 Calculate Exponent In this case, the answer is harder.

74. COSC 3101, PROF. J. ELDER 73 Recurrence Relation: T(1) = 1 & T(n) = aT(n/b) Guess: T(n) = cn  Verify: Left Hand SideRight Hand Side T(n) = cn  aT(n/b) = a [c ( n / b )  ] = c a b  n  Solving Technique 3 Calculate Exponent ( log a / log b ) = cn 1 = a b -  b  = a  log b = log a  = log a / log b

75. COSC 3101, PROF. J. ELDER 74 Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) Guess: T(n) = cn  Verify: Left Hand SideRight Hand Side T(n) = cn  aT(n/b) = a [c ( n / b )  ] = c a b  n  Solving Technique 3 Calculate Exponent ( log a / log b ) = cn 1 = a b -  b  = a  log b = log a  = log a / log b

76. COSC 3101, PROF. J. ELDER 75 Recurrence Relation: T(1) = 1 & T(n) = aT(n/b) + f(n) Solving Technique 3 Calculate Exponent If bigger then T(n) =  (f(n)) If bigger then ( log a / log b ) T(n) =  (n ) And if aT(n/b)  f(n) what is T(n) then?

77. COSC 3101, PROF. J. ELDER 76 Technique 4: Recursion Tree Method T(n) = a T(n/b) + f(n) f(n) T(n/b) T(n) = T(1) T(1) = 1 1 = a

78. COSC 3101, PROF. J. ELDER 77 f(n) T(n/b) T(n) = a

79. COSC 3101, PROF. J. ELDER 78 f(n) T(n/b) T(n) = a f(n/b) T(n/b 2 ) a

80. COSC 3101, PROF. J. ELDER 79 f(n) T(n) = a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a

81. COSC 3101, PROF. J. ELDER 80 11111111111111111111111111111111...... 111111111111111111111111111111111 f(n) T(n) = a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a

82. COSC 3101, PROF. J. ELDER 81 Evaluating: T(n) = aT(n/b)+f(n) Level 0 1 2 i h # Instances

83. COSC 3101, PROF. J. ELDER 82 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0 1 2 i h # Instances

84. COSC 3101, PROF. J. ELDER 83 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 2 i h # Instances

85. COSC 3101, PROF. J. ELDER 84 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 i h # Instances

86. COSC 3101, PROF. J. ELDER 85 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i h # Instances

87. COSC 3101, PROF. J. ELDER 86 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i n/b i h n/b h # Instances

88. COSC 3101, PROF. J. ELDER 87 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i n/b i h n/b h # Instances

89. COSC 3101, PROF. J. ELDER 88 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i n/b i h n/b h = 1 base case # Instances

90. COSC 3101, PROF. J. ELDER 89 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i n/b i h n/b h = 1 # Instances

91. COSC 3101, PROF. J. ELDER 90 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i n/b i h = log n / log b n/b h = 1 b h = n h log b = log n h = log n / log b # Instances

92. COSC 3101, PROF. J. ELDER 91 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame 0n 1 n/b 2 n/b 2 i n/b i h = log n / log b 1 # Instances

93. COSC 3101, PROF. J. ELDER 92 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame 0n f(n) 1 n/bf(n/b) 2 n/b 2 f(n/b 2 ) i n/b i f(n/b i ) h = log n / log b 1T(1) # Instances

94. COSC 3101, PROF. J. ELDER 93 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames 0n f(n) 1 n/bf(n/b) 2 n/b 2 f(n/b 2 ) i n/b i f(n/b i ) h = log n / log b n/b h T(1) # Instances

95. COSC 3101, PROF. J. ELDER 94 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames 0n f(n) 1 1 n/bf(n/b) a 2 n/b 2 f(n/b 2 ) a2a2 i n/b i f(n/b i ) aiai h = log n / log b n/b h T(1) ahah # Instances

96. COSC 3101, PROF. J. ELDER 95 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames 0n f(n) 1 1 n/bf(n/b) a 2 n/b 2 f(n/b 2 ) a2a2 i n/b i f(n/b i ) aiai h = log n / log b n/b h T(1) ahah # Instances

97. COSC 3101, PROF. J. ELDER 96 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames 0n f(n) 1 1 n/bf(n/b) a 2 n/b 2 f(n/b 2 ) a2a2 i n/b i f(n/b i ) aiai h = log n / log b n/b h T(1) ahah # Instances

98. COSC 3101, PROF. J. ELDER 97 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames Work in Level 0n f(n) 1 1 n/bf(n/b) a 2 n/b 2 f(n/b 2 ) a2a2 i n/b i f(n/b i ) aiai h = log n / log b n/b h T(1) n log a / log b # Instances

99. COSC 3101, PROF. J. ELDER 98 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames Work in Level 0n f(n) 1 1 · f(n) 1 n/bf(n/b) a a · f(n/b) 2 n/b 2 f(n/b 2 ) a2a2 a 2 · f(n/b 2 ) i n/b i f(n/b i ) aiai a i · f(n/b i ) h = log n / log b n/b h T(1) n log a / log b n · T(1) log a / log b Total Work T(n) = ∑ i=0..h a i  f(n/b i ) # Instances

100. COSC 3101, PROF. J. ELDER 99 Evaluating: T(n) = aT(n/b)+f(n) = ∑ i=0..h a i  f(n/b i ) If a Geometric Sum ∑ i=0..n x i  θ(max(first term, last term))

101. COSC 3101, PROF. J. ELDER 100 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames Work in Level 0n f(n) 1 1 · f(n) 1 n/bf(n/b) a a · f(n/b) 2 n/b 2 f(n/b 2 ) a2a2 a 2 · f(n/b 2 ) i n/b i f(n/b i ) aiai a i · f(n/b i ) h = log n / log b n/b h T(1) n log a / log b n · T(1) log a / log b Dominated by Top Level or Base Cases

102. End of Lecture 7 March 25, 2009

103. COSC 3101, PROF. J. ELDER 102

104. COSC 3101, PROF. J. ELDER 103

105. COSC 3101, PROF. J. ELDER 104 

106. COSC 3101, PROF. J. ELDER 105

107. COSC 3101, PROF. J. ELDER 106 But what about this?

108. COSC 3101, PROF. J. ELDER 107

109. COSC 3101, PROF. J. ELDER 108

110. COSC 3101, PROF. J. ELDER 109

111. COSC 3101, PROF. J. ELDER 110

112. COSC 3101, PROF. J. ELDER 111

113. COSC 3101, PROF. J. ELDER 112 05101520 0 5 10 15 20 25 30 35 n f(n) Master Theorem Case 3: When the Regularity Condition Fails

114. COSC 3101, PROF. J. ELDER 113 05101520 0 5 10 15 20 25 30 35 n f(n) Master Theorem Case 3: When the Regularity Condition Fails

115. COSC 3101, PROF. J. ELDER 114 05101520 0 5 10 15 20 25 30 35 n f(n) Master Theorem Case 3: When the Regularity Condition Fails

116. COSC 3101, PROF. J. ELDER 115 Master Theorem Case 3: When the Regularity Condition Fails Let’s sleep on it.

117. Central Algorithmic Techniques Recursion

118. COSC 3101, PROF. J. ELDER 117 Different Representations of Recursive Algorithms Code- Implement on Computer Stack of Stack Frames- Run on Computer Tree of Stack Frames- View entire computation Friends & Strong Induction- Worry about one step at a time. ProsViews

119. COSC 3101, PROF. J. ELDER 118 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Code Representation of an Algorithm Pros and Cons?

120. COSC 3101, PROF. J. ELDER 119 Code Representation of an Algorithm Runs on computers Precise and succinct I am not a computer I need a higher level of intuition. Prone to bugs Language dependent Pros:Cons:

121. COSC 3101, PROF. J. ELDER 120 Different Representations of Recursive Algorithms Code- Implement on Computer Stack of Stack Frames- Run on Computer Tree of Stack Frames- View entire computation Friends & Strong Induction- Worry about one step at a time. ProsViews

122. COSC 3101, PROF. J. ELDER 121 X = 2133 Y = 2312 ac = bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames Stack Frame: A particular execution of one routine on one particular input instance.

123. COSC 3101, PROF. J. ELDER 122 X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

124. COSC 3101, PROF. J. ELDER 123 X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

125. COSC 3101, PROF. J. ELDER 124 X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

126. COSC 3101, PROF. J. ELDER 125 X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = ? bd = (a+b)(c+d) = XY = X = 2 Y = 2 XY= MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

127. COSC 3101, PROF. J. ELDER 126 X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = ? bd = (a+b)(c+d) = XY = X = 2 Y = 2 XY = 4 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

128. COSC 3101, PROF. J. ELDER 127 X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

129. COSC 3101, PROF. J. ELDER 128 X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = ? (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

130. COSC 3101, PROF. J. ELDER 129 X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = ? (a+b)(c+d) = XY = X = 1 Y = 3 XY = 3 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

131. COSC 3101, PROF. J. ELDER 130 X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

132. COSC 3101, PROF. J. ELDER 131 X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = ? XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

133. COSC 3101, PROF. J. ELDER 132 X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = ? XY = X = 3 Y = 5 XY = 15 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

134. COSC 3101, PROF. J. ELDER 133 X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = ? MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

135. COSC 3101, PROF. J. ELDER 134 X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

136. COSC 3101, PROF. J. ELDER 135 X = 2133 Y = 2312 ac = 483 bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

137. COSC 3101, PROF. J. ELDER 136 X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

138. COSC 3101, PROF. J. ELDER 137 X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = ? bd = (a+b)(c+d) = XY = 15 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

139. COSC 3101, PROF. J. ELDER 138 X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = ? bd = (a+b)(c+d) = XY = 15 X = 3 Y = 1 XY = 3 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

140. COSC 3101, PROF. J. ELDER 139 X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = ? (a+b)(c+d) = XY = 15 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

141. COSC 3101, PROF. J. ELDER 140 X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = ? (a+b)(c+d) = XY = 15 X = 3 Y = 2 XY = 6 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

142. COSC 3101, PROF. J. ELDER 141 X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = ? XY = 15 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

143. COSC 3101, PROF. J. ELDER 142 X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = ? XY = 396 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames An so on ….

144. COSC 3101, PROF. J. ELDER 143 Stack of Stack Frames Representation of an Algorithm Traces what actually occurs in the computer Concrete. Described in words it is impossible to follow Does not explain why it works. Demonstrates for only one of many inputs. Pros:Cons:

145. COSC 3101, PROF. J. ELDER 144 Different Representations of Recursive Algorithms Code- Implement on Computer Stack of Stack Frames- Run on Computer Tree of Stack Frames- View entire computation Friends & Strong Induction- Worry about one step at a time. ProsViews

146. COSC 3101, PROF. J. ELDER 145 X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = 4931496 X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X = 2 Y = 2 XY=4 X = 1 Y = 3 XY=3 X = 3 Y = 5 XY=15 X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18 X = 5 Y = 3 XY=15 X = 4 Y = 5 XY=20 X = 9 Y = 8 XY=72 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Tree of Stack Frames

147. COSC 3101, PROF. J. ELDER 146 Stack of Stack Frames Representation of an Algorithm View the entire computation. Good for computing the running time. Must describe entire tree. –For each stack frame input instance computation solution returned Pros:Cons:

148. COSC 3101, PROF. J. ELDER 147 Different Representations of Recursive Algorithms Code- Implement on Computer Stack of Stack Frames- Run on Computer Tree of Stack Frames- View entire computation Friends & Strong Induction- Worry about one step at a time. ProsViews

149. COSC 3101, PROF. J. ELDER 148 X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = 4931496 X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X = 2 Y = 2 XY=4 X = 1 Y = 3 XY=3 X = 3 Y = 5 XY=15 X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18 X = 5 Y = 3 XY=15 X = 4 Y = 5 XY=20 X = 9 Y = 8 XY=72 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Friends & Strong Induction One Friend for each stack frame. Each worries only about his job.

150. COSC 3101, PROF. J. ELDER 149 X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = 4931496 X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X = 2 Y = 2 XY=4 X = 1 Y = 3 XY=3 X = 3 Y = 5 XY=15 X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18 X = 5 Y = 3 XY=15 X = 4 Y = 5 XY=20 X = 9 Y = 8 XY=72 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Friends & Strong Induction Worry about one step at a time. Imagine that you are one specific friend.

151. COSC 3101, PROF. J. ELDER 150 X = 33 Y = 12 ac = bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Friends & Strong Induction Consider your input instance

152. COSC 3101, PROF. J. ELDER 151 X = 33 Y = 12 ac = ? bd = ? (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Friends & Strong Induction Consider your input instance Allocate work Construct one or more subinstances X = 3 Y = 1 XY=? X = 3 Y = 2 XY=? X = 6 Y = 3 XY=?

153. COSC 3101, PROF. J. ELDER 152 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Friends & Strong Induction Consider your input instance Allocate work Construct one or more subinstances Assume by magic your friends give you the answer for these. X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18

154. COSC 3101, PROF. J. ELDER 153 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Friends & Strong Induction Consider your input instance Allocate work Construct one or more subinstances Assume by magic your friends give you the answer for these. Use this help to solve your own instance. X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18

155. COSC 3101, PROF. J. ELDER 154 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Friends & Strong Induction Consider your input instance Allocate work Construct one or more subinstances Assume by magic your friends give you the answer for these. Use this help to solve your own instance. Do not worry about anything else, e.g., Who your boss is. How your friends solve their instance. X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18

156. COSC 3101, PROF. J. ELDER 155 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Friends & Strong Induction This technique is often referred to as Divide and Conquer X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18

157. COSC 3101, PROF. J. ELDER 156 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Friends & Strong Induction Consider generic instances. ac bd(a+b)(c+d) MULT(X,Y): Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f MULT(X,Y): If |X| = |Y| = 1 then RETURN XY

158. COSC 3101, PROF. J. ELDER 157 X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = 4931496 X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X = 2 Y = 2 XY=4 X = 1 Y = 3 XY=3 X = 3 Y = 5 XY=15 X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18 X = 5 Y = 3 XY=15 X = 4 Y = 5 XY=20 X = 9 Y = 8 XY=72 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Friends & Strong Induction This solves the problem for every possible instance.

159. COSC 3101, PROF. J. ELDER 158 Friends & Strong Induction Recursive Algorithm: Assume you have an algorithm that works. Use it to write an algorithm that works.

160. COSC 3101, PROF. J. ELDER 159 Friends & Strong Induction Recursive Algorithm: Assume you have an algorithm that works. Use it to write an algorithm that works. If I could get in, I could get the key. Then I could unlock the door so that I can get in. Circular Argument!

161. COSC 3101, PROF. J. ELDER 160 Friends & Strong Induction Recursive Algorithm: Assume you have an algorithm that works. Use it to write an algorithm that works. To get into my house I must get the key from a smaller house

162. COSC 3101, PROF. J. ELDER 161 X = 33 Y = 12 ac = ? bd = ? (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Friends & Strong Induction Allocate work Construct one or more subinstances X = 3 Y = 1 XY=? X = 3 Y = 2 XY=? X = 6 Y = 3 XY=? Each subinstance must be a smaller instance to the same problem.

163. COSC 3101, PROF. J. ELDER 162 Friends & Strong Induction Recursive Algorithm: Assume you have an algorithm that works. Use it to write an algorithm that works. Use brute force to get into the smallest house.

164. COSC 3101, PROF. J. ELDER 163 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Friends & Strong Induction MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Use brute force to solve the base case instances.

165. COSC 3101, PROF. J. ELDER 164 Friends & Strong Induction Carefully write the specifications for the problem. Preconditions:Set of legal instances (inputs) Why? Postconditions:Required output

166. COSC 3101, PROF. J. ELDER 165 Friends & Strong Induction Carefully write the specifications for the problem. Preconditions:Set of legal instances (inputs) To be sure that we solve the problem for every legal instance. So that we know –what we can give to a friend. Postconditions:Required outputSo that we know –what is expected of us. –what we can expect from our friend. Related to Loop Invariants

167. Applications of Recursion Another Numerical Computation Example

168. COSC 3101, PROF. J. ELDER 167 Given two integers, what is their greatest common divisor? e.g., gcd(56,24) = The Greatest Common Divisor (GCD) Problem 

169. COSC 3101, PROF. J. ELDER 168 Euclid’s Trick Good! Better! Too Far! Euclid of Alexandria, "The Father of Geometry" c. 300 BC

170. COSC 3101, PROF. J. ELDER 169 Euclid’s Algorithm (circa 300 BC)

171. End of Lecture 8 March 30, 2009

172. COSC 3101, PROF. J. ELDER 171 Time Complexity

173. COSC 3101, PROF. J. ELDER 172 Time Complexity

174. Applications of Recursion Data Organization

175. A Simple Example: The Tower of Hanoi

176. COSC 3101, PROF. J. ELDER 175 Tower of Hanoi This job of mine is a bit daunting. Where do I start? And I am lazy.

177. COSC 3101, PROF. J. ELDER 176 Tower of Hanoi At some point, the biggest disk moves. I will do that job.

178. COSC 3101, PROF. J. ELDER 177 Tower of Hanoi To do this, the other disks must be in the middle.

179. COSC 3101, PROF. J. ELDER 178 Tower of Hanoi How will these move? I will get a friend to do it. And another to move these. I only move the big disk.

180. COSC 3101, PROF. J. ELDER 179 Tower of Hanoi

181. COSC 3101, PROF. J. ELDER 180 Tower of Hanoi Time: T(1) = 1, T(n) = ≈ 2(2T(n-2)) ≈ 4(2T(n-3)) ≈ 2T(n-1) ≈ 4T(n-2) ≈ 8T(n-3) ≈ 2 i T(n-i) ≈ 2 n 1 + 2T(n-1)

182. More Data Organization Examples Sorting

183. COSC 3101, PROF. J. ELDER 182 Recursive Sorts Given list of objects to be sorted Split the list into two sublists. Recursively have a friend sort the two sublists. Combine the two sorted sublists into one entirely sorted list.

184. Example: Merge Sort

185. COSC 3101, PROF. J. ELDER 184 Merge Sort 88 14 98 25 62 52 79 30 23 31 Divide and Conquer

186. COSC 3101, PROF. J. ELDER 185 Merge Sort 88 14 98 25 62 52 79 30 23 31 Split Set into Two (no real work) 25,31,52,88,98 Get one friend to sort the first half. 14,23,30,62,79 Get one friend to sort the second half.

187. COSC 3101, PROF. J. ELDER 186 Merge Sort Merge two sorted lists into one 25,31,52,88,9814,23,30,62,7914,23,25,30,31,52,62,79,88,98

188. COSC 3101, PROF. J. ELDER 187 Merge Sort Time: T(n) = =  (n log(n)) 2T(n/2) +  (n)

189. Example: Quick Sort

190. COSC 3101, PROF. J. ELDER 189 Quick Sort 88 14 98 25 62 52 79 30 23 31 Divide and Conquer

191. COSC 3101, PROF. J. ELDER 190 Quick Sort 88 14 98 25 62 52 79 30 23 31 Partition set into two using randomly chosen pivot 14 25 30 23 31 88 98 62 79 ≤ 52 ≤

192. COSC 3101, PROF. J. ELDER 191 Quick Sort 14 25 30 23 31 88 98 62 79 ≤ 52 ≤ 14,23,25,30,31 Get one friend to sort the first half. 62,79,98,88 Get one friend to sort the second half.

193. COSC 3101, PROF. J. ELDER 192 Quick Sort 14,23,25,30,31 62,79,98,8852 Glue pieces together. (No real work) 14,23,25,30,31,52,62,79,88,98

194. COSC 3101, PROF. J. ELDER 193

195. COSC 3101, PROF. J. ELDER 194 Quick Sort 88 14 98 25 62 52 79 30 23 31 Let pivot be the first element in the list? 14 25 30 23 88 98 62 79 ≤ 31 ≤ 52

196. COSC 3101, PROF. J. ELDER 195 Quick Sort ≤ 14 ≤ 14,23,25,30,31,52,62,79,88,98 23,25,30,31,52,62,79,88,98 If the list is already sorted, then the list is worst case unbalanced.

197. COSC 3101, PROF. J. ELDER 196 Quick Sort Best Time: Worst Time: Expected Time: T(n) = 2T(n/2) +  (n) =  (n log(n))

198. COSC 3101, PROF. J. ELDER 197 Quick Sort T(n) = 2T(n/2) +  (n) =  (n log(n)) Best Time: Worst Time: Expected Time: =  (n 2 ) T(n) = T(1) + T(n-1) +  (n)

199. COSC 3101, PROF. J. ELDER 198 Quick Sort T(n) = 2T(n/2) +  (n) =  (n log(n)) Best Time: T(n) = T(0) + T(n-1) +  (n) Worst Time: Expected Time: =  (n 2 ) T(n) =  (nlog(n)) (The proof is not difficult, but it’s a little long)

200. COSC 3101, PROF. J. ELDER 199 Expected Time Complexity for Quick Sort

201. COSC 3101, PROF. J. ELDER 200 Expected Time Complexity for Quick Sort

202. COSC 3101, PROF. J. ELDER 201 Properties of QuickSort In-place? Stable? Fast? –Depends. –Worst Case: –Expected Case:

203. COSC 3101, PROF. J. ELDER 202 Heaps, Heap Sort, & Priority Queues

204. COSC 3101, PROF. J. ELDER 203 Heapsort O(nlogn) worst case – like merge sort Sorts in place – like insertion sort Combines the best of both algorithms

205. COSC 3101, PROF. J. ELDER 204 Heap Definition (MaxHeap) Balanced binary tree The value of each node  each of the node's children. Left or right child could be next largest. Where can 1 go? Maximum is at root. Where can 8 go? Where can 9 go?

206. COSC 3101, PROF. J. ELDER 205 Some Additional Properties of Heaps h =0 1 0 2 1 3

207. COSC 3101, PROF. J. ELDER 206 Some Additional Properties of Heaps i = 1 i = 2 i = 3 i = 4 i = 5 i = 6i = 7 i = 8 i = 9

208. COSC 3101, PROF. J. ELDER 207 Some Additional Properties of Heaps i = 1 i = 2 i = 3 i = 4 i = 5 i = 6i = 7 i = 8 i = 9

209. COSC 3101, PROF. J. ELDER 208 Heap Data Structure Balanced Binary Tree Implemented by an Array

210. COSC 3101, PROF. J. ELDER 209 Make Heap Get help from friends Now we are just left with this problem

211. COSC 3101, PROF. J. ELDER 210 Heapify ? Maximum must be at root. Where should the maximum be? : Left and right subtrees of A[i] are max heaps. : Subtree rooted at i is a heap. Max-Heapify(A, i, n) i

212. COSC 3101, PROF. J. ELDER 211 Find the maximum. ? Repeat Heapify : Left and right subtrees of A[i] are max heaps. : Subtree rooted at i is a heap. Max-Heapify(A, i, n) i Put it in place

213. COSC 3101, PROF. J. ELDER 212 Heap Running Time: Heapify : Left and right subtrees of A[i] are max heaps. : Subtree rooted at i is a heap. Max-Heapify(A, i, n)

214. COSC 3101, PROF. J. ELDER 213 : Left and right subtrees of A[i] are max heaps. : Subtree rooted at i is a heap. Max-Heapify(A, i, n) e.g., Max-Heapify (A,2,10)  Max-Heapify (A,4,10)  Max-Heapify (A,9,10)

215. End of Lecture 9 April 1, 2009

216. COSC 3101, PROF. J. ELDER 215 MakeHeap MakeHeap uses Max-Heapify to reorganize the tree from bottom to top to make it a heap. MakeHeap can be written concisely in either recursive or iterative form.

217. COSC 3101, PROF. J. ELDER 216 Recursive MakeHeap T(n) = 2T(n/2) + log(n) Running time: =  (n) i n Invoke as MakeHeap (A, 1, n)

218. COSC 3101, PROF. J. ELDER 217 Recursive MakeHeap i n Question from last class: what if i = 4?? What then is n??

219. COSC 3101, PROF. J. ELDER 218 Recursive MakeHeap i = 1 i = 2 i = 3 i = 4 n = ? i = 5 i = 6 i = 7

220. COSC 3101, PROF. J. ELDER 219 Heaps Heap ? : Left and right subtrees of A[i] are max heaps. : Subtree rooted at i is a heap. Max-Heapify(A, i, n) Iterative MakeHeap

221. COSC 3101, PROF. J. ELDER 220 : Left and right subtrees of A[i] are max heaps. : Subtree rooted at i is a heap. Max-Heapify(A, i, n) Iterative MakeHeap ? Heap

222. COSC 3101, PROF. J. ELDER 221 : Left and right subtrees of A[i] are max heaps. : Subtree rooted at i is a heap. Max-Heapify(A, i, n) Iterative MakeHeap ? Heap

223. COSC 3101, PROF. J. ELDER 222 : Left and right subtrees of A[i] are max heaps. : Subtree rooted at i is a heap. Max-Heapify(A, i, n) Iterative MakeHeap ?

224. COSC 3101, PROF. J. ELDER 223 : Left and right subtrees of A[i] are max heaps. : Subtree rooted at i is a heap. Max-Heapify(A, i, n) Iterative MakeHeap Heap

225. COSC 3101, PROF. J. ELDER 224 Iterative MakeHeap

226. COSC 3101, PROF. J. ELDER 225 Iterative MakeHeap Recursive and Iterative MakeHeap do essentially the same thing: Heapify from bottom to top. Difference: –Recursive is “depth-first” –Iterative is “breadth-first”

227. Using Heaps for Sorting

228. COSC 3101, PROF. J. ELDER 227 Selection Sort Largest i values are sorted on the right. Remaining values are off to the left. 6,7,8,9 < 3 4 1 5 2 Max is easier to find if a heap.

229. COSC 3101, PROF. J. ELDER 228 4 3 1 2 Heap Sort Largest i values are sorted on side. Remaining values are in a heap.

230. COSC 3101, PROF. J. ELDER 229 4 3 1 2 4 3 1 2 Heap Data Structure HeapArray 5 43 12 9 8 7 6 Heap Array

231. COSC 3101, PROF. J. ELDER 230 Heap Sort Largest i values are sorted on side. Remaining values are in a heap Put next value where it belongs. Heap ? : Left and right subtrees of A[i] are max heaps. : Subtree rooted at i is a heap. Max-Heapify(A, i, n)

232. COSC 3101, PROF. J. ELDER 231 Heap Sort ? ?? ?? ?? Sorted

233. COSC 3101, PROF. J. ELDER 232 Heap Sort Running Time:

234. Other Applications of Heaps

235. COSC 3101, PROF. J. ELDER 234 Priority Queue Maintains dynamic set, A, of n elements, each with a key. Max-priority queue supports: 1.MAXIMUM(A) 2.EXTRACT-MAX(A, n) 3.INCREASE-KEY(A, i, key) 4.INSERT(A, key, n) Example Application: Schedule jobs on a shared computer.

236. COSC 3101, PROF. J. ELDER 235 Priority Queues cont’d... MAXIMUM(A): EXTRACT-MAX(A,n):

237. COSC 3101, PROF. J. ELDER 236 Priority Queue cont’d... INCREASE-KEY(A, i, key): INSERT(A, key, n):

238. COSC 3101, PROF. J. ELDER 237 Binary Search Trees Support many dynamic-set operations Basic operations take time proportional to height h of tree.

239. COSC 3101, PROF. J. ELDER 238 38 25 17 421 31 2835 51 42 4049 63 5571 Binary Search Tree Left children ≤ Node ≤ Right children ≤ ≤

240. COSC 3101, PROF. J. ELDER 239 BST Data Structure

241. COSC 3101, PROF. J. ELDER 240 Insertion

242. COSC 3101, PROF. J. ELDER 241 38 25 17 421 31 2835 51 42 4049 63 5571 Insertion ? 36

243. COSC 3101, PROF. J. ELDER 242 Building a Tree

244. COSC 3101, PROF. J. ELDER 243 38 25 17 421 31 2835 51 42 4049 63 5571 Searching the Tree PreConditions –Key 25 –A binary search tree. –PostConditions –Find key in BST (if there).

245. COSC 3101, PROF. J. ELDER 244 Searching the Tree Maintain a sub-tree. If the key is contained in the original tree, then the key is contained in the sub-tree. key 17 38 25 17 421 31 2835 51 42 4049 63 5571

246. COSC 3101, PROF. J. ELDER 245 Define Step Cut sub-tree in half. Determine which half the key would be in. Keep that half. key 17 38 25 17 421 31 2835 51 42 4049 63 5571 If key < root, then key is in left half. If key > root, then key is in right half. If key = root, then key is found

247. COSC 3101, PROF. J. ELDER 246 Searching the Tree

248. COSC 3101, PROF. J. ELDER 247 Why use (balanced) binary search trees? What is the advantage over a sorted linear array? –Search time is the same –However, maintaining (inserting, deleting, modifying) is  (logn) for balanced BSTs  (n) for arrays