2. http://www.cse.unl.edu/~goddard/Courses/CSCE310J Fundamentals of Algorithm Analysis Dr. Steve Goddard goddard@cse.unl.edu CSCE 310J: Data Structures & Algorithms

3. Most of the slides in this lecture were originally created by Jeff Edmonds from York University

4. Relevant Mathematics Jeff Edmonds York University Classifying Functions The Time Complexity of an Algorithm Adding Made Easy Recurrence Relations

5. Start With Some Math Input Size Time Classifying Functions f(i) = n  (n) Recurrence Relations T(n) = a T(n/b) + f(n) Adding Made Easy ∑ i=1 f(i). Time Complexity t(n) =  (n 2 )

6. Assumed Knowledge Logarithms and Exponentials Existential and Universal Quantifiers, g  b Loves(b,g)  b g Loves(b,g)

7. Quantifiers: all, some “for all x”  x P(x) is true iff P(x) is true for all x – universal quantifier (universe of discourse) “there exist x”  x P(x) is true iff P(x) is true for some value of x –existential quantifier  x A(x) is logically equivalent to   x(  A(x))  x A(x) is logically equivalent to  x(  A(x))  x (A(x)  B(x)) “For all x such that if A(x) holds then B(x) holds”

8. Classifying Functions Giving an idea of how fast a function grows without going into too much detail.

9. Which are more alike?

10. Mammals

11. Which are more alike?

12. Dogs

13. Classifying Animals Vertebrates BirdsMammals Reptiles Fish Dogs Giraffe

14. Classifying Functions T(n)T(n) 101001,00010,000 log n 36913amoeba n 1/2 31031100bird 101001,00010,000human n log n 306009,000130,000my father n2n2 10010,00010 6 10 8 elephant n3n3 1,00010 6 10 9 10 12 dinosaur 2n2n 1,02410 30 10 300 10 3000 the universe Note: The universe contains approximately 10 50 particles. n

15. Which are more alike? n 1000 n2n2 2n2n

16. Which are more alike? Polynomials n 1000 n2n2 2n2n

17. Which are more alike? 1000n 2 3n 2 2n 3

18. Which are more alike? Quadratic 1000n 2 3n 2 2n 3

19. Classifying Functions? Functions

20. Classifying Functions Functions Constant Logarithmic Poly Logarithmic Polynomial Exponential ExpDouble Exp (log n) 5 n5n5 2 5n 5 log n5 2 n5n5 2 5n 2

21. Classifying Functions? Polynomial

22. Classifying Functions Polynomial LinearQuadratic Cubic ? 5n 2 5n 5n 3 5n 4

23. Logarithmic log 10 n = # digits to write n log 2 n = # bits to write n = 3.32 log 10 n log(n 1000 ) = 1000 log(n) Differ only by a multiplicative constant.

24. Poly Logarithmic (log n) 5 = log 5 n

25. Logarithmic << Polynomial For sufficiently large n log 1000 n << n 0.001

26. Linear << Quadratic For sufficiently large n 10000 n << 0.0001 n 2

27. Polynomial << Exponential For sufficiently large n n 1000 << 2 0.001 n

28. Ordering Functions Functions Constant Logarithmic Poly Logarithmic Polynomial Exponential ExpDouble Exp (log n) 5 n5n5 2 5n 5 log n5 2 n5n5 2 5n 2 <<

29. Which Functions are Constant? 5 1,000,000,000,000 0.0000000000001 -5 0 8 + sin(n) Yes No Yes

30. Which Functions are “Constant”? 5 1,000,000,000,000 0.0000000000001 -5 0 8 + sin(n) Yes No Yes Lie in between 7 9 The running time of the algorithm is a “Constant” It does not depend significantly on the size of the input.

31. Which Functions are Quadratic? n 2 … ?

32. Which Functions are Quadratic? n 2 0.001 n 2 1000 n 2 Some constant times n 2.

33. Which Functions are Quadratic? n 2 0.001 n 2 1000 n 2 5n 2 + 3n + 2log n

34. Which Functions are Quadratic? n 2 0.001 n 2 1000 n 2 5n 2 + 3n + 2log n Lie in between

35. Which Functions are Quadratic? n 2 0.001 n 2 1000 n 2 5n 2 + 3n + 2log n Ignore low-order terms Ignore multiplicative constants. Ignore "small" values of n. Write θ(n 2 ).

36. Which Functions are Polynomial? n 5 … ?

37. Which Functions are Polynomial? n c n 0.0001 n 10000 n to some constant power.

38. Which Functions are Polynomial? n c n 0.0001 n 10000 5n 2 + 8n + 2log n 5n 2 log n 5n 2.5

39. Which Functions are Polynomial? n c n 0.0001 n 10000 5n 2 + 8n + 2log n 5n 2 log n 5n 2.5 Lie in between

40. Which Functions are Polynomials? n c n 0.0001 n 10000 5n 2 + 8n + 2log n 5n 2 log n 5n 2.5 Ignore low-order terms Ignore power constant. Ignore "small" values of n. Write n θ(1)

41. Which Functions are Exponential? 2 n … ?

42. Which Functions are Exponential? 2 n 2 0.0001 n 2 10000 n n times some constant power raises to the power of 2.

43. Which Functions are Exponential? 2 n 2 0.0001 n 2 10000 n 8 n 2 n / n 100 2 n · n 100 too small? too big?

44. Which Functions are Exponential? 2 n 2 0.0001 n 2 10000 n 8 n 2 n / n 100 2 n · n 100 = 2 3n > 2 0.5n < 2 2n Lie in between

45. Which Functions are Exponential? 2 n 2 0.0001 n 2 10000 n 8 n 2 n / n 100 2 n · n 100 = 2 3n > 2 0.5n < 2 2n 2 0.5n > n 100 2 n = 2 0.5n · 2 0.5n > n 100 · 2 0.5n 2 n / n 100 > n 0.5n

46. Which Functions are Exponential? Ignore low-order terms Ignore base. Ignore "small" values of n. Ignore polynomial factors. Write 2 θ(n) 2 n 2 0.0001 n 2 10000 n 8 n 2 n / n 100 2 n · n 100

47. Classifying Functions Functions Constant Logarithmic Poly Logarithmic PolynomialExponential Exp Double Exp (log n) θ(1) n θ(1) 2 θ(n) θ(log n)θ(1) 2 n θ(1) 2 θ(n) 2

48. Notations Theta f(n) = θ(g(n))f(n) ≈ c g(n) BigOh f(n) = O(g(n))f(n) ≤ c g(n) Omega f(n) = Ω(g(n))f(n) ≥ c g(n) Little Oh f(n) = o(g(n))f(n) << c g(n) Little Omega f(n) = ω(g(n))f(n) >> c g(n)

49. Definition of Theta f(n) = θ(g(n))

50. Definition of Theta f(n) is sandwiched between c 1 g(n) and c 2 g(n) f(n) = θ(g(n))

51. Definition of Theta f(n) is sandwiched between c 1 g(n) and c 2 g(n) for some sufficiently small c 1 (= 0.0001) for some sufficiently large c 2 (= 1000) f(n) = θ(g(n))

52. Definition of Theta For all sufficiently large n f(n) = θ(g(n))

53. Definition of Theta For all sufficiently large n For some definition of “sufficiently large” f(n) = θ(g(n))

54. Definition of Theta 3n 2 + 7n + 8 = θ(n 2 )

55. Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) c 1 ·n 2  3n 2 + 7n + 8  c 2 ·n 2 ??

56. Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 34

57. Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·1 2  3·1 2 + 7·1 + 8  4·1 2 1 False

58. Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·7 2  3·7 2 + 7·7 + 8  4·7 2 7 False

59. Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·8 2  3·8 2 + 7·8 + 8  4·8 2 8 True

60. Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·9 2  3·9 2 + 7·9 + 8  4·9 2 9 True

61. Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·10 2  3·10 2 + 7·10 + 8  4·10 2 10 True

62. Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 ?

63. Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 n  8 8

64. Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 True n  8

65. Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 7n + 8  1 · n 2 7 + 8 / n  1 · n n  8 True

66. Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 348 n  8 True

67. Definition of Theta n 2 = θ(n 3 )

68. Definition of Theta n 2 = θ(n 3 ) c 1 · n 3  n 2  c 2 · n 3 ??

69. Definition of Theta n 2 = θ(n 3 ) 0 · n 3  n 2  c 2 · n 3 0 True, but ?

70. Definition of Theta n 2 = θ(n 3 ) 0 Constants c1 and c2 must be positive! False

71. Definition of Theta 3n 2 + 7n + 8 = θ(n)

72. Definition of Theta 3n 2 + 7n + 8 = θ(n) c 1 ·n  3n 2 + 7n + 8  c 2 ·n ??

73. Definition of Theta 3n 2 + 7n + 8 = θ(n) 3·n  3n 2 + 7n + 8  100·n 3 100

74. Definition of Theta 3n 2 + 7n + 8 = θ(n) 3·n  3n 2 + 7n + 8  100·n ?

75. Definition of Theta 3n 2 + 7n + 8 = θ(n) 3·100  3·100 2 + 7·100 + 8  100·100 100 False

76. Definition of Theta 3n 2 + 7n + 8 = θ(n) 3·n  3n 2 + 7n + 8  10,000·n 3 10,000

77. Definition of Theta 3n 2 + 7n + 8 = θ(n) 3·10,000  3·10,000 2 + 7·10,000 + 8  100·10,000 10,000 False

78. Definition of Theta 3n 2 + 7n + 8 = θ(n) What is the reverse statement?

79. Understand Quantifiers!!! SamMary BobBeth John Marilin Monro FredAnn SamMary BobBeth John Marilin Monro FredAnn  b,  Loves(b, MM)  b, Loves(b, MM)  b,  Loves(b, MM)]  b, Loves(b, MM)]

80. Definition of Theta 3n 2 + 7n + 8 = θ(n) The reverse statement

81. Definition of Theta 3n 2 + 7n + 8 = θ(n) 3n 2 + 7n + 8 > 100·n 3 100 ?

82. Definition of Theta 3n 2 + 7n + 8 = θ(n) 3 · 100 2 + 7 · 100 + 8 > 100 · 100 3 100 True 100

83. Definition of Theta 3n 2 + 7n + 8 = θ(n) 3n 2 + 7n + 8 > 10,000·n 3 10,000 ?

84. Definition of Theta 3n 2 + 7n + 8 = θ(n) 3 · 10,000 2 + 7 · 10,000 + 8 > 10,000 · 10,000 3 10,000 True 10,000

85. Definition of Theta 3n 2 + 7n + 8 = θ(n) 3n 2 + 7n + 8 > c 2 ·n c1c1 c2c2 ?

86. Definition of Theta 3n 2 + 7n + 8 = θ(n) 3 · c 2 2 + 7 · c 2 + 8 > c 2 · c 2 c1c1 c2c2 True c2c2

87. Definition of Theta 3n 2 + 7n + 8 = θ(n) 3n 2 + 7n + 8 > c 2 ·n c1c1 c2c2 ? n0n0

88. Definition of Theta 3n 2 + 7n + 8 = θ(n) 3 · c 2 2 + 7 · c 2 + 8 > c 2 · c 2 c1c1 max(c 2,n 0 ) True c2c2 n0n0

89. Definition of Theta 3n 2 + 7n + 8 = g(n) θ(1)  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 3n 2 + 7n + 8 = n θ(1)

90. Definition of Theta 3n 2 + 7n + 8 = n θ(1) n c1  3n 2 + 7n + 8  n c2  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 ??

91. Definition of Theta 3n 2 + 7n + 8 = n θ(1) n 2  3n 2 + 7n + 8  n 3  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 23

92. Definition of Theta 3n 2 + 7n + 8 = n θ(1) n 2  3n 2 + 7n + 8  n 3  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 23? n  ?

93. Definition of Theta 3n 2 + 7n + 8 = n θ(1) n 2  3n 2 + 7n + 8  n 3  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 235 n  5

94. Definition of Theta 3n 2 + 7n + 8 = n θ(1) n 2  3n 2 + 7n + 8  n 3 True  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 235 n  5

95. Order of Quantifiers f(n) = θ(g(n)) ?

96. Understand Quantifiers!!! One girl Could be a separate girl for each boy. SamMary BobBeth John Marilin Monro FredAnn SamMary BobBeth John Marilin Monro FredAnn

97. Order of Quantifiers No! It cannot be a different c 1 and c 2 for each n. f(n) = θ(g(n))

98. Other Notations Theta f(n) = θ(g(n))f(n) ≈ c g(n) BigOh f(n) = O(g(n))f(n) ≤ c g(n) Omega f(n) = Ω(g(n))f(n) ≥ c g(n) Little Oh f(n) = o(g(n))f(n) << c g(n) Little Omega f(n) = ω(g(n))f(n) >> c g(n)

99. BigOh Notation n 2 = O(n 3 ) n 3 = O(n 2 )

100. BigOh Notation O(n 2 ) = O(n 3 ) O(n 3 ) = O(n 2 ) Odd Notation f(n) = O(g(n)) Standard f(n) ≤ O(g(n)) Stresses one function dominating another. f(n)  O(g(n)) Stress function is member of class.

101. The Time Complexity of an Algorithm Specifies how the running time depends on the size of the input.

102. Purpose?

103. Purpose To estimate how long a program will run. To estimate the largest input that can reasonably be given to the program. To compare the efficiency of different algorithms. To help focus on the parts of code that are executed the largest number of times. To choose an algorithm for an application.

104. Time Complexity Is a Function Specifies how the running time depends on the size of the input. A function mapping “size” of input “time” T(n) executed.

105. Definition of Time?

106. Definition of Time # of seconds (machine dependent). # lines of code executed. # of times a specific operation is performed (e.g., addition). Which?

107. Definition of Time # of seconds (machine dependent). # lines of code executed. # of times a specific operation is performed (e.g., addition). These are all reasonable definitions of time, because they are within a constant factor of each other.

108. Size of Input Instance? 83920

109. Size of Input Instance Size of paper # of bits # of digits Value - n = 2 in 2 - n = 17 bits - n = 5 digits - n = 83920 2’’ 83920 5 1’’ Which are reasonable?

110. Size of Input Instance Size of paper # of bits # of digits Value - n = 2 in 2 - n = 17 bits - n = 5 digits - n = 83920 Intuitive 2’’ 83920 5 1’’

111. Size of Input Instance Size of paper # of bits # of digits Value - n = 2 in 2 - n = 17 bits - n = 5 digits - n = 83920 Intuitive Formal 2’’ 83920 5 1’’

112. Size of Input Instance Size of paper # of bits # of digits Value - n = 2 in 2 - n = 17 bits - n = 5 digits - n = 83920 Intuitive Formal Reasonable # of bits = 3.32 * # of digits 2’’ 83920 5 1’’

113. Size of Input Instance Size of paper # of bits # of digits Value - n = 2 in 2 - n = 17 bits - n = 5 digits - n = 83920 Intuitive Formal Reasonable Unreasonable # of bits = log 2 (Value) Value = 2 # of bits 2’’ 83920 5 1’’

114. Two Example Algorithms Sum N array entries: A(1) + A(2) + A(3) + … Factor value N: Input N=5917 & Output N=97*61. Algorithm N/2, N/3, N/4, …. ? Time?

115. Two Example Algorithms Sum N array entries: A(1) + A(2) + A(3) + … Factor value N: Input N=5917 & Output N=97*61. Algorithm N/2, N/3, N/4, …. ? Time = N

116. Two Example Algorithms Sum N array entries: A(1) + A(2) + A(3) + … Factor value N: Input N=5917 & Output N=97*61. Algorithm N/2, N/3, N/4, …. ? Is this reasonable? Time = N

117. Two Example Algorithms Sum N array entries: A(1) + A(2) + A(3) + … Factor value N: Input N=5917 & Output N=97*61. Algorithm N/2, N/3, N/4, …. ? No! Time = N One is considered fast and the other slow!

118. Two Example Algorithms Sum N array entries: A(1) + A(2) + A(3) + … Factor value N: Input N=5917 & Output N=97*61. Algorithm N/2, N/3, N/4, …. ? Size of Input Instance?

119. Two Example Algorithms Sum N array entries: A(1) + A(2) + A(3) + … Factor value N: Input N=5917 & Output N=97*61. Algorithm N/2, N/3, N/4, …. ? size n = N size n = log N

120. Two Example Algorithms Sum N array entries: A(1) + A(2) + A(3) + … Factor value N: Input N=5917 & Output N=97*61. Algorithm N/2, N/3, N/4, …. ? Time as function of input size? size n = N size n = log N

121. Two Example Algorithms Sum N array entries: A(1) + A(2) + A(3) + … Factor value N: Input N=5917 & Output N=97*61. Algorithm N/2, N/3, N/4, …. ? Time = N = n Time = N = 2 n size n = N size n = log N

122. Two Example Algorithms Sum N array entries: A(1) + A(2) + A(3) + … Factor value N: Input N=5917 & Output N=97*61. Algorithm N/2, N/3, N/4, …. ? Linear vs Exponential Time! Time = N = n Time = N = 2 n size n = N size n = log N

123. Size of Input Instance? 14,23,25,30,31,52,62,79,88,98

124. Size of Input Instance # of elements - n = 10 elements 14,23,25,30,31,52,62,79,88,98 10

125. Size of Input Instance # of elements - n = 10 elements 14,23,25,30,31,52,62,79,88,98 10 Is this reasonable?

126. Size of Input Instance # of elements - n = 10 elements ~ Reasonable 14,23,25,30,31,52,62,79,88,98 10 If each element has size c # of bits = c * # of elements

127. Size of Input Instance # of elements - n = 10 elements Reasonable 14,23,25,30,31,52,62,79,88,98 10 If each element is in [1..n] each element has size log n # of bits = n log n ≈ n

128. Time Complexity Is a Function Specifies how the running time depends on the size of the input. A function mapping # of bits n needed to represent the input # of operations T(n) executed.

129. Which Input of size n? There are 2 n inputs of size n. Which do we consider for the time T(n)?

130. Which Input of size n? Typical Input Average Case Worst Case

131. Which Input of size n? Typical InputBut what is typical? Average CaseFor what distribution? Worst CaseTime for all inputs is bound. Easiest to compute

132. Time Complexity of Algorithm O(n 2 ): Prove that for every input of size n, the algorithm takes no more than cn 2 time. Ω(n 2 ): Find one input of size n, for which the algorithm takes at least this much time. θ (n 2 ): Do both. The time complexity of an algorithm is the largest time required on any input of size n.

133. Time Complexity of Problem O(n 2 ): Provide an algorithm that solves the problem in no more than this time. Ω(n 2 ): Prove that no algorithm can solve it faster. θ (n 2 ): Do both. The time complexity of a problem is the time complexity of the fastest algorithm that solves the problem.

134. Adding The Classic Techniques Evaluating ∑ i=1 f(i). n

135. Gauss ∑ i=1..n i = 1 + 2 + 3 +... + n = ? Arithmetic Sum

136. 1+2+3+...+ n-1+n=S n+n-1+n-2+... +2+1=S (n+1) + (n+1) + (n+1) +... + (n+1) + (n+1) = 2S n (n+1) = 2S

137. 1+2+3+...+ n-1+n=S n+n-1+n-2+... +2+1=S (n+1) + (n+1) + (n+1) +... + (n+1) + (n+1) = 2S n (n+1) = 2S

138. 1+2+3+...+ n-1+n=S n+n-1+n-2+... +2+1=S (n+1) + (n+1) + (n+1) +... + (n+1) + (n+1) = 2S n (n+1) = 2S

139. 1+2+3+...+ n-1+n=S n+n-1+n-2+... +2+1=S (n+1) + (n+1) + (n+1) +... + (n+1) + (n+1) = 2S n (n+1) = 2S

140. Let’s restate this argument using a geometric representation Algebraic argument

141. 1 2........ n = number of white dots.

142. 1 2........ n = number of white dots = number of yellow dots n....... 2 1

143. n+1 n+1 n+1 n+1 n+1 = number of white dots = number of yellow dots n n n n n n There are n(n+1) dots in the grid

144. n+1 n+1 n+1 n+1 n+1 = number of white dots = number of yellow dots n n n n n n Note =  (# of terms · last term))

145. Gauss ∑ i=1..n i = 1 + 2 + 3 +... + n =  (# of terms · last term) Arithmetic Sum True when ever terms increase slowly

146. ∑ i=0..n 2 i = 1 + 2 + 4 + 8 +... + 2 n = ? Geometric Sum

148. ∑ i=0..n 2 i = 1 + 2 + 4 + 8 +... + 2 n = 2 · last term - 1 Geometric Sum

149. ∑ i=0..n r i = r 0 + r 1 + r 2 +... + r n = ? Geometric Sum

150. S Srrrr...rr S 1r S r1 r1 23nn1 n1 n1  1rrr... 23 r n          Geometric Sum

151. ∑ i=0..n r i r1 r1 n1     Geometric Sum When r>1?

152. ∑ i=0..n r i r1 r1 n1     Geometric Sum θ(rn)θ(rn) Biggest Term When r>1

153. ∑ i=0..n r i = r 0 + r 1 + r 2 +... + r n =  (biggest term) Geometric Increasing True when ever terms increase quickly

154. ∑ i=0..n r i  Geometric Sum When r<1? 1 r n1   

155. ∑ i=0..n r i  Geometric Sum 1 r n1     θ(1) When r<1 Biggest Term

156. ∑ i=0..n r i = r 0 + r 1 + r 2 +... + r n =  (1) Bounded Tail True when ever terms decrease quickly

157. ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + …+ 1 / n = ? Harmonic Sum

158. f(i) = 1 ∑ i=1..n f(i) = n n Sum of Shrinking Function

159. f(i) = 1/2 i ∑ i=1..n f(i) = 2  Sum of Shrinking Function

160. f(i) = 1/i ∑ i=1..n f(i) = ? n Sum of Shrinking Function

161. Harmonic Sum

162. ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + …+ 1 / n =  (log(n)) Harmonic Sum

163. Approximating Sum by Integrating The area under the curve approximates the sum ∑ i=1..n f(i) ≈ ∫ x=1..n f(x) dx

164. Approximating Sum by Integrating ∫ x=1..n x c dx = 1 / c+1 n c+1 ∑ i=1..n i c ≈ Arithmetic Sums  θ(# of terms · last term) True when ever terms increase slowly = θ(n c+1 )

165. Approximating Sum by Integrating ∫ x=1..n b x dx = 1 / ln(b) b n ∑ i=1..n b i ≈ Geometric Sums  θ(last term) True when ever terms increase quickly = θ(b n )

166. Harmonic Sum Approximating Sum by Integrating

167. Problem: Integrating may be hard too.

168. Outline II) Math proofs of sums V) Ability to know the answer fast I) What is the sum Eg. ∑ i=1..n f(i) = ∑ i=1..n 1 / i 0.9999 =θ(n 0.0001 ) III) Pattern of sums ∑ i=1..n f(i) = θ(n · f(n)) IV) Intuition why this is the sum I flash it up If you follow great. If not don’t panic.

169. Adding Made Easy We will now classify (most) functions f(i) into four classes: –Geometric Like –Arithmetic Like –Harmonic –Bounded Tail

170. Adding Made Easy We will now classify (most) functions f(i) into four classes For each class, we will give an easy rule for approximating it’s sum θ( ∑ i=1..n f(i) )

171. Classifying Animals Vertebrates Birds Mammals Reptiles Fish Dogs Giraffe Mammal  Complex social networks Dog  Man’s best friend

172. Classifying Functions Functions Poly. Exp. 2 θ(n)  ∑ i=1..n f(i) = θ(f(n)) 8 · 2 n / n 100 + n 3 θ(2 n / n 100 ) 8 · 2 n + n 3 n θ(1) 2 θ(n) θ(2n)θ(2n) f(n) = 8 · 2 n / n 100 + n 3 θ(2 n / n 100 )  ∑ i=1..n f(i) = θ(2 n / n 100 ) 2 0.5n << 2 n 8 · 2 n / n 100 + n 3 Significant Less significant Irrelevant

173. Adding Made Easy 11 1 2 3 4 Four Classes of Functions

174. Adding Made Easy

175. If the terms f(i) grow sufficiently quickly, then the sum will be dominated by the largest term. Silly example: 1+2+3+4+5+ 1,000,000,000 ≈ 1,000,000,000 f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

176. If the terms f(i) grow sufficiently quickly, then the sum will be dominated by the largest term. Classic example: ? f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

177. If the terms f(i) grow sufficiently quickly, then the sum will be dominated by the largest term. Classic example: ∑ i=1..n 2 i = 2 n+1 -1 ≈ 2 f(n) f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

178. If the terms f(i) grow sufficiently quickly, then the sum will be dominated by the largest term. For which functions f(i) is this true? How fast and how slow can it grow? f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

179. r1 r1 n1      θ( ) r n Last Term when r>1. ∑ i=1..n r i 1rrr... 23 r n    θ(f(n)) ∑ i=1..n (1.0001) i ≈ 10,000 (1.0001) n = 10,000 f(n) Lower Extreme: Upper Extreme: ∑ i=1..n (1000) i ≈ 1.001(1000) n = 1.001 f(n) Recall, when f(n) = r i = 2 θ(n) f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

180. Because the constant is sooo big, the statement is just barely true. ∑ i=1..n (1.0001) i ≈ 10,000 (1.0001) n = 10,000 f(n) Lower Extreme: Upper Extreme: ∑ i=1..n (1000) i ≈ 1.001(1000) n = 1.001 f(n) f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

181. ∑ i=1..n (1.0001) i ≈ 10,000 (1.0001) n = 10,000 f(n) Lower Extreme: Upper Extreme: ∑ i=1..n (1000) i ≈ 1.001(1000) n = 1.001 f(n) Even bigger? f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

182. 2n2n 2i2i ∑ i=1..n 2 2 ≈ 2 2 = 1f(n) No Upper Extreme: Even bigger? ∑ i=1..n (1.0001) i ≈ 10,000 (1.0001) n = 10,000 f(n) Lower Extreme: f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

183. 2n2n 2i2i ∑ i=1..n 2 2 ≈ 2 2 = 1f(n) No Upper Extreme: Functions in between? ∑ i=1..n (1.0001) i ≈ 10,000 (1.0001) n = 10,000 f(n) Lower Extreme: f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

184. ∑ i=1..n 2 i = 2 · 2 n –2 = 2 1 + 2 2 + 2 3 + 2 4 +…+ 2 n 8 · [ ] 8·8· 8·8· 8·8· 8·8· 8·8· 8·8· = θ( 2 n ) 1 100 2 100 3 100 4 100 n 100 i 100 n 100 + i 3 + 1 3 + 2 3 + 3 3 + 4 3 + n 3 Dominated by the largest term. = θ(f(n)) f(n) = 2 n 8·8· n 100 + n 3 = θ( 2 n ) n 100 f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like: n 100 ≈

185. f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like: f(n) = 2 n 8·8· n 100 ∑ i=1..n f(i) f(n)  c · f(n) f(i)  · f(n) f(i)  · f(i +1 ) EasyHard (i +1 ) 100 i 100 2 =  (1 + 1 / i ) 100 1 2 1.9 = 8 · 2 (i+1) (i +1 ) 100 f(i +1 ) f(i) 8·2i8·2i i 100 =  1/.51  · f(i +1 ) f(i)

186. f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like: f(n) = 2 n 8·8· n 100 ∑ i=1..n f(i) f(n)  c · f(n) f(i)  · f(n)   · f(i +2 )  · f(i +1 ) f(i)  · f(i +1 ) f(i)   · f(i +3 ) .51 (n-i) ·f(i +(n-i) ) .51 (n-i) ·f( n ) f(i) .51 (n-i) ·f( n ) Eg. i = n, f(n) .51 (n-n) ·f( n )  1·f( n ) i = 0, f(0) .51 n ·f( n )

187. f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like: f(n) = 2 n 8·8· n 100 ∑ i=1..n f(i) f(n)  c · f(n) ∑ i=1..n f(i)  ∑ i=1..n.51 (n-i) ·f( n ) = θ(1) · f(n) f(i)  · f(n)  · f(i +1 ) f(i) f(i) .51 (n-i) ·f( n ) = [∑ i=1..n.51 (n-i) ] · f( n )

188. Functions Poly. Exp. 2 θ(n)  ∑ i=1..n f(i) = θ(f(n)) 8 · 2 n / n 100 + n 3 θ(2 n / n 100 ) 8 · 2 n + n 3 n θ(1) 2 θ(n) θ(2n)θ(2n) f(n) = 8 · 2 n / n 100 + n 3 θ(2 n / n 100 )  ∑ i=1..n f(i) = θ(2 n / n 100 ) 2 0.5n << 2 n 8 · 2 n / n 100 + n 3 Significant Less significant Irrelevant f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

189. If f(n) = c b an n d log e (n) c  ? a  ? b  ? d  ? e  ? f(n) = Ω, θ ? then ∑ i=1..n f(i) = θ(f(n)).  2 Ω(n) > 0  0  1 (- ,  ) f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

190. f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

191. All functions in 2 Ω(n) ? Maybe not. f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

192. Functions that oscillate continually do not have the property. f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

193. Functions expressed with +, -, , , exp, log do not oscillate continually. They are well behaved for sufficiently large n. These do have the property. f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

194. Adding Made Easy Done

195. Adding Made Easy

196. If most of the terms f(i) have roughly the same value, then the sum is roughly the number of terms, n, times this value. Silly example: 1,001 + 1,002 + 1,003 + 1,004 + 1,005 ≈ 5 · 1,000 f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

197. If most of the terms f(i) have roughly the same value, then the sum is roughly the number of terms, n, times this value. Another silly example: ∑ i=1..n 1 = n · 1 f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

198. If most of the terms f(i) have roughly the same value, then the sum is roughly the number of terms, n, times this value. Is the statement true for this function? ∑ i=1..n i = 1 + 2 + 3 +... + n f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

199. If most of the terms f(i) have roughly the same value, then the sum is roughly the number of terms, n, times this value. Is the statement true for this function? ∑ i=1..n i = 1 + 2 + 3 +... + n The terms are not roughly the same. f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

200. But half the terms are roughly the same. ∑ i=1..n i = 1 +... + n/2 +... + n f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

201. But half the terms are roughly the same and the sum is roughly the number terms, n, times this value ∑ i=1..n i = 1 +... + n/2 +... + n ∑ i=1..n i = θ(n · n) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

202. Is the statement true for this function? ∑ i=1..n i 2 = 1 2 + 2 2 + 3 2 +... + n 2 Even though, the terms are not roughly the same. f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

203. Again half the terms are roughly the same. ∑ i=1..n i = 1 2 +... + (n/2) 2 +... + n 2 1 / 4 n 2 f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

204. Again half the terms are roughly the same. ∑ i=1..n i = 1 2 +... + (n/2) 2 +... + n 2 1 / 4 n 2 ∑ i=1..n i 2 = θ(n · n 2 ) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

205. ∑ i=1..n f(i) ≈ area under curve f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

206. area of small square  ∑ i=1..n f(i) ≈ area under curve  area of big square f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

207. n / 2 · f( n / 2 ) = area of small square  ∑ i=1..n f(i) ≈ area under curve  area of big square = n · f(n) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

208. θ(n · f(n)) = n / 2 · f( n / 2 ) = area of small square  ∑ i=1..n f(i) ≈ area under curve  area of big square = n · f(n) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

209. θ(n · f(n)) = n / 2 · f( n / 2 ) = area of small square  ∑ i=1..n f(i) ≈ area under curve  area of big square = n · f(n) ? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

210. ∑ i=1..n i 2 = 1 2 + 2 2 + 3 2 +... + n 2 f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like: f(i) = n 2 f( n / 2 ) = θ(f(n)) The key property is = ?

211. The key property is f( n / 2 ) = (n/2) 2 = 1 / 4 n 2 = θ( n 2 ) = θ(f(n)) ∑ i=1..n i 2 = 1 2 + 2 2 + 3 2 +... + n 2 f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like: f(i) = n 2  = θ(n·f(n)) = θ(n · n 2 )

212. ∑ i=1..n f(i) = ? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like: f(i) = n θ(1)

213. ∑ i=1..n i r = 1 r + 2 r + 3 r +... + n r f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like: f(i) = n θ(1) f( n / 2 ) = θ(f(n)) The key property is

214. f( n / 2 ) = (n/2) r = 1 / 2 r n r = θ( n r ) = θ(f(n)) ∑ i=1..n i r = 1 r + 2 r + 3 r +... + n r f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:  = θ(n·f(n)) = θ(n · n r ) f(i) = n θ(1)

215. ∑ i=1..n 2 i = 2 1 + 2 2 + 2 3 +... + 2 n f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like: f(i) = n θ(1) f( n / 2 ) = θ(f(n)) The key property is

216. ∑ i=1..n 2 i = 2 1 + 2 2 + 2 3 +... + 2 n f( n / 2 ) = 2 (n/2) = θ( 2 n ) = θ(f(n)) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like: f(i) = n θ(1)  = θ(n·f(n)) = θ(n · 2 n )

217. ∑ i=1..n 1 = n · 1 = n · f(n) Middle Extreme: Upper Extreme: ∑ i=1..n i 1000 = 1 / 1001 n 1001 = 1 / 1001 n · f(n) All functions in between. f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

218. Adding Made Easy Half done

219. f(i) = 1 ∑ i=1..n f(i) = n n Sum of Shrinking Function

220. f(i) = ? ∑ i=1..n f(i) = n 1/2 n Sum of Shrinking Function 1/i 1/2

221. f(i) = 1/i ∑ i=1..n f(i) = log n n Sum of Shrinking Function

222. f(i) = 1/2 i ∑ i=1..n f(i) = 2 Sum of Shrinking Function 

223. If most of the terms f(i) have roughly the same value, then the sum is roughly the number of terms, n, times this value. Does the statement hold for functions f(i) that shrink? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

224. If most of the terms f(i) have roughly the same value, then the sum is roughly the number of terms, n, times this value. ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + … + 1 / n Does the statement hold for the Harmonic Sum? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

225. Does the statement hold for the Harmonic Sum? ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + … + 1 / n ∑ i=1..n f(i) = ? θ(n · f(n)) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

226. Does the statement hold for the Harmonic Sum? ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + … + 1 / n ∑ i=1..n f(i) θ(n · f(n)) = ∑ i=1..n 1 / i = ? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

227. Does the statement hold for the Harmonic Sum? ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + … + 1 / n ∑ i=1..n f(i) θ(n · f(n)) = ∑ i=1..n 1 / i =θ(log n) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

228. Does the statement hold for the Harmonic Sum? ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + … + 1 / n ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i =θ(log n) ? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

229. Does the statement hold for the Harmonic Sum? ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + … + 1 / n ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i =θ(log n) θ(1) = θ(n · 1 / n ) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

230. Does the statement hold for the Harmonic Sum? ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + … + 1 / n ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i =θ(log n) θ(1) = θ(n · 1 / n ) ≠ No the statement does not hold! f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

231. Adding Made Easy not included

232. Does the statement hold for the almost Harmonic Sum? ∑ i=1..n 1 / i 0.9999 Shrinks slightly slower than harmonic. f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

233. Does the statement hold for the almost Harmonic Sum? ∑ i=1..n 1 / i 0.9999 ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i 0.9999 = ? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

234. Does the statement hold for the almost Harmonic Sum? ∑ i=1..n 1 / i 0.9999 ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i 0.9999 =θ(n 0.0001 ) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

235. Does the statement hold for the almost Harmonic Sum? ∑ i=1..n 1 / i 0.9999 ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i 0.9999 =θ(n 0.0001 ) ? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

236. Does the statement hold for the almost Harmonic Sum? ∑ i=1..n 1 / i 0.9999 ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i 0.9999 =θ(n 0.0001 ) θ(n 0.0001 ) = θ(n · 1 / n 0.9999 ) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

237. Does the statement hold for the almost Harmonic Sum? ∑ i=1..n 1 / i 0.9999 ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i 0.9999 =θ(n 0.0001 ) θ(n 0.0001 ) = θ(n · 1 / n 0.9999 ) = The statement does hold! f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

238. ∑ i=1..n 1 = n · 1 = n · f(n) Middle Extreme: Lower Extreme: ∑ i=1..n 1 / i 0.9999 = θ(n 0.0001 ) = θ(n · f(n)) All functions in between. f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

239. ∑ i=1..n 1 = n · 1 = n · f(n) Middle Extreme: Lower Extreme: ∑ i=1..n 1 / i 0.9999 = θ(n 0.0001 ) = θ(n · f(n)) Upper Extreme: ∑ i=1..n i 1000 = 1 / 1001 n 1001 = 1 / 1001 n · f(n) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

240. Arithmetic Like If f(n) = c b an n d log e (n) c  ? a  ? b  ? d  ? e  ? f(n) = Ω, θ ? then ∑ i=1..n f(i) = θ(n · f(n)). n -1+θ(1) > 0  0 or b  1 > -1 (- ,  ) (For +, -, , , exp, log functions f(n)) Conclusion

241. Adding Made Easy Done

242. Adding Made Easy Harmonic

243. Harmonic Sum ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + …+ 1 / n =  (log(n))

244. Adding Made Easy

245. If the terms f(i) decrease towards zero sufficiently quickly, then the sum will be a constant. The classic example ∑ i=0..n 1 / 2 i = 1 + 1 / 2 + 1 / 4 + 1 / 8 + … = 2. f(n)  n -1-Ω(1)  ∑ i=1..n f(i) = θ(1) Bounded Tail:

246. If f(i) decays even faster, then the tail of the sum is more bounded. 2i2i ∑ i=1..n 2 2 = θ(1). 1 f(n)  n -1-Ω(1)  ∑ i=1..n f(i) = θ(1) Bounded Tail:

247. Upper Extreme: ∑ i=1..n 1 / i 1.0001 = θ(1) f(n)  n -1-Ω(1)  ∑ i=1..n f(i) = θ(1) Bounded Tail:

248. Upper Extreme: ∑ i=1..n 1 / i 1.0001 = θ(1) No Lower Extreme: 2i2i ∑ i=1..n 2 2 = θ(1). 1 All functions in between. f(n)  n -1-Ω(1)  ∑ i=1..n f(i) = θ(1) Bounded Tail:

249. Bounded Tail If f(n) = c b an n d log e (n) c  ? a  ? b  ? d  ? e  ? f(n) = Ω, θ ? then ∑ i=1..n f(i) = θ(1). > 0  (- ,  ) Conclusion or b  1 c n d log e (n) c  a  b  b an

250. Bounded Tail If f(n) = c n d log e (n) c  ? a  b = 1 d  ? e  ? f(n) = Ω, θ ? then ∑ i=1..n f(i) = θ(1).  n θ(1)-1 > 0 < -1 (- ,  ) Conclusion

251. Adding Made Easy Done

252. Adding Made Easy Missing Functions 1 / nlogn logn / n

253. Adding Made Easy Geometric Like: If f(n)  2 Ω(n), then ∑ i=1..n f(i) = θ(f(n)). Arithmetic Like: If f(n) = n θ(1)-1, then ∑ i=1..n f(i) = θ(n · f(n)). Harmonic: If f(n) = 1 / n, then ∑ i=1..n f(i) = log e n + θ(1). Bounded Tail: If f(n)  n -1-Ω(1), then ∑ i=1..n f(i) = θ(1). ( For +, -, , , exp, log functions f(n) ) This may seem confusing, but it is really not. It should help you compute most sums easily.

254. Recurrence Relations T(1) = 1 T(n) = a T(n/b) + f(n)

255. Recurrence Relations  Time of Recursive Program procedure Eg(int n) if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a Eg(n/b) Recurrence relations arise from the timing of recursive programs. Let T(n) be the # of “Hi”s on an input of “size” n.

256. procedure Eg(int n) if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a Eg(n/b) Given size 1, the program outputs T(1)=1 Hi’s. Recurrence Relations  Time of Recursive Program

257. procedure Eg(int n) if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a Eg(n/b) Given size n, the stackframe outputs f(n) Hi’s. Recurrence Relations  Time of Recursive Program

258. procedure Eg(int n) if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a Eg(n/b) Recursing on a instances of size n/b generates T(n/b) “Hi”s. Recurrence Relations  Time of Recursive Program

259. procedure Eg(int n) if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a Eg(n/b) Recursing a times generates a·T(n/b) “Hi”s. Recurrence Relations  Time of Recursive Program

260. procedure Eg(int n) if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a Eg(n/b) For a total of T(1) = 1 T(n) = a·T(n/b) + f(n) “Hi”s. Recurrence Relations  Time of Recursive Program

261. X = 2133 Y = 2312 ac = bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames Stack Frame: A particular execution of one routine on one particular input instance.

262. X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

263. X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

264. X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = ? bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

265. X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = ? bd = (a+b)(c+d) = XY = X = 2 Y = 2 XY= MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

266. X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = ? bd = (a+b)(c+d) = XY = X = 2 Y = 2 XY = 4 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

267. X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

268. X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = ? (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

269. X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = ? (a+b)(c+d) = XY = X = 1 Y = 3 XY = 3 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

270. X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

271. X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = ? XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

272. X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = ? XY = X = 3 Y = 5 XY = 15 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

273. X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = ? MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

274. X = 2133 Y = 2312 ac = ? bd = (a+b)(c+d) = XY = X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

275. X = 2133 Y = 2312 ac = 483 bd = (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

276. X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

277. X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = ? bd = (a+b)(c+d) = XY = 15 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

278. X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = ? bd = (a+b)(c+d) = XY = 15 X = 3 Y = 1 XY = 3 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

279. X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = ? (a+b)(c+d) = XY = 15 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

280. X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = ? (a+b)(c+d) = XY = 15 X = 3 Y = 2 XY = 6 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

281. X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = ? XY = 15 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames

282. X = 2133 Y = 2312 ac = 483 bd = ? (a+b)(c+d) = XY = X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = ? XY = 396 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Stack of Stack Frames An so on ….

283. X = 2133 Y = 2312 ac = 483 bd = 396 (a+b)(c+d) = 1890 XY = 4931496 X = 21 Y = 23 ac = 4 bd = 3 (a+b)(c+d) = 15 XY = 483 X = 33 Y = 12 ac = 3 bd = 6 (a+b)(c+d) = 18 XY = 396 X = 54 Y = 35 ac = 15 bd = 20 (a+b)(c+d) = 72 XY = 1890 X = 2 Y = 2 XY=4 X = 1 Y = 3 XY=3 X = 3 Y = 5 XY=15 X = 3 Y = 1 XY=3 X = 3 Y = 2 XY=6 X = 6 Y = 3 XY=18 X = 5 Y = 3 XY=15 X = 4 Y = 5 XY=20 X = 9 Y = 8 XY=72 MULT(X,Y): If |X| = |Y| = 1 then RETURN XY Break X into a;b and Y into c;d e = MULT(a,c) and f =MULT(b,d) RETURN e 10 n + (MULT(a+b, c+d) – e - f) 10 n/2 + f Tree of Stack Frames

284. Solving Technique 1 Guess and Verify Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: G(n) = 2n 2 – n Verify: Left Hand SideRight Hand Side T(1) = 2(1) 2 – 1 T(n) = 2n 2 – n 1 4T(n/2) + n = 4 [2( n / 2 ) 2 – ( n / 2 )] + n = 2n 2 – n

285. Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: G(n) = an 2 + bn + c Verify: Left Hand SideRight Hand Side T(1) = a+b+c T(n) = an 2 +bn+c 1 4T(n/2) + n = 4 [a ( n / 2 ) 2 + b ( n / 2 ) +c] + n = an 2 +(2b+1)n + 4c Solving Technique 2 Guess Form and Calculate Coefficients

286. Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: G(n) = an 2 + bn + 0 Verify: Left Hand SideRight Hand Side T(1) = a+b+c T(n) = an 2 +bn+c 1 4T(n/2) + n = 4 [a ( n / 2 ) 2 + b ( n / 2 ) +c] + n = an 2 +(2b+1)n + 4c Solving Technique 2 Guess Form and Calculate Coefficients c=4c c=0

287. Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: G(n) = an 2 - 1n + 0 Verify: Left Hand SideRight Hand Side T(1) = a+b+c T(n) = an 2 +bn+c 1 4T(n/2) + n = 4 [a ( n / 2 ) 2 + b ( n / 2 ) +c] + n = an 2 +(2b+1)n + 4c Solving Technique 2 Guess Form and Calculate Coefficients b=2b+1 b=-1

288. Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: G(n) = an 2 - 1n + 0 Verify: Left Hand SideRight Hand Side T(1) = a+b+c T(n) = an 2 +bn+c 1 4T(n/2) + n = 4 [a ( n / 2 ) 2 + b ( n / 2 ) +c] + n = an 2 +(2b+1)n + 4c Solving Technique 2 Guess Form and Calculate Coefficients a=a

289. Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: G(n) = 2n 2 - 1n + 0 Verify: Left Hand SideRight Hand Side T(1) = a+b+c T(n) = an 2 +bn+c 1 4T(n/2) + n = 4 [a ( n / 2 ) 2 + b ( n / 2 ) +c] + n = an 2 +(2b+1)n + 4c Solving Technique 2 Guess Form and Calculate Coefficients a+b+c=1 a-1+0=1 a=2

290. Recurrence Relation: T(1) = 1 & T(n) = aT(n/b) + f(n) Solving Technique 3 Approximate Form and Calculate Exponent which is bigger? Guess

291. Recurrence Relation: T(1) = 1 & T(n) = aT(n/b) + f(n) Guess: aT(n/b) << f(n) Simplify: T(n)  f(n) Solving Technique 3 Calculate Exponent In this case, the answer is easy. T(n) =  (f(n))

292. Recurrence Relation: T(1) = 1 & T(n) = aT(n/b) + f(n) Guess: aT(n/b) >> f(n) Simplify: T(n)  aT(n/b) Solving Technique 3 Calculate Exponent In this case, the answer is harder.

293. Recurrence Relation: T(1) = 1 & T(n) = aT(n/b) Guess: G(n) = cn  Verify: Left Hand SideRight Hand Side T(n) = cn  aT(n/b) = a [c ( n / b )  ] = c a b  n  Solving Technique 3 Calculate Exponent ( log a / log b ) = cn 1 = a b -  b  = a  log b = log a  = log a / log b

294. Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) Guess: G(n) = cn  Verify: Left Hand SideRight Hand Side T(n) = cn  aT(n/b) + f(n) = c [a ( n / b )  ] = c a b  n  Solving Technique 3 Calculate Exponent 1 = a b -  b  = a  log b = log a  = log a / log b ( log a / log b ) = cn ( log 4 / log 2 ) = cn 2

295. Recurrence Relation: T(1) = 1 & T(n) = aT(n/b) + f(n) Solving Technique 3 Calculate Exponent If bigger then T(n) =  (f(n)) If bigger then ( log a / log b ) T(n) =  (n ) And if aT(n/b)  f(n) what is T(n) then?

296. Technique 4: Decorate The Tree T(n) = a T(n/b) + f(n) f(n) T(n/b) T(n) = T(1) T(1) = 1 1 = a

297. f(n) T(n/b) T(n) = a

298. f(n) T(n/b) T(n) = a f(n/b) T(n/b 2 ) a

299. f(n) T(n) = a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a

300. 11111111111111111111111111111111...... 111111111111111111111111111111111 f(n) T(n) = a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a

301. Evaluating: T(n) = aT(n/b)+f(n) Level 0 1 2 i h

302. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0 1 2 i h

303. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 2 i h

304. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 i h

305. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i h

306. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i n/b i h n/b h

307. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i n/b i h n/b h

308. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i n/b i h n/b h = 1 base case

309. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i n/b i h n/b h = 1

310. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i n/b i h = log n / log b n/b h = 1 b h = n h log b = log n h = log n / log b

311. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame 0n 1 n/b 2 n/b 2 i n/b i h = log n / log b 1

312. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame 0n f(n) 1 n/bf(n/b) 2 n/b 2 f(n/b 2 ) i n/b i f(n/b i ) h = log n / log b 1T(1)

313. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames 0n f(n) 1 n/bf(n/b) 2 n/b 2 f(n/b 2 ) i n/b i f(n/b i ) h = log n / log b n/b h T(1)

314. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames 0n f(n) 1 1 n/bf(n/b) a 2 n/b 2 f(n/b 2 ) a2a2 i n/b i f(n/b i ) aiai h = log n / log b n/b h T(1) ahah

315. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames 0n f(n) 1 1 n/bf(n/b) a 2 n/b 2 f(n/b 2 ) a2a2 i n/b i f(n/b i ) aiai h = log n / log b n/b h T(1) ahah

316. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames 0n f(n) 1 1 n/bf(n/b) a 2 n/b 2 f(n/b 2 ) a2a2 i n/b i f(n/b i ) aiai h = log n / log b n/b h T(1) ahah a h = a = n log n / log b log a / log b

317. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames Work in Level 0n f(n) 1 1 n/bf(n/b) a 2 n/b 2 f(n/b 2 ) a2a2 i n/b i f(n/b i ) aiai h = log n / log b n/b h T(1) n log a / log b

318. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames Work in Level 0n f(n) 1 1 · f(n) 1 n/bf(n/b) a a · f(n/b) 2 n/b 2 f(n/b 2 ) a2a2 a 2 · f(n/b 2 ) i n/b i f(n/b i ) aiai a i · f(n/b i ) h = log n / log b n/b h T(1) n log a / log b n · T(1) log a / log b Total Work T(n) = ∑ i=0..h a i  f(n/b i )

319. Evaluating: T(n) = aT(n/b)+f(n) = ∑ i=0..h a i  f(n/b i ) If a Geometric Sum ∑ i=0..n x i  θ(max(first term, last term))

320. Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames Work in Level 0n f(n) 1 1 · f(n) 1 n/bf(n/b) a a · f(n/b) 2 n/b 2 f(n/b 2 ) a2a2 a 2 · f(n/b 2 ) i n/b i f(n/b i ) aiai a i · f(n/b i ) h = log n / log b n/b h T(1) n log a / log b n · T(1) log a / log b Dominated by Top Level or Base Cases

321. Evaluating: T(n) = aT(n/b)+f(n) Is the sum Geometric? Simplify by letting f(n) = n c log k n T(n) = ∑ i=0..h a i  f(n/b i ) = ∑ i=0..h a i  (n/b i ) c  log k (n/b i ) = n c ∑ i=0..h 2 [log a - c log b] · i  log k (n/b i ) Geometrically IncreasingArithmetic SumGeometrically Decreasing Sum = θ(last term) = θ( ) Sum = θ(any term  # terms) = θ(f(n)  log n) Sum = θ(first term) = θ(f(n)). log a - c log b < 0log a - c log b = 0log a - c log b < 0 c < log a / log b c = log a / log b & k<-1 c = log a / log b & k>-1 c > log a / log b n log a / log b (n/..) c = n c a i = 2 [log a]·i (.. /b i ) c = 2 [-c log b]·i = n c ∑ i=0..h 2 -d i  log k (n/b i )

322. Evaluating: T(n) = aT(n/b)+f(n) Is the sum Geometric? Simplify by letting f(n) = n c log k n T(n) = ∑ i=0..h a i  f(n/b i ) = ∑ i=0..h a i  (n/b i ) c  log k (n/b i ) = n c ∑ i=0..h 2 [log a - c log b] · i  log k (n/b i ) Geometrically IncreasingArithmetic SumGeometrically Decreasing Sum = θ(last term) = θ( ) Sum = θ(any term  # terms) = θ(f(n)  log n) Sum = θ(first term) = θ(f(n)). log a - c log b < 0log a - c log b = 0log a - c log b < 0 c < log a / log b c = log a / log b & k<-1 c = log a / log b & k>-1 c > log a / log b n log a / log b k =-1  log k ( n /b i ) = 1 / (logn - i  logb)  1 / i (backwards) = n c ∑ i=0..h 2 0 i  log k (n/b i ) (back wards) de

323. Evaluating: T(n) = aT(n/b)+f(n) Is the sum Geometric? Simplify by letting f(n) = n c log k n T(n) = ∑ i=0..h a i  f(n/b i ) = ∑ i=0..h a i  (n/b i ) c  log k (n/b i ) = n c ∑ i=0..h 2 [log a - c log b] · i  log k (n/b i ) Geometrically IncreasingArithmetic SumGeometrically Decreasing Sum = θ(last term) = θ( ) Sum = θ(any term  # terms) = θ(f(n)  log n) Sum = θ(first term) = θ(f(n)). log a - c log b < 0log a - c log b = 0log a - c log b < 0 c < log a / log b c = log a / log b & k<-1 c = log a / log b & k>-1 c > log a / log b n log a / log b = n c ∑ i=0..h 2 +d i  log k (n/b i )

324. Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n

325. Time for top level?: Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n

326. Time for top level: f(n) = n 3 / log 5 n Time for base cases?: Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n

327. Time for top level: f(n) = n 3 / log 5 n Time for base cases: Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n θ(n ) = log a / log b θ(n ) log ? / log ?

328. Time for top level: f(n) = n 3 / log 5 n Time for base cases: Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n θ(n ) = log a / log b θ(n ) = ? log 4 / log 2

329. Time for top level: f(n) = n 3 / log 5 n Time for base cases: Dominated?: c = ? Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 log a / log b = ?

330. Time for top level: f(n) = n 3 / log 5 n Time for base cases: Dominated?: c = 3 > 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = ?

331. Time for top level: f(n) = n 3 / log 5 n Time for base cases: Dominated?: c = 3 > 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = θ(top) = θ(f(n)) = θ(n 3 / log 5 n ).

332. Time for top level: f(n) = 2 n Time for base cases: Dominated?: c = ? > 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ 2 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = θ(top) = θ(f(n)) = θ(2 n ). bigger >big The time is even more dominated by the top level of recursion

333. Time for top level: f(n) = n log 5 n Time for base cases: Dominated?: c = ? 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = ? 1< = θ(base cases) = θ(n ) = θ(n 2 ). log a / log b

334. Time for top level: f(n) = log 5 n Time for base cases: Dominated?: c = ? 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = ? 0< = θ(base cases) = θ(n ) = θ(n 2 ). log a / log b

335. Time for top level: f(n) = n 2 Time for base cases: Dominated?: c = ? 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 2 θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = ? 2= = θ(f(n) log(n) ) = θ(n 2 log(n) ).

336. Time for top level: f(n) = n 2 log 5 n Time for base cases: Dominated?: c = 2 = 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 2 log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 k = ? 5 ? Hence, T(n) = ? > -1 = θ(f(n) log(n) ) = θ(n 2 log 6 (n) ).

337. Time for top level: f(n) = n 2 / log 5 n Time for base cases: Dominated?: c = 2 = 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 2 / log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 k = ? Hence, T(n) = ? -5 ?< -1 = θ(base cases) = θ(n ) = θ(n 2 ). log a / log b

338. Time for top level: f(n) = n 2 / log n Time for base cases: Dominated?: c = 2 = 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 2 / log n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 k = ? Hence, T(n) = ? -1 ?= -1 = θ(n c loglog n ) = θ(n 2 loglog n ). Did not do before.

339. Sufficiently close to: T(n) = 4T(n/2)+ n 3 = θ(n 3 ). T 2 (n) = ? Evaluating: T 2 (n) = 4 T 2 (n/2+n 1/2 )+ n 3 θ(n 3 ).

340. Evaluating: T(n) = aT(n-b)+f(n) h = ? n-hb n-ib n-2b n-b T(0) f(n-ib) f(n-2b) f(n-b) f(n) n Work in Level # stack frames Work in stack frame Instance size a aiai a2a2 a 1 n/bn/b a · T(0) a i · f(n-ib) a 2 · f(n-2b) a · f(n-b) 1 · f(n) n/bn/b |base case| = 0 = n-hb h = n / b i 2 1 0 Level Likely dominated by base cases Exponential

341. Evaluating: T(n) = 1T(n-b)+f(n) h = ? Work in Level # stack frames Work in stack frame Instance size 1 1 1 1 1 f(0) f(n-ib) f(n-2b) f(n-b) f(n) n-b n n-hb n-ib n-2b T(0) f(n-ib) f(n-2b) f(n-b) f(n) h = n / b i 2 1 0 Level Total Work T(n) = ∑ i=0..h f(b·i) = θ(f(n)) or θ(n · f(n))

342. Math Review: Exponents X A X B = X A+B (not X AB !!) X A / X B = X A-B (X A ) B = X AB X A + X A = 2X A 2 A + 2 A = 2 A+1

343. Logarithms Definition: X A = B if and only if log X B = A (x is the “base” of the logarithm). –Example: 10 2 = 100 means log 10 100 = 2. Theorems log X AB = log X A + log X B log X A/B = log X A – log X B log X A B = B log X A

344. Logarithms… Theorem: log A B = log C B / log C A Proof: Let X = log C B, Y = log C A, and Z = log A B. By the definition of logarithm: C X = B, C Y = A, and A Z = B. Thus C X = B = A Z = C YZ, X = YZ, Z = X/Y.

345. Logarithms… The notation for logs can be confusing. There are two alternatives: –log 2 (x) = log (log x) or –log 2 (x) = (log x) 2 Generally, we use the 2 nd definition. Note: log 2 (x) is not (log x 2 ) Note: log is not a multiplicative entity, it is a function.

346. End

347. Restart Relevant Mathematics Iterative Algorithms