Presentation is loading. Please wait.

Presentation is loading. Please wait.

Relevant Mathematics Jeff Edmonds York University COSC 3101 Lecture skipped. Lecture skipped. (Done when needed) Logic Quantifiers The Time Complexity.

Similar presentations


Presentation on theme: "Relevant Mathematics Jeff Edmonds York University COSC 3101 Lecture skipped. Lecture skipped. (Done when needed) Logic Quantifiers The Time Complexity."— Presentation transcript:

1

2 Relevant Mathematics Jeff Edmonds York University COSC 3101 Lecture skipped. Lecture skipped. (Done when needed) Logic Quantifiers The Time Complexity of an Algorithm Classifying Functions Adding Made Easy Recurrence Relations In More Details

3 Some Math Time Complexity t(n) =  (n 2 ) Logic Quantifiers  g  b Loves(b,g)  b  g Loves(b,g) In Iterative Algs (GCD) See logic slides

4 Some Math Logs and Exps 2 a × 2 b = 2 a+b 2 log n = n You are on your own. Input Size Time Classifying Functions f(i) = n  (n) See logic slides

5 Some Math Recurrence Relations T(n) = a T(n/b) + f(n) Adding Made Easy ∑ i=1 f(i). In Iterative Algs (Insertion Sort) In Recursive Algs (Multiplying) In Recursive Algs (Multiplying, Towers of Hanoi, Merge Sort)

6 The Time Complexity of an Algorithm Specifies how the running time depends on the size of the input.

7 Purpose?

8 Purpose To estimate how long a program will run. To estimate the largest input that can reasonably be given to the program. To compare the efficiency of different algorithms. To help focus on the parts of code that are executed the largest number of times. To choose an algorithm for an application.

9 Time Complexity Is a Function Specifies how the running time depends on the size of the input. A function mapping “size” of input “time” T(n) executed.

10 Definition of Time?

11 Definition of Time # of seconds (machine dependent). # lines of code executed. # of times a specific operation is performed (e.g., addition). Which?

12 Definition of Time # of seconds (machine dependent). # lines of code executed. # of times a specific operation is performed (e.g., addition). These are all reasonable definitions of time, because they are within a constant factor of each other.

13 Size of Input Instance? 83920

14 Size of Input Instance Size of paper # of bits # of digits Value - n = 2 in 2 - n = 17 bits - n = 5 digits - n = 83920 2’’ 83920 5 1’’ Which are reasonable?

15 Size of Input Instance Size of paper # of bits # of digits Value - n = 2 in 2 - n = 17 bits - n = 5 digits - n = 83920 Intuitive 2’’ 83920 5 1’’

16 Size of Input Instance Size of paper # of bits # of digits Value - n = 2 in 2 - n = 17 bits - n = 5 digits - n = 83920 Intuitive Formal 2’’ 83920 5 1’’

17 Size of Input Instance Size of paper # of bits # of digits Value - n = 2 in 2 - n = 17 bits - n = 5 digits - n = 83920 Intuitive Formal Reasonable # of bits = 3.32 * # of digits 2’’ 83920 5 1’’

18 Size of Input Instance Size of paper # of bits # of digits Value - n = 2 in 2 - n = 17 bits - n = 5 digits - n = 83920 Intuitive Formal Reasonable Unreasonable # of bits = log 2 (Value) Value = 2 # of bits 2’’ 83920 5 1’’

19 Two Example Algorithms Sum N array entries: A(1) + A(2) + A(3) + … Factor value N: Input N=5917 & Output N=97*61. Algorithm N/2, N/3, N/4, …. ? Time? Time = N Is this reasonable? No! One is considered fast and the other slow!

20 Two Example Algorithms Sum N array entries: A(1) + A(2) + A(3) + … Factor value N: Input N=5917 & Output N=97*61. Algorithm N/2, N/3, N/4, …. ? Standard input size? Time = N N = hard drive = 60G < 2 36 N = crypto key = 2 100

21 Two Example Algorithms Sum N array entries: A(1) + A(2) + A(3) + … Factor value N: Input N=5917 & Output N=97*61. Algorithm N/2, N/3, N/4, …. ? Size of Input Instance? Time = N size n = N size n = log N

22 Two Example Algorithms Sum N array entries: A(1) + A(2) + A(3) + … Factor value N: Input N=5917 & Output N=97*61. Algorithm N/2, N/3, N/4, …. ? size n = N size n = log N Time as function of input size? Time for you to solve problem instance as a function of time for me to give you the problem instance. = n = 2 n Time = N

23 Two Example Algorithms Sum N array entries: A(1) + A(2) + A(3) + … Factor value N: Input N=5917 & Output N=97*61. Algorithm N/2, N/3, N/4, …. ? Linear vs Exponential Time! size n = N size n = log N = n = 2 n Time = N

24 Size of Input Instance? 14,23,25,30,31,52,62,79,88,98

25 Size of Input Instance # of elements = n = 10 elements 14,23,25,30,31,52,62,79,88,98 10

26 Size of Input Instance # of elements = n = 10 elements 14,23,25,30,31,52,62,79,88,98 10 Is this reasonable?

27 Size of Input Instance # of elements = n = 10 elements Reasonable 14,23,25,30,31,52,62,79,88,98 10 If each element has size c # of bits = c * # of elements

28 Size of Input Instance # of elements = n = 10 elements Reasonable 14,23,25,30,31,52,62,79,88,98 10 If each element is in [1..n] each element has size log n # of bits = n log n ≈ n ~

29 Time Complexity Is a Function Specifies how the running time depends on the size of the input. A function mapping # of bits n needed to represent the input # of operations T(n) executed.

30 Which Input of size n? There are 2 n inputs of size n. Which do we consider for the time T(n)?

31 Which Input of size n? Typical Input Average Case Worst Case

32 Which Input of size n? Typical InputBut what is typical? Average CaseFor what distribution? Worst CaseTime for all inputs is bound. Easiest to compute

33 What is the height of tallest person in the class? Bigger than this? Need to look at only one person Need to look at every person Smaller than this?

34 Time Complexity of Algorithm O(n 2 ): Prove that for every input of size n, the algorithm takes no more than cn 2 time. Ω(n 2 ): Find one input of size n, for which the algorithm takes at least this much time. θ (n 2 ): Do both. The time complexity of an algorithm is the largest time required on any input of size n.

35 Time Complexity of Problem O(n 2 ): Provide an algorithm that solves the problem in no more than this time. Ω(n 2 ): Prove that no algorithm can solve it faster. θ (n 2 ): Do both. The time complexity of a problem is the time complexity of the fastest algorithm that solves the problem.

36 Classifying Functions T(n)T(n) 101001,00010,000 log n 36913amoeba n 1/2 31031100bird 101001,00010,000human n log n 306009,000130,000my father n2n2 10010,00010 6 10 8 elephant n3n3 1,00010 6 10 9 10 12 dinosaur 2n2n 1,02410 30 10 300 10 3000 the universe Note: The universe contains approximately 10 50 particles. n

37 Classifying Functions Functions Poly Logarithmic Polynomial Exponential ExpDouble Exp Constant (log n) 5 n5n5 2 5n 5 2 n5n5 2 << (log n) θ(1) n θ(1) 2 θ(n) θ(1)2 n θ(1) 2 θ(n) 2

38 Classifying Functions LinearQuadratic Cubic ? θ(n 2 ) θ(n) θ(n 3 ) Polynomial = n θ(1) θ(n 4 ) Others θ(n 3 log 7 (n)) log(n) not absorbed because not Mult-constant

39 Classifying Functions Functions Poly. Exp. 8 · 2 4n / n 100 θ(2 4n / n 100 ) 8 · 2 4n + n 3 n θ(1) 2 θ(n) θ(2 4n ) 8·n28·n2 θ(n 2 ) θ(n 3 log(n)) 7 · n 3 log(n) We consider two (of many) levels in this hierarchy Individuals Ignore Mult-constant Ignore Power-constant

40 BigOh and Theta? 5n 2 + 8n + 2log n =  (n 2 ) Drop low-order terms. Drop multiplicative constant. 5n 2 log n + 8n + 2log n =  (n 2 log n)

41 Which Functions are Polynomials? n c n 0.0001 n 10000 5n 2 + 8n + 2log n 5n 2 log n 5n 2.5 Drop low-order terms. Drop constant, log, (and poly) factors. Ignore power constant. Write n θ(1)

42 Which Functions are Exponential? 2 n 2 0.0001 n 2 10000 n 8 n 2 n / n 100 2 n · n 100 Drop low-order terms. Drop constant, log, and poly factors. Ignore power constant. Write 2 θ(n)

43 Notations Theta f(n) = θ(g(n))f(n) ≈ c g(n) BigOh f(n) = O(g(n))f(n) ≤ c g(n) Omega f(n) = Ω(g(n))f(n) ≥ c g(n) Little Oh f(n) = o(g(n))f(n) << c g(n) Little Omega f(n) = ω(g(n))f(n) >> c g(n)

44 Definition of Theta f(n) = θ(g(n))

45 Definition of Theta f(n) is sandwiched between c 1 g(n) and c 2 g(n) f(n) = θ(g(n))

46 Definition of Theta f(n) is sandwiched between c 1 g(n) and c 2 g(n) for some sufficiently small c 1 (= 0.0001) for some sufficiently large c 2 (= 1000) f(n) = θ(g(n))

47 Definition of Theta For all sufficiently large n f(n) = θ(g(n))

48 Definition of Theta For all sufficiently large n For some definition of “sufficiently large” f(n) = θ(g(n))

49 Definition of Theta 3n 2 + 7n + 8 = θ(n 2 )

50 Order of Quantifiers No! It cannot be a different c 1 and c 2 for each n. f(n) = θ(g(n))

51 Gauss ∑ i=1..n i = 1 + 2 + 3 +... + n = ? Arithmetic Sum

52 1 2........ n = number of white dots 1+2+3+...+n-1+n=S Adding Made Easy

53 1+2+3+...+n-1+n=S n+ +n-2+... +2+1=S 1 2........ n = number of yellow dots n....... 2 1 = number of white dots Adding Made Easy

54 1+2+3+...+n-1+n=S n+ +n-2+... +2+1=S = number of white dots = number of yellow dots n+1 n+1 n+1 n+1 n+1 n n n n n n = n(n+1) dots in the grid 2S dots S = n (n+1) 2 Adding Made Easy

55 Gauss ∑ i=1..n i = 1 + 2 + 3 +... + n =  (# of terms · last term) Arithmetic Sum True when ever terms increase slowly

56 ∑ i=0..n 2 i = 1 + 2 + 4 + 8 +... + 2 n = ? Geometric Sum

57

58 ∑ i=0..n 2 i = 1 + 2 + 4 + 8 +... + 2 n = 2 · last term - 1 Geometric Sum

59 ∑ i=0..n r i = r 0 + r 1 + r 2 +... + r n = ? Geometric Sum

60 S Srrrr...rr S 1r S r1 r1 23nn1 n1 n1  1rrr... 23 r n          Geometric Sum

61 ∑ i=0..n r i r1 r1 n1     Geometric Sum When r>1?

62 ∑ i=0..n r i r1 r1 n1     Geometric Sum θ(rn)θ(rn) Biggest Term When r>1

63 ∑ i=0..n r i = r 0 + r 1 + r 2 +... + r n =  (biggest term) Geometric Increasing True when ever terms increase quickly

64 ∑ i=0..n r i  Geometric Sum When r<1? 1 r n1   

65 ∑ i=0..n r i  Geometric Sum 1 r n1     θ(1) When r<1 Biggest Term

66 ∑ i=0..n r i = r 0 + r 1 + r 2 +... + r n =  (1) Bounded Tail True when ever terms decrease quickly

67 ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + …+ 1 / n = ? Harmonic Sum

68 f(i) = 1 ∑ i=1..n f(i) = n n Sum of Shrinking Function

69 f(i) = ? ∑ i=1..n f(i) = n 1/2 n Sum of Shrinking Function

70 f(i) = 1/2 i ∑ i=1..n f(i) = 2  Sum of Shrinking Function

71 f(i) = 1/i ∑ i=1..n f(i) = ? n Sum of Shrinking Function

72 Harmonic Sum

73 ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + …+ 1 / n =  (log(n)) Harmonic Sum

74 Approximating Sum by Integrating The area under the curve approximates the sum ∑ i=1..n f(i) ≈ ∫ x=1..n f(x) dx

75 Adding Made Easy (For +, -, ×, , exp, log functions f(n))

76 Adding Made Easy

77 Geometric Like: If f(n)  2 Ω(n), then ∑ i=1..n f(i) = θ(f(n)). Arithmetic Like: If f(n) = n θ(1)-1, then ∑ i=1..n f(i) = θ(n · f(n)). Harmonic: If f(n) = 1 / n, then ∑ i=1..n f(i) = log e n + θ(1). Bounded Tail: If f(n)  n -1-Ω(1), then ∑ i=1..n f(i) = θ(1). ( For +, -, , , exp, log functions f(n) ) This may seem confusing, but it is really not. It should help you compute most sums easily. Adding Made Easy

78 Recurrence Relations T(1) = 1 T(n) = a T(n/b) + f(n)

79 Recurrence Relations  Time of Recursive Program With all this recursing and looping, how do we determine the running time of this recursive program? procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b ) T(1) = 1 T(n) = a T(n/b) + f(n) Recurrence relations.

80 Recurrence Relations  Time of Recursive Program Let n be the “size” of our input. procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b ) T(1) = 1 T(n) = a T(n/b) + f(n)

81 Recurrence Relations  Time of Recursive Program Let T(n) be the # of “Hi”s printed by me procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b ) T(1) = 1 T(n) = a T(n/b) + f(n)

82 Recurrence Relations  Time of Recursive Program Let T(n) be the # of “Hi”s printed by me And by all of my friends. procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b ) T(1) = 1 T(n) = a T(n/b) + f(n)

83 If my input is sufficiently small (according to my measure of size) then I print  (1) “Hi”s. Recurrence Relations  Time of Recursive Program procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b ) T(1) = 1 T(n) = a T(n/b) + f(n)

84 I personally output f(n) “Hi”s, i.e. top level stack frame. Recurrence Relations  Time of Recursive Program procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b ) T(1) = 1 T(n) = a T(n/b) + f(n)

85 I have a friends i.e. recurse a times. Recurrence Relations  Time of Recursive Program procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b ) T(1) = 1 T(n) = a T(n/b) + f(n)

86 Let b be the factor by which I shrink the input before giving it to a friend. i.e. if my input has size n then my friends are of size n/b. Recurrence Relations  Time of Recursive Program procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b ) T(1) = 1 T(n) = a T(n/b) + f(n)

87 By definition of T, the number of “Hi”s printed by one of my friends and all of his friends is T(n/b). Recurrence Relations  Time of Recursive Program procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b ) T(1) = 1 T(n) = a T(n/b) + f(n)

88 Hence, the total number printed by me and my a friends is T(n) = a T(n/b) + f(n) Recurrence Relations  Time of Recursive Program procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b ) T(1) = 1 T(n) = a T(n/b) + f(n)

89 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames Work in Level 0n f(n)f(n) 1 1 · f(n) 1 n/bn/bf(n/b)f(n/b) a a · f(n/b)a · f(n/b) 2 n/b2n/b2 f(n/b2)f(n/b2) a2a2 a 2 · f(n/b 2 ) i n/bin/bi f(n/bi)f(n/bi) aiai a i · f(n/b i ) h = log n / log b n/bhn/bh T(1) n log a / log b n · T(1) log a / log b Dominated by Top Level or Base Cases

90 Evaluating: T(n) = aT(n/b)+f(n)

91 Merge Sort Sort Time: T(n) = =  (n log(n)) 2T(n/2) +  (n) Total work at any level =  (n) # levels =  (log(n))

92 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames Work in Level 0n f(n)f(n) 1 1 · f(n) 1 n/bn/bf(n/b)f(n/b) a a · f(n/b) 2 n/b2n/b2 f(n/b2)f(n/b2) a2a2 a 2 · f(n/b 2 ) i n/bin/bi f(n/bi)f(n/bi) aiai a i · f(n/b i ) h = log n / log b n/bhn/bh T(1) n log a / log b n · T(1) log a / log b Total Work T(n) = ∑ i=0..h a i  f(n/b i ) 2T(n/2) +  (n) 2 i ·  (n/2 i ) =  (n) =  (n) · log(n) All levels basically the same.

93 Evaluating: T(n) = aT(n/b)+f(n)

94 Logs log 27 log 9 =

95 Logs log ? 27 log ? 9 = Which base?

96 Logs log 2 27 log 2 9 = It does not matter. log 2 c · log c 27 · log c 9 = log c 27 log c 9 Which is easiest?

97 Logs log 3 27 log 3 9 = log 3 3 3 log 3 3 2 = 3 2 Please no calculators in exams. And I wont ask log 25 log 9

98 Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n

99 Time for top level?: Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n

100 Time for top level: f(n) = n 3 / log 5 n Time for base cases?: Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n

101 Time for top level: f(n) = n 3 / log 5 n Time for base cases: Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n θ(n ) = log a / log b θ(n ) log ? / log ?

102 Time for top level: f(n) = n 3 / log 5 n Time for base cases: Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n θ(n ) = log a / log b θ(n ) = ? log 4 / log 2

103 Time for top level: f(n) = n 3 / log 5 n Time for base cases: Dominated?: c = ? Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 log a / log b = ?

104 Time for top level: f(n) = n 3 / log 5 n Time for base cases: Dominated?: c = 3 > 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = ?

105 Time for top level: f(n) = n 3 / log 5 n Time for base cases: Dominated?: c = 3 > 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = θ(top) = θ(f(n)) = θ(n 3 / log 5 n ).

106 Time for top level: f(n) = 2 n Time for base cases: Dominated?: c = ? > 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ 2 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = θ(top) = θ(f(n)) = θ(2 n ). bigger >big The time is even more dominated by the top level of recursion

107 Time for top level: f(n) = n log 5 n Time for base cases: Dominated?: c = ? 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = ? 1< = θ(base cases) = θ(n ) = θ(n 2 ). log a / log b

108 Time for top level: f(n) = log 5 n Time for base cases: Dominated?: c = ? 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = ? 0< = θ(base cases) = θ(n ) = θ(n 2 ). log a / log b

109 Time for top level: f(n) = n 2 Time for base cases: Dominated?: c = ? 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 2 θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = ? 2= = θ(f(n) log(n) ) = θ(n 2 log(n) ).

110 Time for top level: f(n) = n 2 log 5 n Time for base cases: Dominated?: c = 2 = 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 2 log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 d = ? 5 ? Hence, T(n) = ? > -1 = θ(f(n) log(n) ) = θ(n 2 log 6 (n) ).

111 Evaluating: T(n) = aT(n/b)+f(n)

112 Evaluating: T(n) = aT(n-b)+f(n) h = ? n-hb n-ib n-2b n-b T(0) f(n-ib) f(n-2b) f(n-b) f(n) n Work in Level # stack frames Work in stack frame Instance size a aiai a2a2 a 1 n/bn/b a · T(0) a i · f(n-ib) a 2 · f(n-2b) a · f(n-b) 1 · f(n) n/bn/b |base case| = 0 = n-hb h = n / b i 2 1 0 Level Likely dominated by base cases Exponential

113 Evaluating: T(n) = 1T(n-b)+f(n) h = ? Work in Level # stack frames Work in stack frame Instance size 1 1 1 1 1 f(0) f(n-ib) f(n-2b) f(n-b) f(n) n-b n n-hb n-ib n-2b T(0) f(n-ib) f(n-2b) f(n-b) f(n) h = n / b i 2 1 0 Level Total Work T(n) = ∑ i=0..h f(b·i) = θ(f(n)) or θ(n · f(n))

114 Evaluating: T(n) = aT(n/b)+f(n)

115 Relevant Mathematics In more detail. Jeff Edmonds York University COSC 3101 Lecture skipped Classifying Functions The Time Complexity of an Algorithm Adding Made Easy Recurrence Relations

116 Assumed Knowledge Read the section on Existential and Universal Quantifiers, Logarithms and Exponentials  g  b Loves(b,g)  b  g Loves(b,g)

117 Start With Some Math Input Size Time Classifying Functions f(i) = n  (n) Recurrence Relations T(n) = a T(n/b) + f(n) Adding Made Easy ∑ i=1 f(i). Time Complexity t(n) =  (n 2 )

118 Classifying Functions Giving an idea of how fast a function grows without going into too much detail.

119 Which are more alike?

120 Mammals

121 Which are more alike?

122 Dogs

123 Classifying Animals Vertebrates Birds Mammals Reptiles Fish Dogs Giraffe We consider two (of many) levels in this hierarchy Class Genus Individuals

124 Classifying Functions T(n)T(n) 101001,00010,000 log n 36913amoeba n 1/2 31031100bird 101001,00010,000human n log n 306009,000130,000my father n2n2 10010,00010 6 10 8 elephant n3n3 1,00010 6 10 9 10 12 dinosaur 2n2n 1,02410 30 10 300 10 3000 the universe Note: The universe contains approximately 10 50 particles. n

125 Which are more alike? n 1000 n2n2 2n2n

126 Which are more alike? Polynomials n 1000 n2n2 2n2n

127 Which are more alike? 1000n 2 3n 2 2n 3 Polynomials

128 Which are more alike? Quadratic 1000n 2 3n 2 2n 3 Polynomials

129 Classifying Functions? Functions

130 Classifying Functions Functions Poly Logarithmic Polynomial Exponential ExpDouble Exp Constant (log n) 5 n5n5 2 5n 5 2 n5n5 2 Others 2 n log(n)

131 Classifying Functions? Polynomial

132 Classifying Functions Polynomial LinearQuadratic Cubic ? 5n 2 5n 5n 3 5n 4 Others 5n 3 log 7 (n)

133 Classifying Functions Functions Poly. Exp. 8 · 2 4n / n 100 θ(2 4n / n 100 ) 8 · 2 4n + n 3 n θ(1) 2 θ(n) θ(2 4n ) 8·n28·n2 θ(n 2 ) θ(n 3 log(n)) 7 · n 2 + n We consider two (of many) levels in this hierarchy Ignore Power-constant Ignore Mult-constant Individuals

134 Logarithmic log 10 n = # digits to write n log 2 n = # bits to write n = 3.32 log 10 n log(n 1000 ) = 1000 log(n) Differ only by a multiplicative constant.

135 Poly Logarithmic (log n) 5 = log 5 n

136 Logarithmic << Polynomial For sufficiently large n log 1000 n << n 0.001

137 Linear << Quadratic For sufficiently large n 10000 n << 0.0001 n 2

138 Polynomial << Exponential For sufficiently large n n 1000 << 2 0.001 n

139 Classifying Functions Functions Poly Logarithmic Polynomial Exponential ExpDouble Exp Constant (log n) 5 n5n5 2 5n 5 2 n5n5 2 <<

140 Which Functions are Constant? 5 1,000,000,000,000 0.0000000000001 -5 0 8 + sin(n) Yes No Yes

141 Which Functions are “Constant”? 5 1,000,000,000,000 0.0000000000001 -5 0 8 + sin(n) Yes No Yes Lie in between 7 9 The running time of the algorithm is a “Constant” It does not depend significantly on the size of the input. Write θ(1).

142 Which Functions are Quadratic? n 2 … ?

143 Which Functions are Quadratic? n 2 0.001 n 2 1000 n 2 Some constant times n 2.

144 Which Functions are Quadratic? n 2 0.001 n 2 1000 n 2 5n 2 + 3000n + 2log n

145 Which Functions are Quadratic? n 2 0.001 n 2 1000 n 2 5n 2 + 3000n + 2log n Lie in between

146 Which Functions are Quadratic? n 2 0.001 n 2 1000 n 2 5n 2 + 3000n + 2log n Ignore multiplicative constant. Low-order terms absorbed into constant. Ignore "small" values of n. Write θ(n 2 ).

147 Classifying Functions Functions Poly. Exp. 8 · 2 4n / n 100 θ(2 4n / n 100 ) 8 · 2 4n + n 3 n θ(1) 2 θ(n) θ(2 4n ) 8·n28·n2 θ(n 2 ) θ(n 3 log(n)) 7 · n 2 + n We consider two (of many) levels in this hierarchy Ignore Mult-constant Individuals Ignore Power-constant

148 Classifying Functions Functions Poly. Exp. 8 · 2 4n / n 100 θ(2 4n / n 100 ) 8 · 2 4n + n 3 n θ(1) 2 θ(n) θ(2 4n ) 8·n28·n2 θ(n 2 ) θ(n 3 log(n)) 7 · n 2 + n We consider two (of many) levels in this hierarchy Ignore Mult-constant Individuals Ignore Power-constant log(n) not absorbed because not Mult-constant

149 Which Functions are Polynomial? n 5 … ?

150 Which Functions are Polynomial? n c n 0.0001 n 10000 n to some constant power.

151 Which Functions are Polynomial? n c n 0.0001 n 10000 5n 2 + 8n + 2log n 5n 2 log n 5n 2.5

152 Which Functions are Polynomial? n c n 0.0001 n 10000 5n 2 + 8n + 2log n 5n 2 log n 5n 2.5 Lie in between

153 Which Functions are Polynomials? n c n 0.0001 n 10000 5n 2 + 8n + 2log n 5n 2 log n 5n 2.5 Ignore power constant. Low-order terms and log factors absorbed into constant. Write n θ(1)

154 Classifying Functions Functions Poly. Exp. 8 · 2 4n / n 100 θ(2 4n / n 100 ) 8 · 2 4n + n 3 n θ(1) 2 θ(n) θ(2 4n ) 8·n28·n2 θ(n 2 ) θ(n 3 log(n)) 7 · n 2 + n We consider two (of many) levels in this hierarchy Ignore Mult-constant Individuals Ignore Power-constant

155 If f(n) = c b an n d log e (n) c  ? a  ? b  ? d  ? e  ? > 0  0 or b  1 > 0 (- ,  ) Which Functions are Polynomials? then f(n)  n θ(1)

156 Which Functions are Exponential? 2 n … ?

157 Which Functions are Exponential? 2 n 2 0.0001 n 2 10000 n 2 raised to the power of n times some constant power

158 Which Functions are Exponential? 2 n 2 0.0001 n 2 10000 n 8 n 2 n / n 100 2 n · n 100 too small? too big?

159 Which Functions are Exponential? 2 n 2 0.0001 n 2 10000 n 8 n 2 n / n 100 2 n · n 100 = 2 3n > 2 0.5n < 2 2n Lie in between

160 Which Functions are Exponential? 2 n 2 0.0001 n 2 10000 n 8 n 2 n / n 100 2 n · n 100 = 2 3n > 2 0.5n < 2 2n 2 0.5n > n 100 2 n = 2 0.5n · 2 0.5n > n 100 · 2 0.5n 2 n / n 100 > n 0.5n

161 Which Functions are Exponential? Ignore power constant. Low-order terms, change in base, and polynomial factors absorbed into constant. Write 2 θ(n) 2 n 2 0.0001 n 2 10000 n 8 n 2 n / n 100 2 n · n 100

162 Classifying Functions Functions Poly. Exp. 8 · 2 4n / n 100 θ(2 4n / n 100 ) 8 · 2 4n + n 3 n θ(1) 2 θ(n) θ(2 4n ) 8·n28·n2 θ(n 2 ) θ(n 3 log(n)) 7 · n 2 + n We consider two (of many) levels in this hierarchy Ignore Mult-constant Individuals Ignore Power-constant

163 If f(n) = c b an n d log e (n) c  ? a  ? b  ? d  ? e  ? then f(n)  2 θ(n) > 0  0  1 (- ,  ) Which Functions are Exponential?

164 Classifying Functions Functions Poly Logarithmic Polynomial Exponential ExpDouble Exp Constant (log n) θ(1) n θ(1) 2 θ(n) θ(1) 2 n θ(1) 2 θ(n) 2 Ignore Power-constant

165 Classifying Functions LinearQuadratic Cubic ? θ(n 2 ) θ(n) θ(n 3 ) Polynomial = n θ(1) θ(n 4 ) Others θ(n 3 log 7 (n)) Ignore Mult-constant

166 Exponential 2 n vs 8 n Should these be classified together? 8n = 4 n · 2 n >> 1,000,000 · 2 n 8 n = 2 log(8) · n = 2 3n No: 8 n   (2 n ) Yes: 8 n = 2  (n) Ignore Power-constant Ignore Mult-constant

167 Classifying Functions Rank constants in significance. f(n) = 8·2 4n / n 100 + 5·n 3 Tell me the most significant thing about your function

168 Classifying Functions f(n) = 8·2 4n / n 100 + 5·n 3 f(n) = 9·2 4n / n 100 + 5·n 3 vs additive: multiplicative: 9/8 2 4n / n 100 Rank constants in significance.

169 Classifying Functions f(n) = 8·2 4n / n 100 + 5·n 3 f(n) = 8·2 4n / n 100 + 6·n 3 vs ≈ 1 n3n3 Rank constants in significance. additive: multiplicative:

170 Classifying Functions f(n) = 8·2 4n / n 100 + 5·n 3 f(n) = 8·2 4n / n 100 + 5·n 4 vs ≈ 1 ≈ 5· n 4 Rank constants in significance. additive: multiplicative:

171 Classifying Functions f(n) = 8·2 4n / n 100 + 5·n 3 f(n) = 8·2 5n / n 100 + 5·n 3 vs multiplicative: 2n2n Rank constants in significance.

172 Classifying Functions f(n) = 8·2 4n / n 100 + 5·n 3 f(n) = 8·2 4n / n 101 + 5·n 3 vs multiplicative: n Rank constants in significance.

173 Classifying Functions f(n) = 8·2 4n / n 100 + 5·n 3 Rank constants in significance. 1231’45

174 Classifying Functions f(n) = 8·2 4n / n 100 + 5·n 3 Rank constants in significance. Significant Less significant Irrelevant

175 Classifying Functions f(n) = 8·2 4n / n 100 + 5·n 3 Tell me the most significant thing about your function 2 θ(n) 2 3n << 2 4n f(n)because

176 Classifying Functions f(n) = 8·2 4n / n 100 + 5·n 3 Tell me the most significant thing about your function 2 θ(n) 2 4n /n θ(1) θ(2 4n /n 100 ) 8·2 4n / n 100 + θ(n 3 ) 8·2 4n / n 100 + n θ(1) 8·2 4n / n 100 + 5·n 3

177 Classifying Functions Functions Poly. Exp. 8 · 2 4n / n 100 + n 3, θ(2 4n / n 100 ) 8 · 2 4n + n 3 n θ(1) 2 θ(n) θ(2 4n ) Ignore Mult-constant Individuals Ignore Power-constant

178 Notations Theta f(n) = θ(g(n))f(n) ≈ c g(n) BigOh f(n) = O(g(n))f(n) ≤ c g(n) Omega f(n) = Ω(g(n))f(n) ≥ c g(n) Little Oh f(n) = o(g(n))f(n) << c g(n) Little Omega f(n) = ω(g(n))f(n) >> c g(n)

179 Definition of Theta f(n) = θ(g(n))

180 Definition of Theta f(n) is sandwiched between c 1 g(n) and c 2 g(n) f(n) = θ(g(n))

181 Definition of Theta f(n) is sandwiched between c 1 g(n) and c 2 g(n) for some sufficiently small c 1 (= 0.0001) for some sufficiently large c 2 (= 1000) f(n) = θ(g(n))

182 Definition of Theta For all sufficiently large n f(n) = θ(g(n))

183 Definition of Theta For all sufficiently large n For some definition of “sufficiently large” f(n) = θ(g(n))

184 Definition of Theta 3n 2 + 7n + 8 = θ(n 2 )

185 Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) c 1 ·n 2  3n 2 + 7n + 8  c 2 ·n 2 ??

186 Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 34

187 Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·1 2  3·1 2 + 7·1 + 8  4·1 2 1 False

188 Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·7 2  3·7 2 + 7·7 + 8  4·7 2 7 False

189 Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·8 2  3·8 2 + 7·8 + 8  4·8 2 8 True

190 Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·9 2  3·9 2 + 7·9 + 8  4·9 2 9 True

191 Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·10 2  3·10 2 + 7·10 + 8  4·10 2 10 True

192 Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 ?

193 Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 n  8 8

194 Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 True n  8

195 Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 7n + 8  1 · n 2 7 + 8 / n  1 · n n  8 True

196 Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 348 n  8 True

197 Definition of Theta n 2 = θ(n 3 )

198 Definition of Theta n 2 = θ(n 3 ) c 1 · n 3  n 2  c 2 · n 3 ??

199 Definition of Theta n 2 = θ(n 3 ) 0 · n 3  n 2  c 2 · n 3 0 True, but ?

200 Definition of Theta n 2 = θ(n 3 ) 0 Constants c1 and c2 must be positive! False

201 Definition of Theta 3n 2 + 7n + 8 = θ(n)

202 Definition of Theta 3n 2 + 7n + 8 = θ(n) c 1 ·n  3n 2 + 7n + 8  c 2 ·n ??

203 Definition of Theta 3n 2 + 7n + 8 = θ(n) 3·n  3n 2 + 7n + 8  100·n 3 100

204 Definition of Theta 3n 2 + 7n + 8 = θ(n) 3·n  3n 2 + 7n + 8  100·n ?

205 Definition of Theta 3n 2 + 7n + 8 = θ(n) 3·100  3·100 2 + 7·100 + 8  100·100 100 False

206 Definition of Theta 3n 2 + 7n + 8 = θ(n) 3·n  3n 2 + 7n + 8  10,000·n 3 10,000

207 Definition of Theta 3n 2 + 7n + 8 = θ(n) 3·10,000  3·10,000 2 + 7·10,000 + 8  10,000·10,000 10,000 False

208 Definition of Theta 3n 2 + 7n + 8 = θ(n) What is the reverse statement?

209 Understand Quantifiers!!! SamMary BobBeth John Marilyn Monroe FredAnn SamMary BobBeth John Marilyn Monroe FredAnn  b,  Loves(b, MM)  b, Loves(b, MM)  b,  Loves(b, MM)]  b, Loves(b, MM)]

210 Definition of Theta 3n 2 + 7n + 8 = θ(n) The reverse statement

211 Definition of Theta 3n 2 + 7n + 8 = θ(n) 3n 2 + 7n + 8 > 100·n 3 100 ?

212 Definition of Theta 3n 2 + 7n + 8 = θ(n) 3 · 100 2 + 7 · 100 + 8 > 100 · 100 3 100 True 100

213 Definition of Theta 3n 2 + 7n + 8 = θ(n) 3n 2 + 7n + 8 > 10,000·n 3 10,000 ?

214 Definition of Theta 3n 2 + 7n + 8 = θ(n) 3 · 10,000 2 + 7 · 10,000 + 8 > 10,000 · 10,000 3 10,000 True 10,000

215 Definition of Theta 3n 2 + 7n + 8 = θ(n) 3n 2 + 7n + 8 > c 2 ·n c1c1 c2c2 ?

216 Definition of Theta 3n 2 + 7n + 8 = θ(n) 3 · c 2 2 + 7 · c 2 + 8 > c 2 · c 2 c1c1 c2c2 True c2c2

217 Definition of Theta 3n 2 + 7n + 8 = θ(n) 3n 2 + 7n + 8 > c 2 ·n c1c1 c2c2 ? n0n0

218 Definition of Theta 3n 2 + 7n + 8 = θ(n) 3 · c 2 2 + 7 · c 2 + 8 > c 2 · c 2 c1c1 max(c 2,n 0 ) True c2c2 n0n0

219 Definition of Theta 3n 2 + 7n + 8 = g(n) θ(1)  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 3n 2 + 7n + 8 = n θ(1)

220 Definition of Theta 3n 2 + 7n + 8 = n θ(1) n c1  3n 2 + 7n + 8  n c2  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 ??

221 Definition of Theta 3n 2 + 7n + 8 = n θ(1) n 2  3n 2 + 7n + 8  n 3  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 23

222 Definition of Theta 3n 2 + 7n + 8 = n θ(1) n 2  3n 2 + 7n + 8  n 3  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 23? n  ?

223 Definition of Theta 3n 2 + 7n + 8 = n θ(1) n 2  3n 2 + 7n + 8  n 3  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 235 n  5

224 Definition of Theta 3n 2 + 7n + 8 = n θ(1) n 2  3n 2 + 7n + 8  n 3 True  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 235 n  5

225 Order of Quantifiers f(n) = θ(g(n)) ?

226 Understand Quantifiers!!! One girl Could be a separate girl for each boy. SamMary BobBeth John Marilyn Monroe FredAnn SamMary BobBeth John Marilyn Monroe FredAnn

227 Order of Quantifiers No! It cannot be a different c 1 and c 2 for each n. f(n) = θ(g(n))

228 Other Notations Theta f(n) = θ(g(n))f(n) ≈ c g(n) BigOh f(n) = O(g(n))f(n) ≤ c g(n) Omega f(n) = Ω(g(n))f(n) ≥ c g(n) Little Oh f(n) = o(g(n))f(n) << c g(n) Little Omega f(n) = ω(g(n))f(n) >> c g(n)

229 BigOh Notation n 2 = O(n 3 ) n 3 = O(n 2 )

230 BigOh Notation O(n 2 ) = O(n 3 ) O(n 3 ) = O(n 2 ) Odd Notation f(n) = O(g(n)) Standard f(n) ≤ O(g(n)) Stresses one function dominating another. f(n)  O(g(n)) Stress function is member of class.

231 Adding The Classic Techniques Evaluating ∑ i=1 f(i). n

232 Gauss ∑ i=1..n i = 1 + 2 + 3 +... + n = ? Arithmetic Sum

233 1+2+3+...+ n-1+n=S n+n-1+n-2+... +2+1=S (n+1) + (n+1) + (n+1) +... + (n+1) + (n+1) = 2S n (n+1) = 2S

234 1+2+3+...+ n-1+n=S n+n-1+n-2+... +2+1=S (n+1) + (n+1) + (n+1) +... + (n+1) + (n+1) = 2S n (n+1) = 2S

235 1+2+3+...+ n-1+n=S n+n-1+n-2+... +2+1=S (n+1) + (n+1) + (n+1) +... + (n+1) + (n+1) = 2S n (n+1) = 2S

236 1+2+3+...+ n-1+n=S n+n-1+n-2+... +2+1=S (n+1) + (n+1) + (n+1) +... + (n+1) + (n+1) = 2S n (n+1) = 2S

237 Let’s restate this argument using a geometric representation Algebraic argument

238 1 2........ n = number of white dots.

239 1 2........ n = number of white dots = number of yellow dots n....... 2 1

240 n+1 n+1 n+1 n+1 n+1 = number of white dots = number of yellow dots n n n n n n There are n(n+1) dots in the grid

241 n+1 n+1 n+1 n+1 n+1 = number of white dots = number of yellow dots n n n n n n Note =  (# of terms · last term))

242 Gauss ∑ i=1..n i = 1 + 2 + 3 +... + n =  (# of terms · last term) Arithmetic Sum True when ever terms increase slowly

243 ∑ i=0..n 2 i = 1 + 2 + 4 + 8 +... + 2 n = ? Geometric Sum

244

245 ∑ i=0..n 2 i = 1 + 2 + 4 + 8 +... + 2 n = 2 · last term - 1 Geometric Sum

246 ∑ i=0..n r i = r 0 + r 1 + r 2 +... + r n = ? Geometric Sum

247 S Srrrr...rr S 1r S r1 r1 23nn1 n1 n1  1rrr... 23 r n          Geometric Sum

248 ∑ i=0..n r i r1 r1 n1     Geometric Sum When r>1?

249 ∑ i=0..n r i r1 r1 n1     Geometric Sum θ(rn)θ(rn) Biggest Term When r>1

250 ∑ i=0..n r i = r 0 + r 1 + r 2 +... + r n =  (biggest term) Geometric Increasing True when ever terms increase quickly

251 ∑ i=0..n r i  Geometric Sum When r<1? 1 r n1   

252 ∑ i=0..n r i  Geometric Sum 1 r n1     θ(1) When r<1 Biggest Term

253 ∑ i=0..n r i = r 0 + r 1 + r 2 +... + r n =  (1) Bounded Tail True when ever terms decrease quickly

254 ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + …+ 1 / n = ? Harmonic Sum

255 f(i) = 1 ∑ i=1..n f(i) = n n Sum of Shrinking Function

256 f(i) = ? ∑ i=1..n f(i) = n 1/2 n Sum of Shrinking Function

257 f(i) = 1/2 i ∑ i=1..n f(i) = 2  Sum of Shrinking Function

258 f(i) = 1/i ∑ i=1..n f(i) = ? n Sum of Shrinking Function

259 Harmonic Sum

260 ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + …+ 1 / n =  (log(n)) Harmonic Sum

261 Approximating Sum by Integrating The area under the curve approximates the sum ∑ i=1..n f(i) ≈ ∫ x=1..n f(x) dx

262 Approximating Sum by Integrating 1 / c+1 x c+1 d dx = x c ∫ x=1..n x c dx = 1 / c+1 n c+1 ∑ i=1..n i c ≈ Arithmetic Sums  θ(# of terms · last term) True when ever terms increase slowly = θ(n · n c )

263 Approximating Sum by Integrating 1 / ln(b) e ln(b)x d dx = e ln(b)x = b x ∫ x=1..n b x dx = 1 / ln(b) b x ∑ i=1..n b i ≈ Geometric Sums  θ(last term) True when ever terms increase quickly = θ(b n )

264 Harmonic Sum Approximating Sum by Integrating

265 Problem: Integrating may be hard too.

266 Outline II) Math proofs of sums V) Ability to know the answer fast I) What is the sum Eg. ∑ i=1..n f(i) = ∑ i=1..n 1 / i 0.9999 =θ(n 0.0001 ) III) Pattern of sums ∑ i=1..n f(i) = θ(n · f(n)) IV) Intuition why this is the sum I flash it up If you follow great. If not don’t panic.

267 Adding Made Easy We will now classify (most) functions f(i) into four classes: –Geometric Like –Arithmetic Like –Harmonic –Bounded Tail

268 Adding Made Easy We will now classify (most) functions f(i) into four classes For each class, we will give an easy rule for approximating it’s sum θ( ∑ i=1..n f(i) )

269 Classifying Animals Vertebrates Birds Mammals Reptiles Fish Dogs Giraffe Mammal  Complex social networks Dog  Man’s best friend

270 Classifying Functions Functions Poly. Exp. 2 θ(n)  ∑ i=1..n f(i) = θ(f(n)) 8 · 2 n / n 100 + n 3, θ(2 n / n 100 ) 8 · 2 n + n 3 n θ(1) 2 θ(n) θ(2n)θ(2n) f(n) = 8 · 2 n / n 100 + n 3 θ(2 n / n 100 )  ∑ i=1..n f(i) = θ(2 n / n 100 ) 2 0.5n << 2 n 8 · 2 n / n 100 + n 3 Significant Less significant Irrelevant

271 Adding Made Easy 11 1 2 3 4 Four Classes of Functions

272 Adding Made Easy

273 If the terms f(i) grow sufficiently quickly, then the sum will be dominated by the largest term. Silly example: 1+2+3+4+5+ 1,000,000,000 ≈ 1,000,000,000 f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

274 If the terms f(i) grow sufficiently quickly, then the sum will be dominated by the largest term. Classic example: ? f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

275 If the terms f(i) grow sufficiently quickly, then the sum will be dominated by the largest term. Classic example: ∑ i=1..n 2 i = 2 n+1 -1 ≈ 2 f(n) f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

276 If the terms f(i) grow sufficiently quickly, then the sum will be dominated by the largest term. For which functions f(i) is this true? How fast and how slow can it grow? f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

277 r1 r1 n1      θ( ) r n Last Term when r>1. ∑ i=1..n r i 1rrr... 23 r n    θ(f(n)) ∑ i=1..n (1.0001) i ≈ 10,000 (1.0001) n = 10,000 f(n) Lower Extreme: Upper Extreme: ∑ i=1..n (1000) i ≈ 1.001(1000) n = 1.001 f(n) Recall, when f(n) = r i = 2 θ(n) f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

278 Because the constant is sooo big, the statement is just barely true. ∑ i=1..n (1.0001) i ≈ 10,000 (1.0001) n = 10,000 f(n) Lower Extreme: Upper Extreme: ∑ i=1..n (1000) i ≈ 1.001(1000) n = 1.001 f(n) f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

279 ∑ i=1..n (1.0001) i ≈ 10,000 (1.0001) n = 10,000 f(n) Lower Extreme: Upper Extreme: ∑ i=1..n (1000) i ≈ 1.001(1000) n = 1.001 f(n) Even bigger? f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

280 2n2n 2i2i ∑ i=1..n 2 2 ≈ 2 2 = 1f(n) No Upper Extreme: Even bigger? ∑ i=1..n (1.0001) i ≈ 10,000 (1.0001) n = 10,000 f(n) Lower Extreme: f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

281 2n2n 2i2i ∑ i=1..n 2 2 ≈ 2 2 = 1f(n) No Upper Extreme: Functions in between? ∑ i=1..n (1.0001) i ≈ 10,000 (1.0001) n = 10,000 f(n) Lower Extreme: f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

282 ∑ i=1..n 2 i = 2 · 2 n –2 = 2 1 + 2 2 + 2 3 + 2 4 +…+ 2 n 8 · [ ] 8·8· 8·8· 8·8· 8·8· 8·8· 8·8· = θ( 2 n ) 1 100 2 100 3 100 4 100 n 100 i 100 n 100 + i 3 + 1 3 + 2 3 + 3 3 + 4 3 + n 3 + n 4 Dominated by the largest term. = θ(f(n)) f(n) = 2 n 8·8· n 100 + n 3 = θ( 2 n ) n 100 f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like: n 100 ≈

283 f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like: f(n) = 2 n 8·8· n 100 ∑ i=1..n f(i) f(n)  c · f(n) f(i)  · f(n) f(i)  · f(i +1 ) EasyHard  (1 + 1 / i ) 100 1 2 1.9 = 8 · 2 (i+1) (i +1 ) 100 f(i +1 ) f(i) 8·2i8·2i i 100 =  1/.51  · f(i +1 ) f(i) (i +1 ) 100 i 100 2 =

284 f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like: f(n) = 2 n 8·8· n 100 ∑ i=1..n f(i) f(n)  c · f(n) f(i)  · f(n)   · f(i +2 )  · f(i +1 ) f(i)  · f(i +1 ) f(i)   · f(i +3 ) .51 (n-i) ·f(i +(n-i) ) .51 (n-i) ·f( n ) f(i) .51 (n-i) ·f( n ) Eg. i = n, f(n) .51 (n-n) ·f( n )  1·f( n ) i = 0, f(0) .51 n ·f( n )

285 f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like: f(n) = 2 n 8·8· n 100 ∑ i=1..n f(i) f(n)  c · f(n) ∑ i=1..n f(i)  ∑ i=1..n.51 (n-i) ·f( n ) = θ(1) · f(n) f(i)  · f(n)  · f(i +1 ) f(i) f(i) .51 (n-i) ·f( n ) = [∑ i=1..n.51 (n-i) ] · f( n )

286 Functions Poly. Exp. 2 θ(n)  ∑ i=1..n f(i) = θ(f(n)) 8 · 2 n / n 100 + n 3 θ(2 n / n 100 ) 8 · 2 n + n 3 n θ(1) 2 θ(n) θ(2n)θ(2n) f(n) = 8 · 2 n / n 100 + n 3 θ(2 n / n 100 )  ∑ i=1..n f(i) = θ(2 n / n 100 ) f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

287 If f(n) = c b an n d log e (n) c  ? a  ? b  ? d  ? e  ? f(n) = Ω, θ ? then ∑ i=1..n f(i) = θ(f(n)).  2 Ω(n) > 0  0  1 (- ,  ) f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

288 f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

289 All functions in 2 Ω(n) ? Maybe not. f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

290 Functions that oscillate continually do not have the property. f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

291 Functions expressed with +, -, , , exp, log do not oscillate continually. They are well behaved for sufficiently large n. These do have the property. f(i)  2 Ω(n)  ∑ i=1..n f(i) = θ(f(n)) Geometric Like:

292 Adding Made Easy Done

293 Adding Made Easy

294 If most of the terms f(i) have roughly the same value, then the sum is roughly the number of terms, n, times this value. Silly example: 1,001 + 1,002 + 1,003 + 1,004 + 1,005 ≈ 5 · 1,000 f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

295 If most of the terms f(i) have roughly the same value, then the sum is roughly the number of terms, n, times this value. Another silly example: ∑ i=1..n 1 = n · 1 f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

296 If most of the terms f(i) have roughly the same value, then the sum is roughly the number of terms, n, times this value. Is the statement true for this function? ∑ i=1..n i = 1 + 2 + 3 +... + n f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

297 If most of the terms f(i) have roughly the same value, then the sum is roughly the number of terms, n, times this value. Is the statement true for this function? ∑ i=1..n i = 1 + 2 + 3 +... + n The terms are not roughly the same. f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

298 But half the terms are roughly the same. ∑ i=1..n i = 1 +... + n/2 +... + n f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

299 But half the terms are roughly the same and the sum is roughly the number terms, n, times this value ∑ i=1..n i = 1 +... + n/2 +... + n ∑ i=1..n i = θ(n · n) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

300 Is the statement true for this function? ∑ i=1..n i 2 = 1 2 + 2 2 + 3 2 +... + n 2 Even though, the terms are not roughly the same. f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

301 Again half the terms are roughly the same. ∑ i=1..n i = 1 2 +... + (n/2) 2 +... + n 2 1 / 4 n 2 f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

302 Again half the terms are roughly the same. ∑ i=1..n i = 1 2 +... + (n/2) 2 +... + n 2 1 / 4 n 2 ∑ i=1..n i 2 = θ(n · n 2 ) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

303 ∑ i=1..n f(i) ≈ area under curve f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

304 area of small square  ∑ i=1..n f(i) ≈ area under curve  area of big square f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

305 n / 2 · f( n / 2 ) = area of small square  ∑ i=1..n f(i) ≈ area under curve  area of big square = n · f(n) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

306 θ(n · f(n)) = n / 2 · f( n / 2 ) = area of small square  ∑ i=1..n f(i) ≈ area under curve  area of big square = n · f(n) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

307 θ(n · f(n)) = n / 2 · f( n / 2 ) = area of small square  ∑ i=1..n f(i) ≈ area under curve  area of big square = n · f(n) ? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

308 ∑ i=1..n i 2 = 1 2 + 2 2 + 3 2 +... + n 2 f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like: f(i) = n 2 f( n / 2 ) = θ(f(n)) The key property is = ?

309 The key property is f( n / 2 ) = (n/2) 2 = 1 / 4 n 2 = θ( n 2 ) = θ(f(n)) ∑ i=1..n i 2 = 1 2 + 2 2 + 3 2 +... + n 2 f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like: f(i) = n 2  = θ(n·f(n)) = θ(n · n 2 )

310 ∑ i=1..n f(i) = ? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like: f(i) = n θ(1)

311 ∑ i=1..n i r = 1 r + 2 r + 3 r +... + n r f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like: f(i) = n θ(1) f( n / 2 ) = θ(f(n)) The key property is

312 f( n / 2 ) = (n/2) r = 1 / 2 r n r = θ( n r ) = θ(f(n)) ∑ i=1..n i r = 1 r + 2 r + 3 r +... + n r f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:  = θ(n·f(n)) = θ(n · n r ) f(i) = n θ(1)

313 ∑ i=1..n 2 i = 2 1 + 2 2 + 2 3 +... + 2 n f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like: f(i) = n θ(1) f( n / 2 ) = θ(f(n)) The key property is

314 ∑ i=1..n 2 i = 2 1 + 2 2 + 2 3 +... + 2 n f( n / 2 ) = 2 (n/2) = θ( 2 n ) = θ(f(n)) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like: f(i) = n θ(1)  = θ(n·f(n)) = θ(n · 2 n )

315 ∑ i=1..n 1 = n · 1 = n · f(n) Middle Extreme: Upper Extreme: ∑ i=1..n i 1000 = 1 / 1001 n 1001 = 1 / 1001 n · f(n) All functions in between. f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

316 Adding Made Easy Half done

317 f(i) = 1 ∑ i=1..n f(i) = n n Sum of Shrinking Function

318 f(i) = ? ∑ i=1..n f(i) = n 1/2 n Sum of Shrinking Function 1/i 1/2

319 f(i) = 1/i ∑ i=1..n f(i) = log n n Sum of Shrinking Function

320 f(i) = 1/2 i ∑ i=1..n f(i) = 2 Sum of Shrinking Function 

321 If most of the terms f(i) have roughly the same value, then the sum is roughly the number of terms, n, times this value. Does the statement hold for functions f(i) that shrink? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

322 If most of the terms f(i) have roughly the same value, then the sum is roughly the number of terms, n, times this value. ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + … + 1 / n Does the statement hold for the Harmonic Sum? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

323 Does the statement hold for the Harmonic Sum? ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + … + 1 / n ∑ i=1..n f(i) = ? θ(n · f(n)) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

324 Does the statement hold for the Harmonic Sum? ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + … + 1 / n ∑ i=1..n f(i) θ(n · f(n)) = ∑ i=1..n 1 / i = ? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

325 Does the statement hold for the Harmonic Sum? ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + … + 1 / n ∑ i=1..n f(i) θ(n · f(n)) = ∑ i=1..n 1 / i =θ(log n) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

326 Does the statement hold for the Harmonic Sum? ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + … + 1 / n ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i =θ(log n) ? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

327 Does the statement hold for the Harmonic Sum? ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + … + 1 / n ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i =θ(log n) θ(1) = θ(n · 1 / n ) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

328 Does the statement hold for the Harmonic Sum? ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + … + 1 / n ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i =θ(log n) θ(1) = θ(n · 1 / n ) ≠ No the statement does not hold! f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

329 Adding Made Easy not included

330 Does the statement hold for the almost Harmonic Sum? ∑ i=1..n 1 / i 0.9999 Shrinks slightly slower than harmonic. f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

331 Does the statement hold for the almost Harmonic Sum? ∑ i=1..n 1 / i 0.9999 ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i 0.9999 = ? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

332 Does the statement hold for the almost Harmonic Sum? ∑ i=1..n 1 / i 0.9999 ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i 0.9999 =θ(n 0.0001 ) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

333 Does the statement hold for the almost Harmonic Sum? ∑ i=1..n 1 / i 0.9999 ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i 0.9999 =θ(n 0.0001 ) ? f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

334 Does the statement hold for the almost Harmonic Sum? ∑ i=1..n 1 / i 0.9999 ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i 0.9999 =θ(n 0.0001 ) θ(n 0.0001 ) = θ(n · 1 / n 0.9999 ) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

335 Does the statement hold for the almost Harmonic Sum? ∑ i=1..n 1 / i 0.9999 ∑ i=1..n f(i) = θ(n · f(n)) = ∑ i=1..n 1 / i 0.9999 =θ(n 0.0001 ) θ(n 0.0001 ) = θ(n · 1 / n 0.9999 ) = The statement does hold! f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

336 ∑ i=1..n 1 = n · 1 = n · f(n) Middle Extreme: Lower Extreme: ∑ i=1..n 1 / i 0.9999 = θ(n 0.0001 ) = θ(n · f(n)) All functions in between. f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

337 ∑ i=1..n 1 = n · 1 = n · f(n) Middle Extreme: Lower Extreme: ∑ i=1..n 1 / i 0.9999 = θ(n 0.0001 ) = θ(n · f(n)) Upper Extreme: ∑ i=1..n i 1000 = 1 / 1001 n 1001 = 1 / 1001 n · f(n) f(i) = n θ(1)-1  ∑ i=1..n f(i) = θ(n·f(n)) Arithmetic Like:

338 Arithmetic Like If f(n) = c b an n d log e (n) c  ? a  ? b  ? d  ? e  ? f(n) = Ω, θ ? then ∑ i=1..n f(i) = θ(n · f(n)). n -1+θ(1) > 0  0 or b  1 > -1 (- ,  ) (For +, -, , , exp, log functions f(n)) Conclusion

339 Adding Made Easy Done

340 Adding Made Easy Harmonic

341 Harmonic Sum ∑ i=1..n 1 / i = 1 / 1 + 1 / 2 + 1 / 3 + 1 / 4 + 1 / 5 + …+ 1 / n =  (log(n))

342 Adding Made Easy

343 If the terms f(i) decrease towards zero sufficiently quickly, then the sum will be a constant. The classic example ∑ i=0..n 1 / 2 i = 1 + 1 / 2 + 1 / 4 + 1 / 8 + … = 2. f(n)  n -1-Ω(1)  ∑ i=1..n f(i) = θ(1) Bounded Tail:

344 If f(i) decays even faster, then the tail of the sum is more bounded. 2i2i ∑ i=1..n 2 2 = θ(1). 1 f(n)  n -1-Ω(1)  ∑ i=1..n f(i) = θ(1) Bounded Tail:

345 Upper Extreme: ∑ i=1..n 1 / i 1.0001 = θ(1) f(n)  n -1-Ω(1)  ∑ i=1..n f(i) = θ(1) Bounded Tail:

346 Upper Extreme: ∑ i=1..n 1 / i 1.0001 = θ(1) No Lower Extreme: 2i2i ∑ i=1..n 2 2 = θ(1). 1 All functions in between. f(n)  n -1-Ω(1)  ∑ i=1..n f(i) = θ(1) Bounded Tail:

347 Bounded Tail If f(n) = c b an n d log e (n) c  ? a  ? b  ? d  ? e  ? f(n) = Ω, θ ? then ∑ i=1..n f(i) = θ(1). > 0  (- ,  ) Conclusion or b  1 c n d log e (n) c  a  b  b an

348 Bounded Tail If f(n) = c n d log e (n) c  ? a  b = 1 d  ? e  ? f(n) = Ω, θ ? then ∑ i=1..n f(i) = θ(1).  n -1-Ω(1) > 0 < -1 (- ,  ) Conclusion

349 Adding Made Easy Done

350 Adding Made Easy Missing Functions 1 / nlogn logn / n

351 Adding Made Easy Geometric Like: If f(n)  2 Ω(n), then ∑ i=1..n f(i) = θ(f(n)). Arithmetic Like: If f(n) = n θ(1)-1, then ∑ i=1..n f(i) = θ(n · f(n)). Harmonic: If f(n) = 1 / n, then ∑ i=1..n f(i) = log e n + θ(1). Bounded Tail: If f(n)  n -1-Ω(1), then ∑ i=1..n f(i) = θ(1). ( For +, -, , , exp, log functions f(n) ) This may seem confusing, but it is really not. It should help you compute most sums easily.

352 Recurrence Relations T(1) = 1 T(n) = a T(n/b) + f(n)

353 Recurrence Relations  Time of Recursive Program Recurrence relations arise from the timing of recursive programs. Let T(n) be the # of “Hi”s on an input of “size” n. procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b ) ?

354 Given size 1, the program outputs T(1)=1 Hi’s. Recurrence Relations  Time of Recursive Program procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b )

355 Given size n, the stackframe outputs f(n) Hi’s. Recurrence Relations  Time of Recursive Program procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b )

356 Recursing on an instances of size n/b generates T(n/b) “Hi”s. Recurrence Relations  Time of Recursive Program procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b )

357 Recursing a times generates a·T(n/b) “Hi”s. Recurrence Relations  Time of Recursive Program procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b )

358 For a total of T(1) = 1 T(n) = a·T(n/b) + f(n) “Hi”s. Recurrence Relations  Time of Recursive Program procedure Eg(I n ) n = |I n | if(n  1) then put “Hi” else loop i=1..f(n) put “Hi” loop i=1..a I n/b = I n cut in b pieces Eg(I n/b )

359 Solving Technique 1 Guess and Verify Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: G(n) = 2n 2 – n Verify: Left Hand SideRight Hand Side T(1) = 2(1) 2 – 1 T(n) = 2n 2 – n 1 4T(n/2) + n = 4 [2( n / 2 ) 2 – ( n / 2 )] + n = 2n 2 – n

360 Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: G(n) = an 2 + bn + c Verify: Left Hand SideRight Hand Side T(1) = a+b+c T(n) = an 2 +bn+c 1 4T(n/2) + n = 4 [a ( n / 2 ) 2 + b ( n / 2 ) +c] + n = an 2 +(2b+1)n + 4c Solving Technique 2 Guess Form and Calculate Coefficients

361 Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: G(n) = an 2 + bn + 0 Verify: Left Hand SideRight Hand Side T(1) = a+b+c T(n) = an 2 +bn+c 1 4T(n/2) + n = 4 [a ( n / 2 ) 2 + b ( n / 2 ) +c] + n = an 2 +(2b+1)n + 4c Solving Technique 2 Guess Form and Calculate Coefficients c=4c c=0

362 Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: G(n) = an 2 - 1n + 0 Verify: Left Hand SideRight Hand Side T(1) = a+b+c T(n) = an 2 +bn+c 1 4T(n/2) + n = 4 [a ( n / 2 ) 2 + b ( n / 2 ) +c] + n = an 2 +(2b+1)n + 4c Solving Technique 2 Guess Form and Calculate Coefficients b=2b+1 b=-1

363 Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: G(n) = an 2 - 1n + 0 Verify: Left Hand SideRight Hand Side T(1) = a+b+c T(n) = an 2 +bn+c 1 4T(n/2) + n = 4 [a ( n / 2 ) 2 + b ( n / 2 ) +c] + n = an 2 +(2b+1)n + 4c Solving Technique 2 Guess Form and Calculate Coefficients a=a

364 Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) + n Guess: G(n) = 2n 2 - 1n + 0 Verify: Left Hand SideRight Hand Side T(1) = a+b+c T(n) = an 2 +bn+c 1 4T(n/2) + n = 4 [a ( n / 2 ) 2 + b ( n / 2 ) +c] + n = an 2 +(2b+1)n + 4c Solving Technique 2 Guess Form and Calculate Coefficients a+b+c=1 a-1+0=1 a=2

365 Recurrence Relation: T(1) = 1 & T(n) = aT(n/b) + f(n) Solving Technique 3 Approximate Form and Calculate Exponent which is bigger? Guess

366 Recurrence Relation: T(1) = 1 & T(n) = aT(n/b) + f(n) Guess: aT(n/b) << f(n) Simplify: T(n)  f(n) Solving Technique 3 Calculate Exponent In this case, the answer is easy. T(n) =  (f(n))

367 Recurrence Relation: T(1) = 1 & T(n) = aT(n/b) + f(n) Guess: aT(n/b) >> f(n) Simplify: T(n)  aT(n/b) Solving Technique 3 Calculate Exponent In this case, the answer is harder.

368 Recurrence Relation: T(1) = 1 & T(n) = aT(n/b) Guess: G(n) = cn  Verify: Left Hand SideRight Hand Side T(n) = cn  aT(n/b) = a [c ( n / b )  ] = c a b  n  Solving Technique 3 Calculate Exponent ( log a / log b ) = cn 1 = a b -  b  = a  log b = log a  = log a / log b

369 Recurrence Relation: T(1) = 1 & T(n) = 4T(n/2) Guess: G(n) = cn  Verify: Left Hand SideRight Hand Side T(n) = cn  aT(n/b) + f(n) = c [a ( n / b )  ] = c a b  n  Solving Technique 3 Calculate Exponent 1 = a b -  b  = a  log b = log a  = log a / log b ( log a / log b ) = cn ( log 4 / log 2 ) = cn 2

370 Logs log 27 log 9 =

371 Logs log ? 27 log ? 9 = Which base?

372 Logs log 2 27 log 2 9 = It does not matter. log 2 c · log c 27 · log c 9 = log c 27 log c 9 Which is easiest?

373 Logs log 3 27 log 3 9 = log 3 3 3 log 3 3 2 = 3 2 Please no calculators in exams. And I wont ask log 25 log 9

374 Recurrence Relation: Solving Technique 3 Calculate Exponent If bigger then T(n) =  (f(n)) If bigger then ( log a / log b ) T(n) =  (n ) And if aT(n/b)  f(n) what is T(n) then? T(1) = 1 & T(n) = aT(n/b) + f(n)

375 4: Decorate The Tree T(n) = a T(n/b) + f(n) T(1) = 1 & T(n) = aT(n/b) + f(n) f(n) T(n/b) T(n) = T(1) T(1) = 1 1 = a

376 f(n) T(n/b) T(n) = a

377 f(n) T(n/b) T(n) = a f(n/b) T(n/b 2 ) a

378 f(n) T(n) = a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a

379 11111111111111111111111111111111...... 111111111111111111111111111111111 f(n) T(n) = a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a f(n/b) T(n/b 2 ) a

380 Evaluating: T(n) = aT(n/b)+f(n) Level 0 1 2 i h

381 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0 1 2 i h

382 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 2 i h

383 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 i h

384 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i h

385 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i n/b i h n/b h

386 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i n/b i h n/b h

387 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i n/b i h n/b h = 1 base case

388 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i n/b i h n/b h = 1

389 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size 0n 1 n/b 2 n/b 2 i n/b i h = log n / log b n/b h = 1 b h = n h log b = log n h = log n / log b

390 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame 0n 1 n/b 2 n/b 2 i n/b i h = log n / log b 1

391 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame 0n f(n) 1 n/bf(n/b) 2 n/b 2 f(n/b 2 ) i n/b i f(n/b i ) h = log n / log b 1T(1)

392 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames 0n f(n) 1 n/bf(n/b) 2 n/b 2 f(n/b 2 ) i n/b i f(n/b i ) h = log n / log b n/b h T(1)

393 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames 0n f(n) 1 1 n/bf(n/b) a 2 n/b 2 f(n/b 2 ) a2a2 i n/b i f(n/b i ) aiai h = log n / log b n/b h T(1) ahah

394 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames 0n f(n) 1 1 n/bf(n/b) a 2 n/b 2 f(n/b 2 ) a2a2 i n/b i f(n/b i ) aiai h = log n / log b n/b h T(1) ahah

395 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames 0n f(n) 1 1 n/bf(n/b) a 2 n/b 2 f(n/b 2 ) a2a2 i n/b i f(n/b i ) aiai h = log n / log b n/b h T(1) ahah a h = a = n log n / log b log a / log b

396 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames Work in Level 0n f(n) 1 1 n/bf(n/b) a 2 n/b 2 f(n/b 2 ) a2a2 i n/b i f(n/b i ) aiai h = log n / log b n/b h T(1) n log a / log b

397 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames Work in Level 0n f(n) 1 1 · f(n) 1 n/bf(n/b) a a · f(n/b) 2 n/b 2 f(n/b 2 ) a2a2 a 2 · f(n/b 2 ) i n/b i f(n/b i ) aiai a i · f(n/b i ) h = log n / log b n/b h T(1) n log a / log b n · T(1) log a / log b Total Work T(n) = ∑ i=0..h a i  f(n/b i )

398 Evaluating: T(n) = aT(n/b)+f(n) = ∑ i=0..h a i  f(n/b i ) If a Geometric Sum ∑ i=0..n x i  θ(max(first term, last term))

399 Evaluating: T(n) = aT(n/b)+f(n) Level Instance size Work in stack frame # stack frames Work in Level 0n f(n) 1 1 · f(n) 1 n/bf(n/b) a a · f(n/b) 2 n/b 2 f(n/b 2 ) a2a2 a 2 · f(n/b 2 ) i n/b i f(n/b i ) aiai a i · f(n/b i ) h = log n / log b n/b h T(1) n log a / log b n · T(1) log a / log b Dominated by Top Level or Base Cases

400 Evaluating: T(n) = aT(n/b)+f(n) Is the sum Geometric? Simplify by letting f(n) = n c log k n T(n) = ∑ i=0..h a i  f(n/b i ) = ∑ i=0..h a i  (n/b i ) c  log k (n/b i ) = n c ∑ i=0..h 2 [log a - c log b] · i  log k (n/b i ) Geometrically IncreasingArithmetic SumGeometrically Decreasing Sum = θ(last term) = θ( ) Sum = θ(any term  # terms) = θ(f(n)  log n) Sum = θ(first term) = θ(f(n)). log a - c log b > 0log a - c log b = 0log a - c log b < 0 c < log a / log b c = log a / log b & k<-1 c = log a / log b & k>-1 c > log a / log b n log a / log b (n/..) c = n c a i = 2 [log a]·i (.. /b i ) c = 2 [-c log b]·i = n c ∑ i=0..h 2 -d i  log k (n/b i )

401 Evaluating: T(n) = aT(n/b)+f(n) Is the sum Geometric? Simplify by letting f(n) = n c log k n T(n) = ∑ i=0..h a i  f(n/b i ) = ∑ i=0..h a i  (n/b i ) c  log k (n/b i ) = n c ∑ i=0..h 2 [log a - c log b] · i  log k (n/b i ) Geometrically IncreasingArithmetic SumGeometrically Decreasing Sum = θ(last term) = θ( ) Sum = θ(any term  # terms) = θ(f(n)  log n) Sum = θ(first term) = θ(f(n)). log a - c log b > 0log a - c log b = 0log a - c log b < 0 c < log a / log b c = log a / log b & k<-1 c = log a / log b & k>-1 c > log a / log b n log a / log b k =-1  log k ( n /b i ) = 1 / (logn - i  logb)  1 / i (backwards) = n c ∑ i=0..h 2 0 i  log k (n/b i ) (back wards) de

402 Evaluating: T(n) = aT(n/b)+f(n) Is the sum Geometric? Simplify by letting f(n) = n c log k n T(n) = ∑ i=0..h a i  f(n/b i ) = ∑ i=0..h a i  (n/b i ) c  log k (n/b i ) = n c ∑ i=0..h 2 [log a - c log b] · i  log k (n/b i ) Geometrically IncreasingArithmetic SumGeometrically Decreasing Sum = θ(last term) = θ( ) Sum = θ(any term  # terms) = θ(f(n)  log n) Sum = θ(first term) = θ(f(n)). log a - c log b > 0log a - c log b = 0log a - c log b < 0 c < log a / log b c = log a / log b & k<-1 c = log a / log b & k>-1 c > log a / log b n log a / log b = n c ∑ i=0..h 2 +d i  log k (n/b i )

403 Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n

404 Time for top level?: Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n

405 Time for top level: f(n) = n 3 / log 5 n Time for base cases?: Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n

406 Time for top level: f(n) = n 3 / log 5 n Time for base cases: Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n θ(n ) = log a / log b θ(n ) log ? / log ?

407 Time for top level: f(n) = n 3 / log 5 n Time for base cases: Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n θ(n ) = log a / log b θ(n ) = ? log 4 / log 2

408 Time for top level: f(n) = n 3 / log 5 n Time for base cases: Dominated?: c = ? Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 log a / log b = ?

409 Time for top level: f(n) = n 3 / log 5 n Time for base cases: Dominated?: c = 3 > 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = ?

410 Time for top level: f(n) = n 3 / log 5 n Time for base cases: Dominated?: c = 3 > 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 3 / log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = θ(top) = θ(f(n)) = θ(n 3 / log 5 n ).

411 Time for top level: f(n) = 2 n Time for base cases: Dominated?: c = ? > 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ 2 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = θ(top) = θ(f(n)) = θ(2 n ). bigger >big The time is even more dominated by the top level of recursion

412 Time for top level: f(n) = n log 5 n Time for base cases: Dominated?: c = ? 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = ? 1< = θ(base cases) = θ(n ) = θ(n 2 ). log a / log b

413 Time for top level: f(n) = log 5 n Time for base cases: Dominated?: c = ? 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = ? 0< = θ(base cases) = θ(n ) = θ(n 2 ). log a / log b

414 Time for top level: f(n) = n 2 Time for base cases: Dominated?: c = ? 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 2 θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 Hence, T(n) = ? 2= = θ(f(n) log(n) ) = θ(n 2 log(n) ).

415 Time for top level: f(n) = n 2 log 5 n Time for base cases: Dominated?: c = 2 = 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 2 log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 d = ? 5 ? Hence, T(n) = ? > -1 = θ(f(n) log(n) ) = θ(n 2 log 6 (n) ).

416 Time for top level: f(n) = n 2 / log 5 n Time for base cases: Dominated?: c = 2 = 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 2 / log 5 n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 d = ? Hence, T(n) = ? -5 ?< -1 = θ(base cases) = θ(n ) = θ(n 2 ). log a / log b

417 Time for top level: f(n) = n 2 / log n Time for base cases: Dominated?: c = 2 = 2 = log a / log b Evaluating: T(n) = 4T(n/2)+ n 2 / log n θ(n ) = log a / log b θ(n ) = θ(n 2 ) log 4 / log 2 d = ? Hence, T(n) = ? -1 ?= -1 = θ(n c loglog n ) = θ(n 2 loglog n ). Did not do before.

418 Sufficiently close to: T(n) = 4T(n/2)+ n 3 = θ(n 3 ). T 2 (n) = ? Evaluating: T 2 (n) = 4 T 2 (n/2+n 1/2 )+ n 3 θ(n 3 ).

419 Evaluating: T(n) = aT(n-b)+f(n) h = ? n-hb n-ib n-2b n-b T(0) f(n-ib) f(n-2b) f(n-b) f(n) n Work in Level # stack frames Work in stack frame Instance size a aiai a2a2 a 1 n/bn/b a · T(0) a i · f(n-ib) a 2 · f(n-2b) a · f(n-b) 1 · f(n) n/bn/b |base case| = 0 = n-hb h = n / b i 2 1 0 Level Likely dominated by base cases Exponential

420 Evaluating: T(n) = 1T(n-b)+f(n) h = ? Work in Level # stack frames Work in stack frame Instance size 1 1 1 1 1 f(0) f(n-ib) f(n-2b) f(n-b) f(n) n-b n n-hb n-ib n-2b T(0) f(n-ib) f(n-2b) f(n-b) f(n) h = n / b i 2 1 0 Level Total Work T(n) = ∑ i=0..h f(b·i) = θ(f(n)) or θ(n · f(n))

421 End Restart Relevant Mathematics Iterative Algorithms

422 End of math review More Math


Download ppt "Relevant Mathematics Jeff Edmonds York University COSC 3101 Lecture skipped. Lecture skipped. (Done when needed) Logic Quantifiers The Time Complexity."

Similar presentations


Ads by Google