1. J. Elder COSC 3101N Thinking about Algorithms Abstractly Lecture 2. Relevant Mathematics: Classifying Functions Input Size Time f(i) = n  (n)

2. J. Elder COSC 3101N Assumed Knowledge Read J. Edmonds, How to Think About Algorithms (HTA): Section 1.1 Existential and Universal Quantifiers Section 1.2 Logarithms and Exponentials

3. J. Elder COSC 3101N Classifying Functions Giving an idea of how fast a function grows without going into too much detail.

4. J. Elder COSC 3101N Which are more alike?

5. J. Elder COSC 3101N Which are more alike? Mammals

6. J. Elder COSC 3101N Which are more alike?

7. J. Elder COSC 3101N Which are more alike? Dogs

8. J. Elder COSC 3101N Classifying Animals Vertebrates BirdsMammals Reptiles Fish Dogs Giraffe

9. J. Elder COSC 3101N Classifying Functions T(n)T(n) 101001,00010,000 log n 36913 n 1/2 31031100 101001,00010,000 n log n 306009,000130,000 n2n2 10010,00010 6 10 8 n3n3 1,00010 6 10 9 10 12 2n2n 1,02410 30 10 300 10 3000 Note: The universe contains approximately 10 50 particles. n

10. J. Elder COSC 3101N Which are more alike? n 1000 n2n2 2n2n

11. J. Elder COSC 3101N Which are more alike? Polynomials n 1000 n2n2 2n2n

12. J. Elder COSC 3101N Which are more alike? 1000n 2 3n 2 2n 3

13. J. Elder COSC 3101N Which are more alike? Quadratic 1000n 2 3n 2 2n 3

14. J. Elder COSC 3101N Classifying Functions? Functions

15. J. Elder COSC 3101N Classifying Functions Functions Constant Logarithmic Poly Logarithmic Polynomial Exponential ExpDouble Exp (log n) 5 n5n5 2 5n 5 log n5 2 n5n5 2 5n 2

16. J. Elder COSC 3101N Classifying Functions? Polynomial

17. J. Elder COSC 3101N Classifying Functions Polynomial LinearQuadratic Cubic ? 5n 2 5n 5n 3 5n 4

18. J. Elder COSC 3101N Logarithmic log 10 n = # digits to write n log 2 n = # bits to write n = 3.32 log 10 n log(n 1000 ) = 1000 log(n) Differ only by a multiplicative constant.

19. J. Elder COSC 3101N Poly Logarithmic (log n) 5 = log 5 n

20. J. Elder COSC 3101N Logarithmic << Polynomial For sufficiently large n log 1000 n << n 0.001

21. J. Elder COSC 3101N Linear << Quadratic For sufficiently large n 10000 n << 0.0001 n 2

22. J. Elder COSC 3101N Polynomial << Exponential For sufficiently large n n 1000 << 2 0.001 n

23. J. Elder COSC 3101N Ordering Functions Functions Constant Logarithmic Poly Logarithmic Polynomial Exponential ExpDouble Exp (log n) 5 n5n5 2 5n 5 log n5 2 n5n5 2 5n 2 <<

24. J. Elder COSC 3101N Which Functions are Constant? 5 1,000,000,000,000 0.0000000000001 -5 0 8 + sin(n) Yes No Yes

25. J. Elder COSC 3101N Which Functions are “Constant”? 5 1,000,000,000,000 0.0000000000001 -5 0 8 + sin(n) Yes No Yes Lies in between 7 9 The running time of the algorithm is a “Constant” It does not depend significantly on the size of the input.

26. J. Elder COSC 3101N Which Functions are Quadratic? n 2 … ?

27. J. Elder COSC 3101N Which Functions are Quadratic? n 2 0.001 n 2 1000 n 2 Some constant times n 2.

28. J. Elder COSC 3101N Which Functions are Quadratic? n 2 0.001 n 2 1000 n 2 5n 2 + 3n + 2log n

29. J. Elder COSC 3101N Which Functions are Quadratic? n 2 0.001 n 2 1000 n 2 5n 2 + 3n + 2log n Lies in between

30. J. Elder COSC 3101N Which Functions are Quadratic? n 2 0.001 n 2 1000 n 2 5n 2 + 3n + 2log n Ignore low-order terms Ignore multiplicative constants. Ignore "small" values of n. Write θ(n 2 ).

31. J. Elder COSC 3101N Which Functions are Polynomial? n 5 … ?

32. J. Elder COSC 3101N Which Functions are Polynomial? n c n 0.0001 n 10000 n to some constant power.

33. J. Elder COSC 3101N Which Functions are Polynomial? n c n 0.0001 n 10000 5n 2 + 8n + 2log n 5n 2 log n 5n 2.5

34. J. Elder COSC 3101N Which Functions are Polynomial? n c n 0.0001 n 10000 5n 2 + 8n + 2log n 5n 2 log n 5n 2.5 Lie in between

35. J. Elder COSC 3101N Which Functions are Polynomials? n c n 0.0001 n 10000 5n 2 + 8n + 2log n 5n 2 log n 5n 2.5 Ignore low-order terms Ignore power constant. Ignore "small" values of n. Write n θ(1)

36. J. Elder COSC 3101N Which Functions are Exponential? 2 n … ?

37. J. Elder COSC 3101N Which Functions are Exponential? 2 n 2 0.0001 n 2 10000 n n times some constant power raised to the power of 2.

38. J. Elder COSC 3101N Which Functions are Exponential? 2 n 2 0.0001 n 2 10000 n 8 n 2 n / n 100 2 n · n 100 too small? too big?

39. J. Elder COSC 3101N Which Functions are Exponential? 2 n 2 0.0001 n 2 10000 n 8 n 2 n / n 100 2 n · n 100 = 2 3n > 2 0.5n < 2 2n Lie in between

40. J. Elder COSC 3101N Which Functions are Exponential? 2 n 2 0.0001 n 2 10000 n 8 n 2 n / n 100 2 n · n 100 = 2 3n > 2 0.5n < 2 2n 2 0.5n > n 100 2 n = 2 0.5n · 2 0.5n > n 100 · 2 0.5n 2 n / n 100 > 2 0.5n

41. J. Elder COSC 3101N Which Functions are Exponential? Ignore low-order terms Ignore base. Ignore "small" values of n. Ignore polynomial factors. Write 2 θ(n) 2 n 2 0.0001 n 2 10000 n 8 n 2 n / n 100 2 n · n 100

42. J. Elder COSC 3101N Classifying Functions Functions Constant Logarithmic Poly Logarithmic PolynomialExponential Exp Double Exp (log n) θ(1) n θ(1) 2 θ(n) θ(log n)θ(1) 2 n θ(1) 2 θ(n) 2

43. J. Elder COSC 3101N Notations Theta f(n) = θ(g(n))f(n) ≈ c g(n) BigOh f(n) = O(g(n))f(n) ≤ c g(n) Omega f(n) = Ω(g(n))f(n) ≥ c g(n) Little Oh f(n) = o(g(n))f(n) << c g(n) Little Omega f(n) = ω(g(n))f(n) >> c g(n)

44. J. Elder COSC 3101N Definition of Theta f(n) = θ(g(n))

45. J. Elder COSC 3101N Definition of Theta f(n) is sandwiched between c 1 g(n) and c 2 g(n) f(n) = θ(g(n))

46. J. Elder COSC 3101N Definition of Theta f(n) is sandwiched between c 1 g(n) and c 2 g(n) for some sufficiently small c 1 (= 0.0001) for some sufficiently large c 2 (= 1000) f(n) = θ(g(n))

47. J. Elder COSC 3101N Definition of Theta For all sufficiently large n f(n) = θ(g(n))

48. J. Elder COSC 3101N Definition of Theta For all sufficiently large n For some definition of “sufficiently large” f(n) = θ(g(n))

49. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n 2 )

50. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) c 1 ·n 2  3n 2 + 7n + 8  c 2 ·n 2 ??

51. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 34

52. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·1 2  3·1 2 + 7·1 + 8  4·1 2 1 False

53. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·7 2  3·7 2 + 7·7 + 8  4·7 2 7 False

54. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·8 2  3·8 2 + 7·8 + 8  4·8 2 8 True

55. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·9 2  3·9 2 + 7·9 + 8  4·9 2 9 True

56. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·10 2  3·10 2 + 7·10 + 8  4·10 2 10 True

57. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 ?

58. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 n  8 8

59. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 True n  8

60. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 7n + 8  1 · n 2 7 + 8 / n  1 · n n  8 True

61. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n 2 ) 3·n 2  3n 2 + 7n + 8  4·n 2 348 n  8 True

62. J. Elder COSC 3101N Definition of Theta n 2 = θ(n 3 )

63. J. Elder COSC 3101N Definition of Theta n 2 = θ(n 3 ) c 1 · n 3  n 2  c 2 · n 3 ??

64. J. Elder COSC 3101N Definition of Theta n 2 = θ(n 3 ) 0 · n 3  n 2  c 2 · n 3 0 True, but ?

65. J. Elder COSC 3101N Definition of Theta n 2 = θ(n 3 ) 0 Constants c1 and c2 must be positive! False

66. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n)

67. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) c 1 ·n  3n 2 + 7n + 8  c 2 ·n ??

68. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) 3·n  3n 2 + 7n + 8  100·n 3 100

69. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) 3·n  3n 2 + 7n + 8  100·n ?

70. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) 3·100  3·100 2 + 7·100 + 8  100·100 100 False

71. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) 3·n  3n 2 + 7n + 8  10,000·n 3 10,000

72. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) 3·10,000  3·10,000 2 + 7·10,000 + 8  100·10,000 10,000 False

73. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) What is the reverse statement?

74. J. Elder COSC 3101N Understand Quantifiers!!! SamMary BobBeth John Marilyn Monroe FredAnn SamMary BobBeth John Marilyn Monroe FredAnn  b,  Loves(b, MM)  b, Loves(b, MM)  b,  Loves(b, MM)]  b, Loves(b, MM)]

75. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) The reverse statement

76. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) 3n 2 + 7n + 8 > 100·n 3 100 ?

77. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) 3 · 100 2 + 7 · 100 + 8 > 100 · 100 3 100 True 100

78. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) 3n 2 + 7n + 8 > 10,000·n 3 10,000 ?

79. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) 3 · 10,000 2 + 7 · 10,000 + 8 > 10,000 · 10,000 3 10,000 True 10,000

80. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) 3n 2 + 7n + 8 > c 2 ·n c1c1 c2c2 ?

81. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) 3 · c 2 2 + 7 · c 2 + 8 > c 2 · c 2 c1c1 c2c2 True c2c2

82. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) 3n 2 + 7n + 8 > c 2 ·n c1c1 c2c2 ? n0n0

83. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = θ(n) 3 · c 2 2 + 7 · c 2 + 8 > c 2 · c 2 c1c1 max(c 2,n 0 ) True c2c2 n0n0

84. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = g(n) θ(1)  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 3n 2 + 7n + 8 = n θ(1)

85. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = n θ(1) n c1  3n 2 + 7n + 8  n c2  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 ??

86. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = n θ(1) n 2  3n 2 + 7n + 8  n 3  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 23

87. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = n θ(1) n 2  3n 2 + 7n + 8  n 3  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 23? n  ?

88. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = n θ(1) n 2  3n 2 + 7n + 8  n 3  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 235 n  5

89. J. Elder COSC 3101N Definition of Theta 3n 2 + 7n + 8 = n θ(1) n 2  3n 2 + 7n + 8  n 3 True  c 1, c 2, n 0,  n  n 0 g(n) c1  f(n)  g(n) c2 235 n  5

90. J. Elder COSC 3101N Order of Quantifiers f(n) = θ(g(n)) ?

91. J. Elder COSC 3101N Understand Quantifiers!!! One girl Could be a separate girl for each boy. SamMary BobBeth John Marilin Monro FredAnn SamMary BobBeth John Marilin Monro FredAnn

92. J. Elder COSC 3101N Order of Quantifiers No! It cannot be a different c 1 and c 2 for each n. f(n) = θ(g(n))

93. J. Elder COSC 3101N Other Notations Theta f(n) = θ(g(n))f(n) ≈ c g(n) BigOh f(n) = O(g(n))f(n) ≤ c g(n) Omega f(n) = Ω(g(n))f(n) ≥ c g(n) Little Oh f(n) = o(g(n))f(n) << c g(n) Little Omega f(n) = ω(g(n))f(n) >> c g(n)

94. J. Elder COSC 3101N BigOh Notation n 2 = O(n 3 ) n 3 = O(n 2 )

95. J. Elder COSC 3101N BigOh Notation O(n 2 ) = O(n 3 ) O(n 3 ) = O(n 2 ) Odd Notation f(n) = O(g(n)) Standard f(n) ≤ O(g(n)) Stresses one function dominating another. f(n)  O(g(n)) Stress function is member of class.