1. 1 CSSE 350 Automata, Formal Languages, and Computability

2. 2 Mathematical Preliminaries and Background Sets Relations Functions Logic Graphs Trees Proof Techniques Languages and Strings Automaton

3. 3 Sets

4. 4 A set is a collection of elements, having a property that characterizes those elements. SETS We write Belongs To and Does not Belong To as:

5. 5 Set Representations One way is to enumerate the elements completely. All the elements belonging to the set are explicitly given. –For example, A = {1,2,3,4,5} Alternate way is to give the properties that characterize the elements of the set. –For example, B = {x | x is a positive integer less than or equal to 5} –Or B = {x : x is a positive integer less than or equal to 5} Ellipses may be used in the set specification if the meaning is clear. For example, C = { a, b, …, z}, which stands for all the lower-case letters of the English alphabet

6. 6 Set Representations C = { a, b, c, d, e, f, g, h, i, j, k } C = { a, b, …, k } S = { 2, 4, 6, … } S = { j : j > 0, and j = 2k for some k>0 } S = { j : j is nonnegative and even } finite set infinite set

7. 7 A = { 1, 2, 3, 4, 5 } Universal Set: all possible elements U = { 1, …, 10 } 1 2 3 4 5 A U 6 7 8 9 10

8. 8 Set Operations A = { 1, 2, 3 } B = { 2, 3, 4, 5} Union A U B = { 1, 2, 3, 4, 5 } Intersection A B = { 2, 3 } Difference A - B = { 1 } B - A = { 4, 5 } U A B 2 3 1 4 5 2 3 1 Venn diagrams

9. 9 A Complement Universal set = {1, …, 7} A = { 1, 2, 3 } A = { 4, 5, 6, 7} 1 2 3 4 5 6 7 A A = A

10. 10 0 2 4 6 1 3 5 7 even { even integers } = { odd integers } odd Integers

11. 11 DeMorgan’s Laws A U B = A B U A B = A U B U

12. 12 Empty, Null Set: = { } S U = S S = S - = S - S = U = Universal Set

13. 13 Subset A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 } A B U Proper Subset:A B U A B

14. 14 Disjoint Sets A = { 1, 2, 3 } B = { 5, 6} A B = U AB

15. 15 Set Cardinality For finite sets A = { 2, 5, 7 } |A| = 3 (set size)

16. 16 Powersets A powerset is a set of sets Powerset of S = the set of all the subsets of S S = { a, b, c } 2 S = {, {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} } Observation: | 2 S | = 2 |S| ( 8 = 2 3 )

17. 17 Cartesian Product A = { 2, 4 } B = { 2, 3, 5 } A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) } |A X B| = |A| |B| Generalizes to more than two sets A X B X … X Z

18. 18 Set Identities A, B, C represent arbitrary sets and ø is the empty set and U is the Universal Set. The Commutative laws: A  B = B  A A  B = B  A The Associative laws: A  (B  C) = (A  B)  C A  (B  C) = (A  B)  C The Distributive laws: A  (B  C) = (A  B)  (A  C) A  (B  C) = (A  B)  (A  C)

19. 19 The Idempotent laws: A  A = A A  A = A The Absorptive laws: A  (A  B) = A A  (A  B) = A The De Morgan laws: (A  B)' = A'  B' (A  B)' = A'  B' Other laws involving Complements: ( A' )' = A A  A' = ø A  A' = U

20. 20 Other laws involving the empty set A  ø = A A  ø = ø Other laws involving the Universal Set: A  U = U A  U = A

21. 21 RELATIONS

22. 22 Relations Let A and B be sets. A binary relation from A into B is any subset of the Cartesian product A  B. For example: let A = {2, 3, 5, 6}, and B = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}, and a relation R from A into B by (a, b) R if and only if 2a = b So, R = {(2, 4), (3, 6), (5, 10), (6, 12)}.

23. 23 A typical element in R is an ordered pair (x, y). In some cases R can be described by actually listing the pairs which are in R, as in the previous example. This may not be convenient if R is relatively large. Other notations are used depending on the past practice. Consider the following relation on real numbers. R = {(x, y) | y is the square of x} and S = {(x, y) | x <= y}. R could be more naturally expressed as R(x) = x 2 or R(x) =y where y = x 2.

24. 24 RELATIONS R = {(x 1, y 1 ), (x 2, y 2 ), (x 3, y 3 ), …} x i R y i e. g. if R = ‘>’: 2 > 1, 3 > 2, 3 > 1

25. 25 Relation on a Set A relation from a set A into itself is called a relation on A. For example, let A = {1, 2, 3, 4, 5, 6}, and a relation R from A to A itself by (a, b) R if and only if a is a divisor of b So, R = {(1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6), (2, 2), (2, 4), (2, 6), (3, 3), (3, 6), (4, 4), (5, 5), (6, 6)}. Let A be a set of people and let P = {(a, b) | a  A  b  A  a is a child of b}. Then P is a relation on A which we might call a parent-child relation.

26. 26 Composition Let R be a relation from a set A into set B, and S be a relation from set B into set C. The composition of R and S, written as RS, is the set of pairs of the form (a, c)  A  C, where (a, c)  RS if and only if there exists b B such that (a, b)  R and (b, c)  S. For example PP, where P is the parent-child relation given, is the composition of P with itself and it is a relation which we know as grandparent-grandchild relation.

27. 27 Properties of Relations Assume R is a relation on set A; in other words, R  A  A. Let us write a R b to denote (a, b) R. 1. Reflexive: R is reflexive if for every a  A, a R a. 2. Symmetric: R is symmetric if for every a and b in A, if aRb, then bRa. 3. Transitive: R is transitive if for every a, b and c in A, if aRb and bRc, then aRc. 4. Equivalence: R is an equivalence relation on A if R is reflexive, symmetric and transitive. For example, the equality relation (=) on a set of numbers such as {1, 2, 3} is an equivalence relation.

28. 28 Equivalence Relations Reflexive: x R x Symmetric: x R y y R x Transitive: x R y and y R z x R z Example: R = ‘=‘ x = x x = y y = x x = y and y = z x = z

29. 29 FUNCTIONS

30. 30 Functions A function is something that associates each element of a set with an element of another set (which may or may not be the same as the first set). Or a function is a rule that assigns to elements of one set a unique element of another set. For example, a social security number uniquely identifies a person, in this case, to each member of the set of social security number, some member of the set of person is assigned.

31. 31 FUNCTIONS domain 1 2 3 a b c range f : A -> B A B If A = domain then f is a total function otherwise f is a partial function f(1) = a 4 5

32. 32 Growth Rates of Functions Let f(n) and g(n) to be two functions whose domain is a subset of the positive integers If there exists a positive constant c such that for all sufficiently large n f(n) ≤ cg(n) f has the order at most g f(n) = O(g(n)) cg(n) f(n)

33. 33 If there exists a positive constant c such that for all sufficiently large n f(n)  cg(n) f has the order at least g f(n) =  (g(n)) cg(n) f(n)

34. 34 If there exist positive constant c 1 and c 2, such that for all sufficiently large n c 1 g(n) ≤ f(n) ≤ c 2 g(n) f and g have the same order of magnitude f(n) =  (g(n)) c 1 g(n) f(n) c 2 g(n)

35. 35 Examples Example 1: 2n is Θ(n) f(n) = 2n, g(n) = n For all integers n ≥ 0, 2n ≤ 2n, which is f(n) ≤ 2g(n); so constant c ≥ 2 suffices for the definition of O. For all integers n ≥ 0, 2n ≥ n, which is f(n) ≥ 1g(n); so any 0 < c ≤ 2 suffices for the definition of Ω. Since 2n is both O(n) and Ω(n), it is Θ(n).

36. 36 Example 2: n is O(n 2 ) but not Ω(n 2 ) f(n) = n, g(n) = n 2 For all integers n ≥ 0, n ≤ n 2 ; which is f(n) ≤ 1g(n); so constant c ≥ 1 suffices for the definition of O. However, n cannot be Ω(n 2 ) since no matter how small c>0 is chosen, for all n > 1/c, n = c(1/c)n < cnn = cn 2.

37. 37 Logic Propositional Logic & Predicate Logic

38. 38 Propositional Logic Proposition: a declarative statement having a specific truth-value, true or false. For example: 8 is an even number.  True 4 is a negative number.  False

39. 39 Two or more propositions can be combined together to make compound propositions with the help of logical connectives. For example: above two propositions can be used to make a compound proposition using any of the logical connectives. 8 is an even number AND 4 is a negative number.  False 8 is an even number OR 4 is a negative number.  True For the first compound proposition to be true both the propositions have to be true as the connective is AND. As OR is the connective for the second one, if either of the propositions is true, the truth value of the compound proposition is true.

40. 40 logical Connectives Conjunction: The compound proposition truth-value is true if and only if all the constituent propositions hold true. It is represented as " ^ ". Truth table for two individual propositions p and q with conjunction is given below. pq p qp q TTT TFF FTF FFF

41. 41 Disjunction: This is logical "or" read as either true value of the individual propositions. Truth table is given below. pqp V q TTT TFT FTT FFF

42. 42 Negation: This is the logical "negation" and it is expressed by as ¬p for "not p". Truth table is given below. p ¬ p TF FT

43. 43 Conditional: This is used to define as "a proposition holds true if another proposition is true" i.e. p  q is read as "if p, then q". Truth table is given below. pq p  qp  q TTT TFF FTT FFT

44. 44 Biconditional: A proposition (p  q) ^ (q  p) can be abbreviated using biconditional conjunction as p  q and is read as "if p then q, and if q then p". Tautology: A compound proposition, which is true in every case. E.g., p V ¬ p Contradiction: This is the opposite of tautology, which is false in every case. E.g., p  ¬ p

45. 45 Identities ¬ (P  Q)  (¬ P  ¬Q) ----- DeMorgan's Law ¬ (P  Q)  (¬ P  ¬ Q) ----- DeMorgan's Law (P  Q)  (¬ P  Q) ----- implication [(P  Q)  R]  [P  (Q  R)] ----- exportation (P  Q)  (¬ Q  ¬P) ----- contrapositive

46. 46 Implications [(P  Q)  ¬Q]  P ----- modus tollens [(P  Q)  (R  S)]  [(P  R)  (Q  S)] [(P  Q)  (Q  R)]  (P  R)

47. 47 Predicate Logic The propositional logic is not powerful enough to represent all types of assertions. For example, the statement: x is a non-zero integer, where x is a variable, is not a proposition because you can not tell whether it is true or false unless you know the value of x. Thus the propositional logic can not deal with such sentences. The predicate logic is one of the extensions of propositional logic and it is fundamental to most other types of logic. Central to the predicate logic are the concepts of predicate and quantifier. A predicate is a feature of language which you can use to make a statement about something, e.g. to attribute a property to that thing. If you say "Peter is tall", then you have applied to Peter the predicate "is tall". We also might say that you have predicated tallness of Peter or attributed tallness to Peter.

48. 48 A predicate is a template involving a verb that describes a property of objects, or a relationship among objects represented by the variables. For example, the sentences: The flower is red. The sweater is red. The frame is red. All these sentences come from the template "is red" by placing an appropriate noun/noun phrase in front of it. The phrase "is red" is a predicate and it describes the property of being red. Predicates are often given a name. For example any of "is_red", "Red" or "R" can be used to represent the predicate "is red" among others. We can adopt Red as the name for the predicate “is_red". Sentences that assert an object is red can be represented as "Red(x)", where x represents an arbitrary object. Red(x) reads as "x is red".

49. 49 A predicate with variables, called atomic formula, can be made a proposition by applying one of the following two operations to each of its variables: assign a value to the variable quantify the variable using a quantifier For example, x > 1 becomes 3 > 1 if 3 is assigned to x, and it becomes a true statement, and it becomes a proposition. In general, a quantification is performed on formulas of predicate logic (called wff), such as x > 1 or P(x), by using quantifiers on variables.

50. 50 There are two types of quantifiers: universal quantifier existential quantifier The universal quantifier turns, for example, the statement x > 1 to "for every object x in the universe, x > 1", which is expressed as "  x x > 1". This new statement is true or false in the universe of discourse. Hence it is a proposition once the universe is specified. Similarly the existential quantifier turns, for example, the statement x > 1 to "for some object x in the universe, x > 1", which is expressed as "  x x > 1." Again, it is true or false in the universe of discourse, and hence it is a proposition once the universe is specified.

51. 51 The universe of discourse, also called universe, is the set of objects of interest. The propositions in the predicate logic are statements on objects of a universe. The universe is thus the domain of the (individual) variables. It can be: the set of real numbers the set of integers the set of all cars on a parking lot the set of all students in a classroom etc. The universe is often left implicit in practice. But it should be obvious from the context

52. 52 Predicate logic is more powerful than propositional logic. It allows one to reason about properties and relationships of individual objects. In predicate logic, one can use some additional inference rules, as well as those for propositional logic such as the equivalences, implications and inference rules. Predicate logic permits reasoning about the propositional connectives and also about quantification ("all" or "some"). A classic, if elementary, example of what can be done with the predicate logic is the inference from the premises: All men are mortal. Socrates is a man. to the conclusion Socrates is mortal

53. 53 Important Inference Rules of Predicate Logic First there is the following rule concerning the negation of quantified statement which is very useful: ¬  x P(x)   x ¬P(x) Next there is the following set of rules on quantifiers and connectives:  x [P(x)  Q(x)]  [  x P(x)   x Q(x)] [  x P(x)   x Q(x)]   x [P(x)  Q(x)]  x [P(x)  Q(x)]  [  x P(x)   x Q(x)]  x [P(x)  Q(x)]  [  x P(x)   x Q(x)]

54. 54 GRAPHS

55. 55 GRAPHS A directed graph Nodes (Vertices) V = { a, b, c, d, e } Edges E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) } node edge a b c d e

56. 56 Labeled Graph a b c d e 1 3 5 6 2 6 2

57. 57 Walk a b c d e Walk is a sequence of adjacent edges (e, d), (d, c), (c, a)

58. 58 Path a b c d e Path is a walk where no edge is repeated Simple path: no node is repeated

59. 59 Cycle a b c d e 1 2 3 Cycle: a walk from a node (base) to itself Simple cycle: only the base node is repeated base

60. 60 Euler Tour a b c d e 1 2 3 4 5 6 7 8 base A cycle that contains each edge once

61. 61 Hamiltonian Cycle a b c d e 1 2 3 4 5 base A simple cycle that contains all nodes

62. 62 Finding All Simple Paths a b c d e origin

63. 63 (c, a) (c, e) Step 1 a b c d e origin

64. 64 (c, a) (c, a), (a, b) (c, e) (c, e), (e, b) (c, e), (e, d) Step 2 a b c d e origin

65. 65 Step 3 a b c d e origin (c, a) (c, a), (a, b) (c, a), (a, b), (b, e) (c, e) (c, e), (e, b) (c, e), (e, d)

66. 66 Step 4 a b c d e origin (c, a) (c, a), (a, b) (c, a), (a, b), (b, e) (c, a), (a, b), (b, e), (e,d) (c, e) (c, e), (e, b) (c, e), (e, d)

67. 67 Trees

68. 68 Trees root leaf parent child Trees have no cycles

69. 69 root leaf Level 0 Level 1 Level 2 Level 3 Height 3

70. 70 Binary Trees

71. 71 PROOF TECHNIQUES

72. 72 PROOF TECHNIQUES Proof by induction Proof by contradiction

73. 73 Induction We have statements P 1, P 2, P 3, … If we know for some b that P 1, P 2, …, P b are true for any k >= b that P 1, P 2, …, P k imply P k+1 Then Every P i is true

74. 74 Proof by Induction Inductive basis Find P 1, P 2, …, P b which are true Inductive hypothesis Let’s assume P 1, P 2, …, P k are true, for any k >= b Inductive step Show that P k+1 is true

75. 75 Example Theorem: A binary tree of height n has at most 2 n leaves. Proof by induction: let L(i) be the maximum number of leaves of any subtree at height i

76. 76 We want to show: L(i) <= 2 i Inductive basis L(0) = 1 (the root node) Inductive hypothesis Let’s assume L(i) <= 2 i for all i = 0, 1, …, k Induction step we need to show that L(k + 1) <= 2 k+1

77. 77 Induction Step From Inductive hypothesis: L(k) <= 2 k height k k+1

78. 78 L(k) <= 2 k L(k+1) <= 2 * L(k) <= 2 * 2 k = 2 k+1 Induction Step height k k+1 (we add at most two nodes for every leaf of level k)

79. 79 Remark Recursion is another thing Example of recursive function: f(n) = f(n-1) + f(n-2) f(0) = 1, f(1) = 1

80. 80 Proof by Contradiction We want to prove that a statement P is true we assume that P is false then we arrive at an incorrect conclusion therefore, statement P must be true

81. 81 Example Theorem: is not rational Proof: Assume by contradiction that it is rational = n/m n and m have no common factors We will show that this is impossible

82. 82 = n/m 2 m 2 = n 2 Therefore, n 2 is even n is even n = 2 k 2 m 2 = 4k 2 m 2 = 2k 2 m is even m = 2 p Thus, m and n have common factor 2 Contradiction!

83. 83 Languages and Strings

84. 84 A language is a set of strings String: A sequence of letters Examples: “cat”, “dog”, “house”, … Defined over an alphabet:

85. 85 Alphabets and Strings We will use small alphabets: Strings

86. 86 String Operations Concatenation

87. 87 Reverse

88. 88 Concatenation and Reverse of Strings Theorem: If w and x are strings, then (w x) R = x R w R. Example: ( nametag ) R = ( tag ) R ( name ) R = gateman

89. 89 Concatenation and Reverse of Strings Proof: By induction on |x|: |x| = 0: Then x = , and (wx) R = (w  ) R = (w) R =  w R =  R w R = x R w R.  n  0 (((|x| = n)  ((w x) R = x R w R ))  ((|x| = n + 1)  ((w x) R = x R w R ))): Consider any string x, where |x| = n + 1. Then x = u a for some character a and |u| = n. So: (w x) R = (w (u a)) R rewrite x as ua = ((w u) a) R associativity of concatenation = a (w u) R definition of reversal = a (u R w R )induction hypothesis = (a u R ) w R associativity of concatenation = (ua) R w R definition of reversal = x R w R rewrite ua as x

90. 90 String Length Length: Examples:

91. 91 Length of Concatenation Example:

92. 92 Empty String A string with no letters: Observations:

93. 93 Substring Substring of string: a subsequence of consecutive characters String Substring

94. 94 Prefix and Suffix Prefixes Suffixes prefix suffix

95. 95 Another Operation (Replication) Example: Definition:

96. 96 The * Operation : the set of all possible strings from alphabet

97. 97 The + Operation : the set of all possible strings from alphabet except

98. 98 Languages A language is any subset of Example: Languages:

99. 99 Note that: Sets Set size String length

100. 100 Another Example An infinite language

101. 101 Operations on Languages The usual set operations Complement:

102. 102 Reverse Definition: Examples:

103. 103 Concatenation Definition: Example:

104. 104 Another Operation Definition: Special case:

105. 105 More Examples

106. 106 Star-Closure (Kleene *) Definition: Example:

107. 107 Positive Closure Definition:

108. 108 Grammar

109. 109 Grammar Mathematically describe language English: informal, inadequate Set notations: limited Grammar English grammar describes whether a sentence is well-formed or not  Then, what is noun_phrase, what is predicate?  Basic idea of formal grammars: Start from the top level concept Reduce it to the irreducible building blocks

110. 110 Grammar Definition A grammar G is defined as a quadruple G = (V, T, S, P) V is a finite set of objects called variables T is a finite set of objects called terminal symbols S  V is a special symbol called the start variable P is a finite set of productions v and T are non-empty and disjoint

111. 111 Production Rules Heart of a grammar Describe how the grammar transforms one string into another Define a language associated with grammar Production rules are of the form x  y, where x  (V  T) +, y  (V  T)* For example, given w = uxv, x  y, then z = uyv, z is a new string written as: w  z, means that w derives z or z is derived from w w 1  w 2  …  w n is the same as w 1 w n

112. 112 Definition Let G = (V, T, S, P) be a grammar. Then the set L(G) = { w  T* : S w} is the language generated by G. If w  L(G), S  w 1  w 2  …  w n  w is a derivation of the sentence w. Strings S, w 1, w 2, …, w n are called sentential forms of the derivation. All these strings may contain variables and terminals.

113. 113 Automaton

114. 114 Automaton Input “Accept” or “Reject” Output S1S1 S2S2 S3S3 S4S4 S5S5 Storage Control Unit

115. 115 Automata An abstract model of a digital computer Input mechanism –Can read but not change an input file Input file –Contains a string over a given alphabet –Divided into cells, each cell holds one symbol –The reading is from left to right, one symbol at a time –Can sense the end of the input string Storage device –Consist of an unlimited number of cells –Each cell holds a symbol –Can be read and changed Control unit –Internal states –Transition function (or next-state): gives the next state of the control unit Current state Current input symbol Current information in the temporary storage

116. 116 To Illustrate “State” State: status, A state stores information about the past, i.e. it reflects the input changes from the system start to the present moment Example: a robot

117. 117 Configuration –A particular state of the control unit, input file, and temporary storage Move –The transition of the automaton from one configuration to the next Deterministic automata –Each move is uniquely determined by the current configuration –If the internal state, the input, and the contents of the temporary storage are known, the future behavior of the automata can be predicted Nondeterministic automata –At some point, the automata may have several moves –A set of possible actions can be predicted Accepter –An automata whose output response is either “yes” or “no”