1. 1 CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 3 Mälardalen University 2007

2. 2 Content Finite Automata, FA Deterministic Finite Automata, DFA Nondeterministic Automata NFA NFA  DFA Equivalence Grammatik Linear grammar Regular grammar

3. 3 Finite Automata FA (Finite State Machines)

4. 4 There is no formal general definition for "automaton". Instead, there are various kinds of automata, each with it's own formal definition. has some form of input has some form of output has internal states, may or may not have some form of storage is hard-wired rather than programmable Generally, an automaton

5. 5 Finite Automaton Input String Output String Finite Automaton

6. 6 Finite Accepter Input “Accept” or “Reject” String Finite Automaton Output

7. 7 Nodes = States Edges = Transitions An edge with several symbols is a short-hand for several edges: FA as Directed Graph

8. 8 Deterministic there is no element of choice Finite only a finite number of states and arcs Acceptors produce only a yes/no answer Deterministic Finite Automata DFA

9. 9 Transition Graph initial state final state “accept” state transition abba -Finite Acceptor Alphabet =

10. 10 Formal definitions Deterministic Finite Accepter (DFA) : set of states : input alphabet : transition function : initial state : set of final states

11. 11 Input Aplhabet

12. 12 Set of States

13. 13 Initial State

14. 14 Set of Final States

15. 15 Transition Function

16. 16

17. 17

18. 18

19. 19 Transition Function

20. 20 Extended Transition Function

21. 21

22. 22

23. 23 Formally For a DFA Language accepted by : alphabet transition function initial state final states

24. 24 Observation Language accepted by Language rejected by

25. 25 Regular Languages All regular languages form a language family. A language is regular if there is a DFA such that

26. 26 Nondeterministic there is an element of choice: in a given state NFA can act on a given string in different ways. Several start/final states are allowed. -transitions are allowed. Finite only a finite number of states and arcs Acceptors produce only a yes/no answer Nondeterministic Automata NFA

27. 27 Two choices Alphabet = Nondeterministic Finite Accepter (NFA)

28. 28 First Choice

29. 29 First Choice

30. 30 First Choice

31. 31 “accept” First Choice

32. 32 Second Choice

33. 33 Second Choice

34. 34 Second Choice No transition: the automaton hangs

35. 35 Second Choice “reject”

36. 36 Observation An NFA accepts a string if there is a computation of the NFA that accepts the string.

37. 37 Example is accepted by the NFA:

38. 38 Lambda Transitions

39. 39

40. 40

41. 41 (read head doesn’t move)

42. 42

43. 43 “accept” String is accepted

44. 44 Language accepted:

45. 45 Another NFA Example Alphabet =

46. 46

47. 47

48. 48

49. 49 “accept”

50. 50 Another String Alphabet =

51. 51

52. 52

53. 53

54. 54

55. 55

56. 56

57. 57 “accept”

58. 58 Language accepted Alphabet =

59. 59 Further NFA Example Alphabet =

60. 60 Language accepted

61. 61 Formal Definition of NFA Set of states, i.e. Input alphabet, i.e. Transition function Initial state Final states

62. 62 Transition Function

63. 63

64. 64

65. 65

66. 66 Extended Transition Function (Utvidgad övergångsfunktion)

67. 67

68. 68

69. 69 Formally if and only if there is a walk from to with label

70. 70 The Language of an NFA Alphabet =

71. 71

72. 72

73. 73

74. 74

75. 75 Formally The language accepted by NFA is: (final state) where and there is some (at least one)

76. 76

77. 77 Equivalence of NFAs and DFAs Accept the same languages? YES! NFAs  DFAs ? Same power?

78. 78 We will prove: Languages accepted by DFAs Languages accepted by NFAs NFAs and DFAs have the same computation power!

79. 79 Languages accepted by DFAs Languages accepted by NFAs Step 1 Proof Every DFA is also an NFA. A language accepted by a DFA is also accepted by an NFA.

80. 80 Languages accepted by NFAs Languages accepted by DFAs Step 2 Proof Any NFA can be converted to an equivalent DFA. A language accepted by an NFA is also accepted by a DFA.

81. 81 Procedure NFA to DFA 1. Initial state of NFA: Initial state of DFA:

82. 82 Example NFA DFA Step 1

83. 83 Procedure NFA to DFA 2. For every DFA’s state Compute in the NFA Add transition

84. 84 Example NFA DFA Step 2

85. 85 Procedure NFA to DFA Repeat Step 2 for all letters in alphabet, until no more transitions can be added.

86. 86 Example NFA DFA Step 3

87. 87 Procedure NFA to DFA 3. For any DFA state If some is a final state in the NFA Then is a final state in the DFA

88. 88 Example NFA DFA Step 4

89. 89 Theorem Take NFA Apply procedure to obtain DFA Then and are equivalent :

90. 90 Languages accepted by NFAs Languages accepted by DFAs We have proven (proof by construction): Regular Languages END OF PROOF

91. 91 Nondeterministic vs. Deterministic Automata

92. 92 Formal Definition of NFA Set of states, i.e. Input alphabet, i.e. Transition function Initial state Final (accepting) states NFA is a mathematical model defined as a quintuple:

93. 93 Deterministic Finite Automata A deterministic finite automaton (DFA) is a special case of a nondeterministic finite automaton in which 1. no state has an -transition, i.e. a transition on input, and 2. for each state q and input symbol a, there is at most one edge labeled a leaving q.

94. 94 STATE INPUT SYMBOL ab 012012 {0, 1} - {0} {2} {3} Transition table for the finite automaton above A nondeterministic finite automaton b 0 start 1 a 2 bb 3 a Example

95. 95 NFA accepting aa* + bb* 0 start 1 a 2 a 3 b 4 b Example

96. 96 NFA accepting (a+b)*abb 0 start 1 a 2 bb b aa a b 3 a Example

97. 97 NFA recognizing three different patterns. (a) NFA for a, abb, and a*b +. (b) Combined NFA. Example 4 1 start a 2 3 a 65 b b 7 b 8 b a 4 1 a 2 3 a 65 bb 7 b 8 b a 0

98. 98 Ways to think of nondeterminism always make the correct guess “backtracking” (systematically try all possibilities) For a particular string, imagine a tree of possible state transitions: q0q0 q3q3 q0q0 q4q4 q2q2 q1q1 a a a a b a

99. 99 Advantages of nondeterminism An NFA can be smaller, easier to construct and easier to understand than a DFA that accepts the same language. NFA’s can be useful for proving some theorems. NFA’s can be good introduction to nondeterminism in more powerful computational models, where nondeterminism plays an important role.

100. 100 Space and time taken to recognize regular expressions: - NFA more compact but take time to backtrack all choices - DFA take place, but save time AUTOMATONSPACETIME NFA DFA O(|r|) O(2 |r| ) O(|r|  |x|) O(|x|) Determinism vs. nondeterminism (Where r is regular expression, and x is input string)

101. 101 Equivalent automata Two finite automata M 1 and M 2 are equivalent if L(M 1 ) = L(M 2 ) that is, if they both accept the same language.

102. 102 Equivalence of NFAs and DFAs To show that NFAs and DFAs accept the same class of languages, we show two things: –Any language accepted by a DFA can also be accepted by some NFA (As DFA is a special case of NFA). –Any language accepted by a NFA can also be accepted by some (corresponding, specially constructed) DFA.

103. 103 Proof Strategy To show that any language accepted by a NFA is also accepted by some DFA, we describe an algorithm that takes any NFA and converts it into a DFA that accepts the same language. The algorithm is called the “subset construction algorithm”. We can use mathematical induction (on the length of a string accepted by the automaton) to prove the DFA that is constructed accepts the same language as the NFA.

104. 104 Converting NFA to DFA Subset construction Given a NFA construct a DFA that accepts the same language. The equivalent DFA simulates the NFA by keeping track of the possible states it could be in. Each state of the DFA is a subset of the set of states of the NFA - hence, the name of the algorithm. If the NFA has n states, the DFA can have as many as 2 n states, although it usually has many less.

105. 105 Steps of Subset Construction The initial state of the DFA is the set of all states the NFA can be in without reading any input. For any state {q i,q j,…,q k } of the DFA and any input a, the next state of the DFA is the set of all states of the NFA that can result as next states if the NFA is in any of the states q i,q j,…,q k when it reads a. This includes states that can be reached by reading a, followed by any number of -moves. Use this rule to keep adding new states and transitions until it is no longer possible to do so. The accepting states of the DFA are those states that contain an accepting state of the NFA.

106. 106 Example Here is a NFA that we want to convert to an equivalent DFA. 0 1 2

107. 107 {0,1} The start state of the DFA is the set of states the NFA can be in before reading any input. This includes the start state of the NFA and any states that can be reached by a -transition. NFA DFA

108. 108 {0,1} a b {2} For start state {0,1}, make transitions for each possible input, here a and b. Reading b from start {0,1}, we reach state {2}. Means from either {0}, or {1} we reach {2}. NFA DFA

109. 109 For state {2}, we create a transition for each possible input, a and b. From {2}, with b we are either back to {2} (loop) or we reach {1}- see the little framed original NFA. So from {2}, with b we end in state {1, 2}. Reading a leads us from {2} to {0} in the original NFA, which means state {0, 1} in the new DFA. {0,1} {1,2} {2} NFA DFA

110. 110 For state {1, 2}, we make again transition for each possible input, a and b. From {2} a leads us to {0}. From {1} with a we are back to {1}. So, we reach {0, 1} with a from {1,2}. With b we are back to {1,2}. At this point, a transition is defined for every state-input pair. {0,1} {1,2} {2} DFA NFA

111. 111 The last step is to mark the final states of the DFA. As {1} was the accepting state in NFA, all states containing {1} in DFA will be accepting states: ({0, 1} and {1, 2}). {0,1} {1,2} {2} DFA NFA

112. 112 Subset Construction Algorithm States of nondeterministic M´ will correspond to sets of states of deterministic M. Where q 0 is start state of M, use {q 0 } as start state of M´. Accepting states of M´ will be those state-sets containing at least one accepting state of M.

113. 113 Subset Construction (cont.) For each state-set S and for each s in alphabet of M, we draw an arc labeled s from state S to that state-set consisting of all and only the s-successors of members of S. Eliminate any state-set, as well as all arcs incident upon it, such that there is no path leading to it from {q 0 }.

114. 114 The power set of a finite set, Q, consists of 2 |Q| elements The DFA corresponding to a given NFA with Q states have a finite number of states, 2 |Q|. If |Q| = 0 then Q is the empty set, | P(Q)| = 1 = 2 0. If |Q| = N and N  1, we construct subset of a given set so that for each element of the initial set there are two alternatives, either is the element member of a subset or not. So we have 2 · 2 · 2 · 2 · 2 · 2 · 2…. ·2 = 2 N N times

115. 115 From an NFA to a DFA Subset Construction Operation Description - closure(s) - closure(T) Move(T,a) Set of NFA states reachable from an NFA state s on -transitions along Set of NFA states reachable from some NFA state s in T on -transitions along Set of NFA states reachable from some NFA state set with a transition on input symbol a

116. 116 From an NFA to a DFA Subset Construction Initially, -closure (s 0 ) is the only states in D and it is unmarked while there is an unmarked state T in D do mark T; for each input symbol a do U:= e-closure(move(T,a)); if U is not in D then add U as an unmarked state to D Dtran[T,a]:=U; end(for) end(while)

117. 117 Grammars Grammars express languages Example: the English language

118. 118 barksverb singsverb   dognoun birdnoun   thearticle a  

119. 119 A derivation of “the bird sings”:         birdthe verbbirdthe verbnounthe verbnounarticle predicatenounarticle predicatephrasenounsentence sings _

120. 120 A derivation of “a dog barks”: barksdoga verbdoga verbnouna verbnounarticle verbphrasenoun predicatephrasenounsentence       _ _

121. 121 The language of the grammar:

122. 122 Notation Non-terminal (Variable) Terminal Production rule

123. 123 Example Derivation of sentence: Grammar:

124. 124 Grammar: Derivation of sentence

125. 125 Other derivations

126. 126 The language of the grammar

127. 127 Formal Definition Grammar Set of variables Set of terminal symbols Start variable Set of production rules

128. 128 Example Grammar

129. 129 Sentential Form A sentence that contains variables and terminals Example sentential formsSentence (sats)

130. 130 We write: Instead of:

131. 131 In general we write if By default ()

132. 132 Example Grammar Derivations

133. 133 baaaaaSbbbbaaSbb   S   Grammar Example Derivations

134. 134 Another Grammar Example Derivations Grammar

135. 135 More Derivations

136. 136 The Language of a Grammar For a grammar with start variable String of terminals

137. 137 Example For grammar Since

138. 138 Notation

139. 139 Linear Grammars Grammars with at most one variable (non-terminal) at the right side of a production Examples:

140. 140 A Non-Linear Grammar Grammar

141. 141 Another Linear Grammar Grammar

142. 142 Right-Linear Grammars All productions have form: or Example

143. 143 Left-Linear Grammars All productions have form or Example

144. 144 Regular Grammars Generate Regular Languages

145. 145 Theorem Languages Generated by Regular Grammars Regular Languages

146. 146 Theorem - Part 1 Languages Generated by Regular Grammars Regular Languages Any regular grammar generates a regular language

147. 147 Theorem - Part 2 Any regular language is generated by a regular grammar Languages Generated by Regular Grammars Regular Languages

148. 148 Proof – Part 1 The language generated by any regular grammar is regular Languages Generated by Regular Grammars Regular Languages

149. 149 The case of Right-Linear Grammars Let be a right-linear grammar We will prove: is regular Proof idea We will construct NFA with

150. 150 Grammar is right-linear Example

151. 151 Construct NFA such that every state is a grammar variable: special final state

152. 152 Add edges for each production:

153. 153

154. 154

155. 155

156. 156

157. 157

158. 158 Grammar NFA

159. 159 In General A right-linear grammar has variables: and productions: or

160. 160 We construct the NFA such that: each variable corresponds to a node: special final state ….

161. 161 For each production: we add transitions and intermediate nodes ………

162. 162 Example

163. 163 The case of Left-Linear Grammars Let be a left-linear grammar We will prove: is regular Proof idea We will construct a right-linear grammar with

164. 164 Since is left-linear grammar the productions look like:

165. 165 Construct right-linear grammar In :

166. 166 Construct right-linear grammar In :

167. 167 It is easy to see that: Since is right-linear, we have: Regular Language Regular Language Regular Language

168. 168 Proof - Part 2 Any regular language is generated by some regular grammar Languages Generated by Regular Grammars Regular Languages

169. 169 Proof idea Any regular language is generated by some regular grammar Construct from a regular grammar such that Since is regular there is an NFA such that

170. 170 Example

171. 171 Convert to a right-linear grammar

172. 172 In General For any transition: Add production: variableterminalvariable

173. 173 For any final state: Add production:

174. 174 Since is right-linear grammar is also a regular grammar with

175. 175 Regular Grammars A regular grammar is any right-linear or left-linear grammar Examples

176. 176 Observation Regular grammars generate regular languages Examples

177. 177 Regular Languages Chomsky’s Language Hierarchy Non-regular languages