1. 1 CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 3 Mälardalen University 2006

2. 2 Content Finite Automata, FA Deterministic Finite Automata, DFA Nondeterministic Automata NFA NFA  DFA Equivalence Grammatik Linjär grammatik Reguljär grammatik

3. 3 Finite Automata FA (Finite State Machines)

4. 4 There is no formal general definition for "automaton". Instead, there are various kinds of automata, each with it's own formal definition. has some form of input has some form of output has internal states, may or may not have some form of storage is hard-wired rather than programmable Generally, an automaton

5. 5 Finite Automaton Input String Output String Finite Automaton

6. 6 Finite Accepter Input “Accept” or “Reject” String Finite Automaton Output

7. 7 Nodes = States Edges = Transitions An edge with several symbols is a short-hand for several edges: FA as Directed Graph

8. 8 Deterministic Finite Automata DFA

9. 9 Deterministic there is no element of choice Finite only a finite number of states and arcs Acceptors produce only a yes/no answer DFA

10. 10 Transition Graph initial state final state “accept” state transition abba -Finite Acceptor Alphabet =

11. 11 Formal definitions Deterministic Finite Accepter (DFA) : set of states : input alphabet : transition function : initial state : set of final states

12. 12 Input Aplhabet

13. 13 Set of States

14. 14 Initial State

15. 15 Set of Final States

16. 16 Transition Function

17. 17

18. 18

19. 19

20. 20 Transition Function

21. 21 Extended Transition Function

22. 22

23. 23

24. 24 Another Example accept Alphabet =

25. 25 Formally For a DFA Language accepted by : alphabet transition function initial state final states

26. 26 Observation Language accepted by Language rejected by

27. 27 Regular Languages All regular languages form a language family A language is regular if there is a DFA such that

28. 28 Nondeterministic Automata NFA

29. 29 Nondeterministic there is an element of choice: in a given state NFA can act on a given string in different ways. Several start/final states are allowed. -transitions are allowed. Finite only a finite number of states and arcs Acceptors produce only a yes/no answer NFA

30. 30 Two choices Alphabet = Nondeterministic Finite Accepter (NFA)

31. 31 First Choice

32. 32 First Choice

33. 33 First Choice

34. 34 “accept” First Choice

35. 35 Second Choice

36. 36 Second Choice

37. 37 Second Choice No transition: the automaton hangs

38. 38 Second Choice “reject”

39. 39 Observation An NFA accepts a string if there is a computation of the NFA that accepts the string

40. 40 Example is accepted by the NFA:

41. 41 Lambda Transitions

42. 42

43. 43

44. 44 (read head doesn’t move)

45. 45

46. 46 “accept” String is accepted

47. 47 Language accepted:

48. 48 Another NFA Example Alphabet =

49. 49

50. 50

51. 51

52. 52 “accept”

53. 53 Another String Alphabet =

54. 54

55. 55

56. 56

57. 57

58. 58

59. 59

60. 60 “accept”

61. 61 Language accepted Alphabet =

62. 62 Another NFA Example Alphabet =

63. 63 Language accepted

64. 64 Formal Definition of NFA Set of states, i.e. Input alphabet, i.e. Transition function Initial state Final states

65. 65 Transition Function

66. 66

67. 67

68. 68

69. 69 Extended Transition Function (Utvidgad övergångsfunktion)

70. 70

71. 71

72. 72 Formally if and only if there is a walk from to with label

73. 73 The Language of an NFA Alphabet =

74. 74

75. 75

76. 76

77. 77

78. 78 Formally The language accepted by NFA (final state) where and there is some (at least one) is:

79. 79

80. 80 NFA  DFA Equivalence

81. 81 Equivalence of NFAs and DFAs Accept the same languages? YES! NFAs  DFAs ? Same power?

82. 82 We will prove: Languages accepted by NFAs Languages accepted by DFAs NFAs and DFAs have the same computation power!

83. 83 Languages accepted by NFAs Languages accepted by DFAs Step 1 Proof Every DFA is also an NFA A language accepted by a DFA is also accepted by an NFA

84. 84 Languages accepted by NFAs Languages accepted by DFAs Step 2 Proof Any NFA can be converted to an equivalent DFA A language accepted by an NFA is also accepted by a DFA

85. 85 Procedure NFA to DFA 1. Initial state of NFA: Initial state of DFA:

86. 86 Example NFA DFA Step 1

87. 87 Procedure NFA to DFA 2. For every DFA’s state Compute in the NFA Add transition

88. 88 Example NFA DFA Step 2

89. 89 Procedure NFA to DFA Repeat Step 2 for all letters in alphabet, until no more transitions can be added.

90. 90 Example NFA DFA Step 3

91. 91 Procedure NFA to DFA 3. For any DFA state If some is a final state in the NFA Then is a final state in the DFA

92. 92 Example NFA DFA Step 4

93. 93 Theorem Take NFA Apply procedure to obtain DFA Then and are equivalent :

94. 94 Languages accepted by NFAs Languages accepted by DFAs We have proven (proof by construction): Regular Languages END OF PROOF

95. 95 Nondeterministic vs. Deterministic Automata

96. 96 Formal Definition of NFA Set of states, i.e. Input alphabet, i.e. Transition function Initial state Final (accepting) states NFA is a mathematical model defined as a quintuple:

97. 97 Deterministic Finite Automata A deterministic finite automaton (DFA) is a special case of a nondeterministic finite automaton in which 1. no state has an -transition, i.e. a transition on input, and 2. for each state q and input symbol a, there is at most one edge labeled a leaving q.

98. 98 STATE INPUT SYMBOL ab 012012 {0, 1} - {0} {2} {3} Transition table for the finite automaton above A nondeterministic finite automaton b 0 start 1 a 2 bb 3 a Example

99. 99 NFA accepting aa* + bb* 0 start 1 a 2 a 3 b 4 b Example

100. 100 NFA accepting (a+b)*abb 0 start 1 a 2 bb b aa a b 3 a Example

101. 101 NFA recognizing three different patterns. (a) NFA for a, abb, and a*b +. (b) Combined NFA. Example 4 1 start a 2 3 a 65 b b 7 b 8 b a 4 1 a 2 3 a 65 bb 7 b 8 b a 0

102. 102 Ways to think of nondeterminism always make the correct guess “backtracking” (systematically try all possibilities) For a particular string, imagine a tree of possible state transitions: q0q0 q3q3 q0q0 q4q4 q2q2 q1q1 a a a a b a

103. 103 Advantages of nondeterminism an NFA can be smaller, easier to construct and easier to understand than a DFA that accepts the same language useful for proving some theorems good introduction to nondeterminism in more powerful computational models, where nondeterminism plays an important role

104. 104 Space and time taken to recognize regular expressions: - NFA more compact but take time to backtrack all choices - DFA take place, but save time AUTOMATONSPACETIME NFA DFA O(|r|) O(2 |r| ) O(|r|  |x|) O(|x|) Determinism vs. nondeterminism (Where r is regular expression, and x is input string)

105. 105 Equivalent automata Two finite automata M 1 and M 2 are equivalent if L(M 1 ) = L(M 2 ) that is, if they both accept the same language.

106. 106 Equivalence of NFAs and DFAs To show that NFAs and DFAs accept the same class of languages, we show two things: –Any language accepted by a DFA can also be accepted by some NFA (As DFA is a special case of NFA) –Any language accepted by a NFA can also be accepted by some (corresponding, specially constructed) DFA

107. 107 Proof strategy To show that any language accepted by a NFA is also accepted by some DFA, we describe an algorithm that takes any NFA and converts it into a DFA that accepts the same language. The algorithm is called the “subset construction algorithm”. We can use mathematical induction (on the length of a string accepted by the automaton) to prove the DFA that is constructed accepts the same language as the NFA.

108. 108 Converting NFA to DFA Subset Construction

109. 109 Subset construction Given a NFA constructs a DFA that accepts the same language The equivalent DFA simulates the NFA by keeping track of the possible states it could be in. Each state of the DFA is a subset of the set of states of the NFA -hence, the name of the algorithm. If the NFA has n states, the DFA can have as many as 2 n states, although it usually has many less.

110. 110 Steps of subset construction The initial state of the DFA is the set of all states the NFA can be in without reading any input. For any state {q i,q j,…,q k } of the DFA and any input a, the next state of the DFA is the set of all states of the NFA that can result as next states if the NFA is in any of the states q i,q j,…,q k when it reads a. This includes states that can be reached by reading a, followed by any number of -moves. Use this rule to keep adding new states and transitions until it is no longer possible to do so. The accepting states of the DFA are those states that contain an accepting state of the NFA.

111. 111 Example Here is a NFA that we want to convert to an equivalent DFA. 0 1 2

112. 112 {0,1} The start state of the DFA is the set of states the NFA can be in before reading any input. This includes the start state of the NFA and any states that can be reached by a -transition. NFA DFA

113. 113 {0,1} a b {2} For start state {0,1}, make transitions for each possible input, here a and b. Reading b from start {0,1}, we reach state {2}. Means from either {0}, or {1} we reach {2}. NFA DFA

114. 114 For state {2}, we create a transition for each possible input, a and b. From {2}, with b we are either back to {2} (loop) or we reach {1}- see the little framed original NFA. So from {2}, with b we end in state {1, 2}. Reading a leads us from {2} to {0} in the original NFA, which means state {0, 1} in the new DFA. {0,1} {1,2} {2} NFA DFA

115. 115 For state {1, 2}, we make again transition for each possible input, a and b. From {2} a leads us to {0}. From {1} with a we are back to {1}. So, we reach {0, 1} with a from {1,2}. With b we are back to {1,2}. At this point, a transition is defined for every state-input pair. {0,1} {1,2} {2} DFA NFA

116. 116 The last step is to mark the final states of the DFA. As {1} was the accepting state in NFA, all states containing {1} in DFA will be accepting states: ({0, 1} and {1, 2}). {0,1} {1,2} {2} DFA NFA

117. 117 Subset Construction Algorithm

118. 118 Subset Construction States of nondeterministic M´ will correspond to sets of states of deterministic M Where q 0 is start state of M, use {q 0 } as start state of M´. Accepting states of M´ will be those state-sets containing at least one accepting state of M.

119. 119 Subset Construction (cont.) For each state-set S and for each s in alphabet of M, we draw an arc labeled s from state S to that state-set consisting of all and only the s- successors of members of S. Eliminate any state-set, as well as all arcs incident upon it, such that there is no path leading to it from {q 0 }.

120. 120 The power set of a finite set, Q, consists of 2 |Q| elements The DFA corresponding to a given NFA with Q states have a finite number of states, 2 |Q|. If |Q| = 0 then Q is the empty set, | P(Q)| = 1 = 2 0. If |Q| = N and N  1, we construct subset of a given set so that for each element of the initial set there are two alternatives, either is the element member of a subset or not. So we have 2 · 2 · 2 · 2 · 2 · 2 · 2…. ·2 = 2 N N times

121. 121 From an NFA to a DFA Subset Construction Operation Description - closure(s) - closure(T) Move(T,a) Set of NFA states reachable from an NFA state s on -transitions along Set of NFA states reachable from some NFA state s in T on - transitions along Set of NFA states reachable from some NFA state set with a transition on input symbol a

122. 122 From an NFA to a DFA Subset Construction Initially, -closure (s 0 ) is the only states in D and it is unmarked while there is an unmarked state T in D do mark T; for each input symbol a do U:= e-closure(move(T,a)); if U is not in D then add U as an unmarked state to D Dtran[T,a]:=U; end(for) end(while)

123. 123 Grammars

124. 124 Grammars Grammars express languages Example: the English language

125. 125 barksverb singsverb   dognoun birdnoun   thearticle a  

126. 126 A derivation of “the bird sings”:         birdthe verbbirdthe verbnounthe verbnounarticle predicatenounarticle predicatephrasenounsentence sings _

127. 127 A derivation of “a dog barks”: barksdoga verbdoga verbnouna verbnounarticle verbphrasenoun predicatephrasenounsentence       _ _

128. 128 The language of the grammar:

129. 129 Notation Non-terminal (Variable) Terminal Production rule

130. 130 Example Derivation of sentence: Grammar:

131. 131 Grammar: Derivation of sentence

132. 132 Other derivations

133. 133 The language of the grammar

134. 134 Formal Definition Grammar Set of variables Set of terminal symbols Start variable Set of production rules

135. 135 Example Grammar

136. 136 Sentential Form A sentence that contains variables and terminals Example sentential formsSentence (sats)

137. 137 We write: Instead of:

138. 138 In general we write if By default ()

139. 139 Example Grammar Derivations

140. 140 baaaaaSbbbbaaSbb   S   Grammar Example Derivations

141. 141 Another Grammar Example Derivations Grammar

142. 142 More Derivations

143. 143 The Language of a Grammar For a grammar with start variable String of terminals

144. 144 Example For grammar Since

145. 145 Notation

146. 146 Linear Grammars

147. 147 Linear Grammars Grammars with at most one variable (non-terminal) at the right side of a production Examples:

148. 148 A Non-Linear Grammar Grammar

149. 149 Another Linear Grammar Grammar

150. 150 Right-Linear Grammars All productions have form: or Example

151. 151 Left-Linear Grammars All productions have form or Example

152. 152 Regular Grammars

153. 153 Regular Grammars Generate Regular Languages

154. 154 Theorem Languages Generated by Regular Grammars Regular Languages

155. 155 Theorem - Part 1 Languages Generated by Regular Grammars Regular Languages Any regular grammar generates a regular language

156. 156 Theorem - Part 2 Any regular language is generated by a regular grammar Languages Generated by Regular Grammars Regular Languages

157. 157 Proof – Part 1 The language generated by any regular grammar is regular Languages Generated by Regular Grammars Regular Languages

158. 158 The case of Right-Linear Grammars Let be a right-linear grammar We will prove: is regular Proof idea We will construct NFA with

159. 159 Grammar is right-linear Example

160. 160 Construct NFA such that every state is a grammar variable: special final state

161. 161 Add edges for each production:

162. 162

163. 163

164. 164

165. 165

166. 166

167. 167 Grammar NFA

168. 168 In General A right-linear grammar has variables: and productions: or

169. 169 We construct the NFA such that: each variable corresponds to a node: special final state ….

170. 170 For each production: we add transitions and intermediate nodes ………

171. 171 Example

172. 172 The case of Left-Linear Grammars Let be a left-linear grammar We will prove: is regular Proof idea We will construct a right-linear grammar with

173. 173 Since is left-linear grammar the productions look like:

174. 174 Construct right-linear grammar In :

175. 175 Construct right-linear grammar In :

176. 176 It is easy to see that: Since is right-linear, we have: Regular Language Regular Language Regular Language

177. 177 Proof - Part 2 Any regular language is generated by some regular grammar Languages Generated by Regular Grammars Regular Languages

178. 178 Proof idea Any regular language is generated by some regular grammar Construct from a regular grammar such that Since is regular there is an NFA such that

179. 179 Example

180. 180 Convert to a right-linear grammar

181. 181 In General For any transition: Add production: variableterminalvariable

182. 182 For any final state: Add production:

183. 183 Since is right-linear grammar is also a regular grammar with

184. 184 Regular Grammars A regular grammar is any right-linear or left-linear grammar Examples

185. 185 Observation Regular grammars generate regular languages Examples

186. 186 Regular Languages Chomsky’s Language Hierarchy Non-regular languages