1. 1 1 CDT314 FABER Formal Languages, Automata and Models of Computation Lecture 4 School of Innovation, Design and Engineering Mälardalen University 2012

2. 2 Regular Expressions and Regular Languages NFA  DFA Subset Construction (Delmängdskonstruktion) FA  RE State Elimination (Tillståndseliminering) Minimizing DFA by Set Partitioning (Särskiljandealgoritmen) Grammar: Linear grammar. Regular grammar. Application: Compiler. Lexical Analysis. Content

3. 3 Two Basic Theorems about Finite State Automata Based on C Busch, RPI, Models of Computation

4. 4 1: KleeneTheorem NFA  DFA Let M be NFA accepting L. Then there exists DFA M´ that accepts L as well.  For each NFA there exists corresponding DFA that accepts the same language.

5. 5 II: Finite Language Theorem Any finite language is regular. Any finite language is FSA-acceptable. Proof: every finite language is a union of single strings which are regular because described by regular expressions Example L = {abba, abb, abab} L is expressed by the regular expression “abba|abb|abab“ FSA = Finite State Automaton = DFA/NFA

6. 6 Some Properties of Regular Languages, Summary

7. 7 Properties Concatenation: Star: Union: Are Regular Languages For regular languages and we will prove that:

8. 8 Regular languages are closed under Concatenation:Star: Union:

9. 9 Regular language Single final state NFA Regular language Single final state NFA

10. 10 Example

11. 11 Union (Thompson’s construction) NFA for

12. 12 NFA for Example

13. 13 Concatenation (Thompson’s construction) NFA for

14. 14 Example NFA for

15. 15 Star Operation (Thompson’s construction) NFA for

16. 16 Example NFA for

17. 17 Reverse of a Regular Language

18. 18 Theorem The reverse of a regular language is a regular language Proof idea Construct NFA that accepts : invert the transitions of the NFA that accepts

19. 19 Proof Since is regular, there is NFA that accepts Example:

20. 20 Invert Transitions

21. 21 Make old initial state a final state and vice versa

22. 22 Add a new initial state

23. 23 Resulting machine accepts is regular

24. 24 Regular Expressions and Regular Languages

25. 25 Theorem: Regular expressions are representation of regular languages and they are equivalent. Regular expressions describe regular languages in formal language theory. They have the same expressive power as regular grammars.

26. 26 Theorem - Part 1 Languages Generated by Regular Expressions Regular Languages 1. For any regular expression the language is regular

27. 27 Theorem - Part 2 Languages Generated by Regular Expressions Regular Languages 2. For any regular language there is a regular expression with

28. 28 Proof - Part 1 For any regular expression the language is regular Proof by induction on the size of

29. 29 Induction Basis Primitive Regular Expressions: NFAs Regular Languages (where )

30. 30 Inductive Hypothesis Assume for regular expressions and that and are regular languages

31. 31 Inductive Step We will prove: regular languages

32. 32 By definition of regular expressions:

33. 33 By inductive hypothesis we know: and are regular languages Regular languages are closed under union concatenation star

34. 34 Therefore regular languages And trivially is a regular language

35. 35 Proof – Part 2 For any regular language there is a regular expression with Proof by construction of regular expression

36. 36 Single final state Since is regular take the NFA that accepts it

37. 37 From construct the equivalent Generalized Transition Graph transition labels are regular expressions Example QED

38. 38 Summary: Operations on Regular Expressions RE Regular language description a+b {a,b} (a+b)(a+b) {aa, ab, ba, bb} a* {, a, aa, aaa, …} a*b {b,ab, aab, aaab, …} (a+b)* {, a, b, aa, ab, ba, aaa, bbb…}

39. 39 Algebraic Properties

40. 40 Operator Precedence 1. Kleene star 2. Concatenation 3. Union allows dropping unnecessary parentheses.

41. 41 Converting Regular Expression to a DFA

42. 42 Example: From a Regular Expression to an NFA Example : (a+b)*abb step by step construction 2 a 45 b 6 1 3 (a+b)

43. 43 Example: From a Regular Expression to an NFA Example : (a+b)*abb 2 a 45 b 1 6 3 0 7 (a+b)*

44. 44 Example: From a Regular Expression to an NFA Example : (a+b)*abb 2 a 45 b 1 0 10 a b b 7 6 3 8 9 (a+b)*abb

45. 45 Converting FA to a Regular Expression State Elimination http://www.math.uu.se/~salling/Movies/FA_to_RegExpression.mov

46. 46 Add a new start (s) and a new final (f) state: From s to the ex-starting state (1) From each ex-final state (1,3) to f Example, 1 2 3

47. 47, 1 2 3 s f Let’s remove state 1! Each combination of input/output to 1 will generate a new path once state 1 is gone ;, ; ;,; The original:

48. 48 When state 1 is gone we must be able to make all those transitions!, 1 2 3 s f  () ;,, Previous:

49. 49 ;, 1 2 3 s f  ()

50. 50, 1 2 3 s f ;,  ()  ()

51. 51 A common mistake: having several arrows between the same pair of states. Join the two arrows (by union of their regular expressions) before going on to the next step., 1 2 3 s f  ()  () 

52. 52, 1 2 3 s f ;  ()  () 

53. 53 Union again.., 1 2 3 s f  ()  ()  

54. 54 2 3 s f  ()  ()  Without state 1... 

55. 55 2 3 s f  ()  (  Now we repeat the same procedure for the state 2... 

56. 56 Following the path s-2-3 we concatenate all strings on our way…don’t forget a* in 2! 2 3 s f  ()  ()    () 

57. 57 When 2 is removed, the path 3 - 2 –3 has also to be preserved, so we concatenate 3-2 string, take a* loop and go back 2-3 with string b! 2 3 s f  ()  (    ()  (  ()   )

58. 58 3 s f This is how the FA looks like without state 2:   ()  (  ()   )

59. 59 3 s f Finally we remove state 3…   ()  (  ()   )

60. 60 3 s f...so we concatenate strings s-3, loop in 3 and 3-f

61. 61 Now we can omit state 3! s f From s we have two choices, empty string or the long expression OR is represented by union , as usually

62. 62 So union the arrows... s f...and we are done!

63. 63 Converting FA to a Regular Expression- An Algorithm for State Elimination

64. 64 We expand our notion of NFA- to allow transitions on arbitrary regular expressions, not simply single symbols or. Successively eliminate states, replacing transitions that enter and leave a state with a more complicated regular expression, until eventually there are only two states left: a start state, an accepting state, and a single transition connecting them, labeled with a regular expression. The resulting regular expression is then our answer.

65. 65 To begin with, the automaton should have a start state that has no transitions into it (including self-loops), and which is not accepting. If your automaton does not obey this property, add a new, non-accepting start state s, and add a -transition from s to the original start state. The automaton should also have a single accepting final state with no transitions leaving it, and no self- loops. If your automaton does not have it, add a new final state q, change all other accepting states to non- accepting, and add -transitions from them to q. This change clearly doesn't change the language accepted by the automaton.

66. 66 Repeat the following steps, which eliminate a state: 1. Pick a non-start, non-accepting state q to eliminate. The state q will have i transitions in and j transitions out. Each will be labeled with a regular expression. For each of the ij combinations of transitions into and out of q, replace: pq r with p r And delete state q.

67. 67 2. If several transitions go between a pair of states, replace them with a single transition that is the union of the individual transitions. E.g. replace: p r with p r

68. 68 Example 1 342 1 34 1 4

69. 69 N.B. common mistake! means See example on s 46 and 47 in Sallings book i.e. NOT

70. 70 Minimizing DFA by Set Partitioning http://www.math.uu.se/~salling/Movies/SubsetConstruction.mov

71. 71 Minimization of DFA The deterministic finite automata are not always the smallest possible accepting the source language. There may be states with the same "acceptance behavior". This applies to states p and q, if for all input words, the automaton always or never moves to a final state from p and q.

72. 72 State Reduction by Set Partitioning (Särskiljandealgoritmen) The set partitioning technique is similar to one used for partitioning people into groups based on their responses to questionnaire. The following slides show the detailed steps for computing equivalent state sets of the starting DFA and constructing the corresponding reduced DFA.

73. 73 b ba b b a a b a b a 4 3 1 2 a 5 0 b a a, b 3 1,2 a b 4,5 0 a, b Starting DFA Reduced DFA

74. 74 Step 0: Partition the states into two groups accepting and non-accepting. State Reduction by Set Partitioning { 3, 4, 5 }{ 0, 1, 2 } P1P2 b ba bb a a b a b a 4 3 1 2 a 5 0

75. 75 Step 1: Get the response of each state for each input symbol. Notice that States 3 and 0 show different responses from the ones of the other states in the same set. P 1 P 2 p 1 p 1 p 1 p 2 p 1 p 1 a    a   {3, 4, 5 } {0,1,2 } b    b   p 2 p 1 p 1 p 2 p 1 p 1 State Reduction by Set Partitioning b ba bb a a b a b a 4 3 1 2 a 5 0 P1P2

76. 76 P 11 P 12 P 21 P 22 p 11 p 11 p 12 p 12 a   a   {4, 5} {3} {1, 2} {0} b   b   p 11 p 11 Step 2: Partition the sets according to the responses, and go to Step 1 until no partition occurs. No further partition is possible for the sets P11 and P21. So the final partition results are as follows: {4, 5} {3} {1, 2} {0}

77. 77 {4, 5} {3} {1, 2} {0} Minimized DFA consists of four states of the final partition, and the transitions are the one corresponding to the starting DFA. b a a, b 3 1,2 a b 4,5 0 a, b b ba bb a a b a b a 4 3 1 2 a 5 0 b a 3 1,2 a b 4,5 0 a, b Minimized DFA Starting DFA

78. 78 DFA Minimization Algorithm The algorithm P  { F, {Q-F}} while ( P is still changing) T  { } for each set s  P for each    partition s by  into s 1, s 2, …, s k T  T  s 1, s 2, …, s k if T  P then P  T Why does this work? Partition P  2 Q (power set) Start off with 2 subsets of Q {F} and {Q-F} While loop takes P i  P i+1 by splitting one or more sets P i+1 is at least one step closer to the partition with |Q | sets Maximum of |Q | splits Partitions are never combined Initial partition ensures that final states are intact This is a fixed-point algorithm!

79. 79 Set Partition Algorithm (Salling book) Example 2.39 We apply step by step set partition algorithm on the following DFA: Two strings x,y are distinguished by language (DFA) if there is a (distinguishing) string z such as that only one of strings xz, yz ends in accepting state (belongs to language).

80. 80 Non-accepting:Accepting: What happens when we read ? From 1 and 6 by a we reach 3 which is accepting. They form a special group. What happens when we read ? We search for strings that are distinguished by DFA! Distinguishing string: or From 5 we end in 6 by b, leaving its group.

81. 81 The minimal automaton has four states:

82. 82 The Chomsky Hierarchy

83. 83 Regular Languages Context-Free Languages Non-regular languages

84. 84 Some Additional Examples for your excersize

85. 85 NFA N for (a+b)*abb Example of subset construction for NFA  DFA

86. 86 Translation table for DFA STATE INPUT SYMBOL ab ABCDEABCDE BBBBBBBBBB CDCECCDCEC Example of subset construction for NFA  DFA

87. 87 Result of applying the subset construction of NFA for (a+b)*abb. Example of subset construction for NFA  DFA

88. 88 Another Example of State Elimination

89. 89 Another Example of State Elimination

90. 90 Another Example of State Elimination Resulting Regular Expression

91. 91 In General Removing states

92. 92 Obtaining the final regular expression

93. 93 Example: From an NFA to a DFA states ab ABC BBD CBC DBE EBC AB C D E ab b a bb b a a a

94. 94 Example: From an NFA to a DFA states ab ABA BBD DBE EBA ABD E a b b b a a a b

95. 95 Example: DFA Minimization s0s0 a s1s1 b s3s3 b s4s4 s2s2 a b b a a a b s 0, s 2 a s1s1 b s3s3 b s4s4 b a a a b

96. 96 Example: DFA Minimization What about a ( b + c ) * ? First, the subset construction: q0q0 q1q1 a q4q4 q5q5 b q6q6 q7q7 c q3q3 q8q8 q2q2 q9q9 s3s3 s2s2 s0s0 s1s1 c b a b b c c Final states

97. 97 Example: DFA Minimization Then, apply the minimization algorithm To produce the minimal DFA s3s3 s2s2 s0s0 s1s1 c b a b b c c s0s0 s1s1 a b, c final states

98. 98 Grammars

99. 99 Grammars Grammars express languages Example: the English language

100. 100 barksverb singsverb   dognoun birdnoun   thearticle a  

101. 101 A derivation of “the bird sings”:         birdthe verbbirdthe verbnounthe verbnounarticle predicatenounarticle predicatephrasenounsentence sings _

102. 102 A derivation of “a dog barks”: barksdoga verbdoga verbnouna verbnounarticle verbphrasenoun predicatephrasenounsentence       _ _

103. 103 The language of the grammar:

104. 104 Notation Non-terminal (Variable) Terminal Production rule

105. 105 Example Derivation of sentence: Grammar:

106. 106 Grammar: Derivation of sentence

107. 107 Other derivations

108. 108 The language of the grammar

109. 109 Formal Definition Grammar Set of variables Set of terminal symbols Start variable Set of production rules

110. 110 Example Grammar

111. 111 Sentential Form A sentence that contains variables and terminals Example sentential formsSentence (sats)

112. 112 We write: Instead of:

113. 113 In general we write if By default ()

114. 114 Example Grammar Derivations

115. 115 baaaaaSbbbbaaSbb   S   Grammar Example Derivations

116. 116 Another Grammar Example Derivations Grammar

117. 117 More Derivations

118. 118 The Language of a Grammar For a grammar with start variable String of terminals

119. 119 Example For grammar Since

120. 120 Notation

121. 121 Linear Grammars

122. 122 Linear Grammars Grammars with at most one variable (non-terminal) at the right side of a production Examples:

123. 123 A Non-Linear Grammar Grammar

124. 124 A Linear Grammar Grammar

125. 125 Right-Linear Grammars All productions have form: or Example

126. 126 Left-Linear Grammars All productions have form or Example

127. 127 Regular Grammars A grammar is a regular grammar if and only if it is right-linear or left-linear.

128. 128 Regular Grammars Generate Regular Languages

129. 129 Theorem Languages Generated by Regular Grammars Regular Languages

130. 130 Theorem - Part 1 Languages Generated by Regular Grammars Regular Languages Any regular grammar generates a regular language

131. 131 Theorem - Part 2 Any regular language is generated by a regular grammar Languages Generated by Regular Grammars Regular Languages

132. 132 Proof – Part 1 The language generated by any regular grammar is regular Languages Generated by Regular Grammars Regular Languages

133. 133 The case of Right-Linear Grammars Let be a right-linear grammar We will prove: is regular Proof idea We will construct NFA with

134. 134 Grammar is right-linear Example

135. 135 Construct NFA such that every state is a grammar variable: special final state

136. 136 Add edges for each production:

137. 137

138. 138

139. 139

140. 140

141. 141

142. 142 Grammar NFA

143. 143 In General A right-linear grammar has variables: and productions: or

144. 144 We construct the NFA such that: each variable corresponds to a node: special final state ….

145. 145 For each production: we add transitions and intermediate nodes ……… Example: http://www.cs.duke.edu/csed/jflap/tutorial/grammar/toFA/index.html Convert Right-Linear Grammar to FA by JFLAP

146. 146 Example

147. 147 The case of Left-Linear Grammars Let be a left-linear grammar We will prove: is regular Proof idea We will construct a right-linear grammar with

148. 148 Since is left-linear grammar the productions look like:

149. 149 Construct right-linear grammar In :

150. 150 Construct right-linear grammar In :

151. 151 It is easy to see that: Since is right-linear, we have: Regular Language Regular Language Regular Language

152. 152 Proof - Part 2 Any regular language is generated by some regular grammar Languages Generated by Regular Grammars Regular Languages

153. 153 Proof idea Any regular language is generated by some regular grammar Construct from a regular grammar such that Since is regular there is an NFA such that

154. 154 Example

155. 155 Convert to a right-linear grammar

156. 156 In General For any transition: Add production: variableterminalvariable

157. 157 For any final state: Add production:

158. 158 Since is right-linear grammar is also a regular grammar with

159. 159 Regular Grammars A regular grammar is any right-linear or left-linear grammar Examples

160. 160 Observation Regular grammars generate regular languages Examples

161. 161 Regular Languages Chomsky’s Language Hierarchy Non-regular languages

162. 162 Application: Compiler Lexical Analysis (Lexikalisk analys i Kompilatorteori)

163. 163 What is a compiler? A compiler is program that translates a source language into an equivalent target language while (i > 3) { a[i] = b[i]; i ++ } mov eax, ebx add eax, 1 cmp eax, 3 jcc eax, edx C program assembly program compiler does this

164. 164 What is a compiler? class foo { int bar;... } struct foo { int bar;... } Java program compiler does this C program

165. 165 What is a compiler? class foo { int bar;... }......................... Java program compiler does this Java virtual machine program

166. 166 Phases of a compiler Lexical Analyzer Scanner Parser Semantic Analyzer Source Program Syntax Analyzer Tokens Parse Tree Abstract Syntax Tree with attributes

167. 167 Compilation in a Nutshell Source code (character stream) Lexical analysis Parsing Token stream Abstract syntax tree (AST) Semantic Analysis if (b == 0) a = b; if(b)a=b;0== if == b0 = ab if == int b int 0 = int a lvalue int b boolean Decorated AST int ; ;

168. 168 Stages of Analysis Lexical Analysis – breaking the input up into individual words/tokens Syntax Analysis – parsing the phrase structure of the program Semantic Analysis – calculating the program’s meaning

169. 169 Lexical Analyzer Input is a stream of characters Produces a stream of names, keywords & punctuation marks Discards white space and comments

170. 170 Recognize “ tokens ” in a program source code. The tokens can be variable names, reserved words, operators, numbers, … etc. Each kind of token can be specified as an RE, e.g., a variable name is of the form [A-Za- z][A-Za-z0-9]*. We can then construct an -NFA to recognize it automatically. Lexical Analyzer

171. 171 By putting all these -NFA ’ s together, we obtain one which can recognize all different kinds of tokens in the input string. We can then convert this -NFA to NFA and then to DFA, and implement this DFA as a deterministic program - the lexical analyzer. Lexical Analyzer

172. 172 RENFADFAMinimal DFA Thompson’s Contruction Subset Contruction Hopcroft Minimization