1. 1 Module 12 Computation and Configurations –Formal Definition –Examples

2. 2 Definitions Configuration –Functional Definition Given the original program and the current configuration of a computation, someone should be able to complete the computation –Contents of a configuration for a C++ program current instruction to be executed current value of all variables Computation –Complete sequence of configurations

3. 3 Computation 1 1 int main(int x,y) { 2 int r = x % y; 3 if (r== 0) goto 8; 4 x = y; 5 y = r; 6 r = x % y; 7 goto 3; 8 return y; } Input: 10 3 Line 1, x=?,y=?,r=? Line 2, x=10, y=3,r=? Line 3, x=10, y=3, r=1 Line 4, x=10, y=3, r=1 Line 5, x= 3, y=3, r=1 Line 6, x=3, y=1, r=1 Line 7, x=3, y=1, r=0 Line 3, x=3, y=1, r=0 Line 8, x=3, y=1, r=0 Output is 1

4. 4 Computation 2 int main(int x,y) { 2 int r = x % y; 3 if (r== 0) goto 8; 4 x = y; 5 y = r; 6 r = x % y; 7 goto 3; 8 return y; } Input: 53 10 Line 1, x=?,y=?,r=? Line 2, x=53, y=10, r=? Line 3, x= 53, y=10, r=3 Line 4, x=53, y=10, r=3 Line 5, x=10, y=10, r=3 Line 6, x=10, y=3, r=3 Line 7, x=10, y=3, r=1 Line 3, x=10, y=3, r=1...

5. 5 Computations 1 and 2 Line 1, x=?,y=?,r=? Line 2, x=53, y=10, r=? Line 3, x= 53, y=10, r=3 Line 4, x=53, y=10, r=3 Line 5, x=10, y=10, r=3 Line 6, x=10, y=3, r=3 Line 7, x=10, y=3, r=1 Line 3, x=10, y=3, r=1... Line 1, x=?,y=?,r=? Line 2, x=10, y=3,r=? Line 3, x=10, y=3, r=1 Line 4, x=10, y=3, r=1 Line 5, x= 3, y=3, r=1 Line 6, x=3, y=1, r=1 Line 7, x=3, y=1, r=0 Line 3, x=3, y=1, r=0 Line 8, x=3, y=1, r=0 Output is 1

6. 6 Observation int main(int x,y) { 2 int r = x % y; 3 if (r== 0) goto 8; 4 x = y; 5 y = r; 6 r = x % y; 7 goto 3; 8 return y; } Line 3, x= 10, y=3, r=1 Program and current configuration –Together, these two pieces of information are enough to complete the computation –Are they enough to determine what the original input was? No! Both previous inputs, 10 3 as well as 53 10 eventually reached the same configuration (Line 3, x=10, y=3, r=1)

7. 7 Module 13 Studying the internal structure of REC, the set of solvable problems –Complexity theory overview –Automata theory preview Motivating Problem –string searching

8. 8 Studying REC Complexity Theory Automata Theory

9. 9 Current picture of all languages  ll Languages RE-REC  ll languages - RE Half Solvable Not even half solvable Which language class should be studied further? REC Solvable

10. 10 Complexity Theory In complexity theory, we differentiate problems by how hard a problem is to solve –Remember, all problems in REC are solvable Which problem is harder and why? –Max: Input: list of n numbers Task: return largest of the n numbers –Element Input: list of n numbers Task: return any of the n numbers REC RE - REC All languages - RE

11. 11 Resource Usage * How do we formally measure the hardness of a problem? We measure the resources required to solve input instances of the problem Typical resources are? We need a notion of size of an input instance –Obviously larger input instances require more resources to solve

12. 12 Poly Language Class * Informal Definition: A problem L 1 is easier than problem L 2 if problem L 1 can be solved in less time than problem L 2. Poly: the set of problems which can be solved in polynomial time (typically referred to as P, not Poly) Major goal: Identify whether or not a problem belongs to Poly Poly Rest of REC REC RE - REC All languages - RE

13. 13 Working with Poly PolyRest of REC How do you prove a problem L is in Poly? How do you prove a problem L is not in Poly? –We are not very good at this. –For a large class of interesting problems, we have techniques (polynomial-time answer- preserving input transformations) that show a problem L probably is not in Poly, but few which prove it.

14. 14 Examples Shortest Path Problem Input –Graph G –nodes s and t Task –Find length of shortest path from s to t in G Longest Path Problem Input –Graph G –nodes s and t Task –Find length of longest path from s to t in G PolyRest of REC Which problem is provably solvable in polynomial time?

15. 15 Automata Theory In automata theory, we will define new models of computation which we call automata or grammars –Finite State Automata (FSA) –Context Free Grammars (CFG) Key concept –FSA’s and CFG’s are restricted models of computation FSA’s and CFG’s cannot solve all the problems that C++ programs can –We then identify which problems can be solved using FSA’s and CFG’s REC RE - REC All languages - RE

16. 16 New language classes REC is the set of solvable languages when we start with a general model of computation like C++ programs We want to identify which problems in REC can be solved when using these restricted automata Rest of REC Solvable by FSA’s and CFG’s Solvable by CFG’s REC RE - REC All languages - RE

17. 17 Recap * Complexity Theory –Studies structure of the set of solvable problems –Method: analyze resources (processing time) used to solve a problem Automata Theory –Studies structure of the set of solvable problems –Method: define automata with restricted capabilities and resources and see what they can solve (and what they cannot solve) –This theory also has important implications in the development of programming languages and compilers

18. 18 Motivating Problem String Searching

19. 19 String Searching Input –String x –String y Tasks –Return location of y in string x –Does string y occur in string x? Can you identify applications of this type of problem in real life? Try and develop an efficient solution to this problem.

20. 20 String Searching II Input –String x –pattern y Tasks –Return location of y in string x –Does pattern y occur in string x? Pattern –[anything].html –$EN4$$

21. 21 String Searching We will show an easy way to solve these string searching problems In particular, we will show that we can solve these problems in the following manner –Write down the pattern –The computer automatically turns this into a program which performs the actual string search

22. 22 Module 14 Regular languages –Inductive definitions –Regular expressions syntax semantics

23. 23 Regular Languages (Regular Expressions)

24. 24 Regular Languages New language class –Elements are languages We will show that this language class is identical to LFSA –Language class to be defined by Finite State Automata (FSA) –Once we have shown this, we will use the term “regular languages” to refer to this language class

25. 25 Inductive Definition of Integers * Base case definition –0 is an integer Inductive case definition –If x is an integer, then x+1 is an integer x-1 is an integer Completeness –Only numbers generated using the above rules are integers

26. 26 Inductive Definition of Regular Languages Base case definition –Let  denote the alphabet –{} is a regular language –{a} is a regular language for any character a in  Inductive case definition –If L 1 and L 2 are regular languages, then L 1 union L 2 is a regular language L 1 concatenate L 2 is a regular language L 1 * is a regular language Completeness –Only languages generated using above rules are regular languages

27. 27 Proving a language is regular * Prove that {aa, bb} is a regular language –{a} and {b} are regular languages base case of definition –{aa} = {a}{a} is a regular language concatenation rule –{bb} = {b}{b} is a regular language concatenation rule –{aa, bb} = {aa} union {bb} is a regular language union rule Typically, we will not go through this process to prove a language is regular

28. 28 Regular Expressions How do we describe a regular language? –Use set notation {aa, bb, ab, ba}* {a}{a,b}*{b} –Use regular expressions R Inductive def of regular languages and regular expressions on page 72 (aa+bb+ab+ba)* a(a+b)*b

29. 29 R and L(R) * How we interpret a regular expression –What does a regular expression R mean to us? aaba represents the regular language {aaba}  represents the regular language {} aa+bb represents the regular language {aa, bb} –We use L(R) to denote the regular language represented by regular expression R.

30. 30 Precedence rules What is L(ab+c*)? –Possible answers: {a}({b} union {c}*} ({a}{b,c})* ({ab} union {c})* {ab} union {c}* –Must know precedence rules * first, then concatenation, then +

31. 31 Precedence rules continued Precedence rules similar to those for arithmetic expressions –ab+c 2 (a times b) + (c times c) exponentiation first, then multiplication, then addition Think of Kleene closure as exponentiation, concatenation as multiplication, and union as addition and the precedence rules are identical

32. 32 Regular expressions are strings * Let L be a regular language over the alphabet  –A regular expression R for L is just a string over the alphabet  union {(, ), +, *,  }. –The set of legal regular expressions is itself a language over the alphabet  union {(, ), +, *} , a*aba are strings in the language of legal reg. exp. )(, *a* are strings NOT in the language of legal reg. exp.

33. 33 Semantics * We give a regular expression R meaning when we interpret it to represent L(R). –aaba is just a string –we interpret it to represent the language {aaba}. We do similar things with arithmetic expressions –10+7 2 is just a string –We interpret this string to represent the number 59

34. 34 Key fact * A language L is a regular language iff there exists a reg. exp. R such that L(R) = L –When I ask for a proof that a language L is regular, rather than going through the inductive proof we saw earlier, I expect you to give me a regular expression R s.t. L(R) = L

35. 35 Summary Regular expressions are strings –syntax for legal regular expressions –semantics for interpreting regular expressions Regular languages are a new language class –A language L is regular iff there exists a regular expression R s.t. L(R) = L We will show that the regular languages are identical to LFSA

36. 36 Module 15 FSA’s –Defining FSA’s –Computing with FSA’s Defining L(M) –Defining language class LFSA –Comparing LFSA to set of solvable languages (REC)

37. 37 Finite State Automata New Computational Model

38. 38 Tape We assume that you have already seen FSA’s in CSE 260 –If not, review material in reference textbook Only data structure is a tape –Input appears on tape followed by a B character marking the end of the input –Tape is scanned by a tape head that starts at leftmost cell and always scans to the right

39. 39 Data type/States The only data type for an FSA is char The instructions in an FSA are referred to as states Each instruction can be thought of as a switch statement with several cases based on the char being scanned by the tape head

40. 40 Example program 1 switch(current tape cell) { case a: goto 2 case b: goto 2 case B: return yes } 2 switch (current tape cell) { case a: goto 1 case b: goto 1 case B: return no; }

41. 41 New model of computation FSA M=(Q, ,q 0,A,  ) –Q = set of states = {1,2} –  = character set = {a,b} don’t need B as we see below –q 0 = initial state = 1 –A = set of accepting (final) states = {1} A is the set of states where we return yes on B Q-A is set of states that return no on B –  = state transition function 1 switch(current tape cell) { case a: goto 2 case b: goto 2 case B: return yes } 2 switch (current tape cell) { case a: goto 1 case b: goto 1 case B: return no; }

42. 42 Textual representations of  * 1 switch(current tape cell) { case a: goto 2 case b: goto 2 case B: return yes } 2 switch (current tape cell) { case a: goto 1 case b: goto 1 case B: return no; } 1 2 ab 11 22  (1,a) = 2,  (1,b)=2,  (2,a)=1,  (2,b) = 1 {(1,a,2), (1,b,2), (2,a,1), (2,b,1)}

43. 43 Transition diagrams 1 2 a,b 1 switch(current tape cell) { case a: goto 2 case b: goto 2 case B: return yes } 2 switch (current tape cell) { case a: goto 1 case b: goto 1 case B: return no; } Note, this transition diagram represents all 5 components of an FSA, not just the transition function 

44. 44 Exercise FSA M = (Q, , q 0, A,  ) –Q = {1, 2, 3} –  = {a, b} –q 0 = 1 –A = {2,3} –  : {  (1,a) = 1,  (1,b) = 2,  (2,a)= 2,  (2,b) = 3,  (3,a) = 3,  (3,b) = 1} Draw this FSA as a transition diagram

45. 45 Transition Diagram 1 2 3 a a a b b b

46. 46 Computing with FSA’s

47. 47 Computation Example * 1 2 3 a a a b b b Input: aabbaa

48. 48 Computation of FSA’s in detail A computation of an FSA M on an input x is a complete sequence of configurations We need to define –Initial configuration of the computation –How to determine the next configuration given the current configuration –Halting or final configurations of the computation

49. 49 Given an FSA M and an input string x, what is the initial configuration of the computation of M on x? –(q 0,x) –Examples x = aabbaa (1, aabbaa) x = abab (1, abab) x = (1, ) Initial Configuration 1 2 3 a a a b b b FSA M

50. 50 (1, aabbaa) ├ M (1, abbaa) –config 1 “yields” config 2 in one step using FSA M (1,aabbaa) ├ M (2, baa) –config 1 “yields” config 2 in 3 steps using FSA M (1, aabbaa) ├ M (2, baa) –config 1 “yields” config 2 in 0 or more steps using FSA M Comment: –├ M determined by transition function  –There must always be one and only one next configuration If not, M is not an FSA Definition of ├ M 1 2 3 a a a b b b 3 FSA M *

51. 51 Halting configuration –(q, ) –Examples (1, ) (3, ) Accepting Configuration –State in halting configuration is in A Rejecting Configuration –State in halting configuration is not in A Halting Configurations * 1 2 3 a a a b b b FSA M

52. 52 Two possibilities for M running on x –M accepts x M accepts x iff the computation of M on x ends up in an accepting configuration (q 0, x) ├ M (q, ) where q is in A –M rejects x M rejects x iff the computation of M on x ends up in a rejecting configuration (q 0, x) ├ M (q, ) where q is not in A –M does not loop or crash on x Why? FSA M on x b b b a a a FSA M * *

53. 53 –For the following input strings, does M accept or reject? aa aabba aab babbb Examples b b b a a a FSA M

54. 54 Notation from the book  (q, c) = p  k (q, x) = p –k is the length of x  * (q, x) = p Examples –  (1, a) = 1 –  (1, b) = 2 –  4 (1, abbb) = 1 –  * (1, abbb) = 1 –   (2, baaaaa) = 3 Definition of  * (q, x) 1 2 3 a a a b b b FSA M

55. 55 L(M) or Y(M) –The set of strings M accepts Basically the same as Y(P) from previous unit –We say that M accepts/decides/recognizes/solves L(M) Remember an FSA will not loop or crash –What is L(M) (or Y(M)) for the FSA M above? N(M) –Rarely used, but it is the set of strings M rejects LFSA –L is in LFSA iff there exists an FSA M such that L(M) = L. L(M) and LFSA * b b b a a a FSA M

56. 56 LFSA Unit Overview Study limits of LFSA –Understand what languages are in LFSA Develop techniques for showing L is in LFSA –Understand what languages are not in LFSA Develop techniques for showing L is not in LFSA Prove Closure Properties of LFSA Identify relationship of LFSA to other language classes

57. 57 Comparing language classes Showing LFSA is a subset of REC, the set of solvable languages

58. 58 LFSA subset REC Proof –Let L be an arbitrary language in LFSA –Let M be an FSA such that L(M) = L M exists by definition of L in LFSA –Construct C++ program P from FSA M –Argue P solves L –L is solvable

59. 59 Visualization LFSA REC FSA’s C++ Programs L L M P Let L be an arbitrary language in LFSA Let M be an FSA such that L(M) = L M exists by definition of L in LFSA Construct C++ program P from FSA M Argue P solves L There exists a program P which solves L L is solvable

60. 60 Construction The construction is an algorithm which solves a problem with a program as input –Input to A: FSA M –Output of A: C++ program P such that P solves L(M) –How do we do this? Construction Algorithm FSA M Program P

61. 61 Comparing computational models The previous slides show one method for comparing the relative power of two different computational models –Computational model CM 1 is at least as general or powerful as computational model CM 2 if Any program P 2 from computational model CM 2 can be converted into an equivalent program P 1 in computational model CM 1. –Question: How can we show two computational models are equivalent?

62. 62 Module 16 Distinguishability –Definition –Help in designing/debugging FSA’s

63. 63 Distinguishability

64. 64 Questions Let L be the set of strings over {a,b} which end with aaba. Let M be an FSA such that L(M) = L. Questions –Can aaba and aab end up in the same state of M? Why or why not? –How about aa and aab? –How about or a? –How about b or bb? –How about  or bbab?

65. 65 Definition * String x is distinguishable from string y with respect to language L iff there exists a string z such that –xz is in L and yz is not in L OR –xz is not in L and yz is in L When reviewing, identify the z for pair of strings on the previous slide

66. 66 Questions Let L be the set of strings over {a,b} that have length 2 mod 5 or 4 mod 5. Let M be an FSA such that L(M) = L. Questions –Are aa and aab distinguishable with respect to L? Can they end up in the same state of M? –How about aa and aaba? –How about and a? –How about b and aabbaa?

67. 67 One design method –Is in L? Implication? –Is a distinguishable from  wrt L? Implication? –Is b distinguishable from  wrt L? Implication? –Is b distinguishable from  a wrt L? Implication? L = set of strings x over {a,b} such that length of x is 2 or 4 mod 5 Design an FSA to accept L

68. 68 Design continued –Is aa distinguishable from  wrt L? Implication? –Is aa distinguishable from  a wrt L? Implication? L = set of strings x over {a,b} such that length of x is 2 or 4 mod 5 Design an FSA to accept L

69. 69 Design continued –What strings would we compare ab to? –What results do we get? –Implications? –How about ba? –How about bb? L = set of strings x over {a,b} such that length of x is 2 or 4 mod 5 Design an FSA to accept L

70. 70 Design continued –We can continue in this vein, but it could go on forever –Now lets try something different –Consider string. What set of strings are indistinguishable from it wrt L? Implications? L = set of strings x over {a,b} such that length of x is 2 or 4 mod 5 Design an FSA to accept L

71. 71 Design continued –Consider string a. What set of strings are indistinguishable from it wrt L? Implications? –Consider string aa. What set of strings are indistinguishable from it wrt L? Implications? L = set of strings x over {a,b} such that length of x is 2 or 4 mod 5 Design an FSA to accept L

72. 72 Debugging an FSA Do essentially the same thing –Identify some strings which end up in each state –Try and generalize each state to describe the language of strings which end up at that state.

73. 73 Example 1 aaa a,b b b b b a IIIIIIIVV VI

74. 74 Example 2 IIIIIIIV V aaa a,b b b b b a

75. 75 Example 3 I II III IV a b a a b b a,b

76. 76 Module 17 Closure Properties of Language class LFSA –Remember ideas used in solvable languages unit –Set complement –Set intersection, union, difference, symmetric difference

77. 77 LFSA is closed under set complement If L is in LFSA, then L c is in LFSA Proof –Let L be an arbitrary language in LFSA –Let M be the FSA such that L(M) = L M exists by definition of L in LFSA –Construct FSA M’ from M –Argue L(M’) = L c –There exists an FSA M’ such that L(M’) = L c –L c is in LFSA

78. 78 Visualization Let L be an arbitrary language in LFSA Let M be the FSA such that L(M) = L M exists by definition of L in LFSA Construct FSA M’ from M Argue L(M’) = L c L c is in LFSA LcLc L LFSA FSA’s M M’

79. 79 Construct FSA M’ from M What did we do when we proved that REC, the set of solvable languages, is closed under set complement? Construct program P’ from program P Can we translate this to the FSA setting?

80. 80 Construct FSA M’ from M M = (Q, , q 0, A,  ) M’ = (Q’,  ’, q’, A’,  ’) –M’ should say yes when M says no –M’ should say no when M says yes –How? Q’ = Q  ’ =  q’ = q 0  ’ =  A’ = Q-A

81. 81 Example 1 2 3 a a a b b b FSA M 1 2 3 a a a b b b FSA M’ Q’ = Q  ’ =  q’ = q 0  ’ =  A’ = Q-A

82. 82 Construction is an algorithm * Set Complement Construction –Algorithm Specification Input: FSA M Output: FSA M’ such that L(M’) = L(M) c –Comments This algorithm can be in any computational model. –It does not have to be (and typically is not) an FSA These set closure constructions are useful. –More on this later Construction Algorithm FSA M FSA M’

83. 83 Specification of the algorithm Your algorithm must give a complete specification of M’ in terms of M –Example: Let input FSA M = (Q, , q 0, A,  ) Output FSA M’ = (Q’,  ’, q’, A’,  ’) where –Q’ = Q –  ’ =  –q’ = q 0 –  ’ =  –A’ = Q-A When I ask for such a construction algorithm specification, this type of answer is what I am looking for. Further algorithmic details on how such an algorithm would work are unnecessary. Construction Algorithm FSA M FSA M’

84. 84 LFSA closed under Set Intersection Operation (also set union, set difference, and symmetric difference)

85. 85 LFSA closed under set intersection operation * Let L 1 and L 2 be arbitrary languages in LFSA Let M 1 and M 2 be FSA’s s.t. L(M 1 ) = L 1, L(M 2 ) = L 2 –M 1 and M 2 exist by definition of L 1 and L 2 in LFSA Construct FSA M 3 from FSA’s M 1 and M 2 Argue L(M 3 ) = L 1 intersect L 2 There exists FSA M 3 s.t. L(M 3 ) = L 1 intersect L 2 L 1 intersect L 2 is in LFSA

86. 86 Visualization Let L 1 and L 2 be arbitrary languages in LFSA Let M 1 and M 2 be FSA’s s.t. L(M 1 ) = L 1, L(M 2 ) = L 2 M 1 and M 2 exist by definition of L 1 and L 2 in LFSA Construct FSA M 3 from FSA’s M 1 and M 2 Argue L(M 3 ) = L 1 intersect L 2 There exists FSA M 3 s.t. L(M 3 ) = L 1 intersect L 2 L 1 intersect L 2 is in LFSA L 1 intersect L 2 L1L1 L2L2 LFSA M3M3 M1M1 M2M2 FSA’s

87. 87 Algorithm Specification Input –Two FSA’s M 1 and M 2 Output –FSA M 3 such that L(M 3 ) = L(M 1 ) intersection L(M 2 ) FSA M 1 FSA M 2 FSA M 3 Alg

88. 88 Use Old Ideas Key concept: Try ideas from previous closure property proofs Example –How did the algorithm that was used to prove that REC is closed under set intersection work? –If we adapt this approach, what should M 3 do with respect to M 1, M 2, and the input string? FSA M 1 FSA M 2 FSA M 3 Alg

89. 89 1 Run M 1 and M 2 Simultaneously 0 2 0 0 0 0 1 1 1 1 M1M1 A B 0 10,1 M2M2,A 0,A1,A2,A,B 0,B1,B2,B M3M3 What happens when M 1 and M 2 run on input string 11010? FSA M 1 FSA M 2 FSA M 3 Alg

90. 90 Construction * Input –FSA M 1 = (Q 1,  , q 1,  , A 1 ) –FSA M 2 = (Q 2,  , q 2,  , A 2 ) Output –FSA M 3 = (Q 3,  , q 3,  , A 3 ) –What is Q 3 ? Q 3 = Q 1 X Q 2 where X is cartesian product In this case, Q 3 = {(,A), (,B), (0,A), (0,B), (1,A), (1,B), (2,A), (2,B)} –What is  3 ?  3 =  1 =  2 In this case,  3 = {0,1} 1 0 2 0 0 0 0 1 1 1 1 M1M1 A B 0 10,1 M2M2

91. 91 Construction * Input –FSA M 1 = (Q 1,  , q 1,  , A 1 ) –FSA M 2 = (Q 2,  , q 2,  , A 2 ) Output –FSA M 3 = (Q 3,  , q 3,  , A 3 ) –What is q 3 ? q 3 = (q 1, q 2 ) In this case, q 3 = (,A) –What is A 3 ? A 3 = {(p, q) | p in A 1 and q in A 2 } In this case, A 3 = {(0,B)} 1 0 2 0 0 0 0 1 1 1 1 M1M1 A B 0 10,1 M2M2

92. 92 Construction Input –FSA M 1 = (Q 1,  , q 1,  , A 1 ) –FSA M 2 = (Q 2,  , q 2,  , A 2 ) Output –FSA M 3 = (Q 3,  , q 3,  , A 3 ) –What is  3 ? For all p in Q 1, q in Q 2, a in ,  3 ((p,q),a) = (  1 (p,a),  2 (q,a)) In this case, –  3 ((0,A),0) = (  1 (0,0),  2 (A,0)) – = (0,B) –  3 ((0,A),1) = (  1 (0,1),  2 (A,1)) – = (1,A) 1 0 2 0 0 0 0 1 1 1 1 M1M1 A B 0 10,1 M2M2

93. 93 Example Summary 1 0 2 0 0 01 1 1 M1M1 A B 0 10,1 M2M2,A 0,A1,A2,A,B 0,B 1,B2,B M3M3 01 0 1 0 1 0 1 0 1 0 1 0 1

94. 94 Observation Input –FSA M 1 = (Q 1,  , q 1,  , A 1 ) –FSA M 2 = (Q 2,  , q 2,  , A 2 ) Output –FSA M 3 = (Q 3,  , q 3,  , A 3 ) –What is A 3 ? A 3 = {(p, q) | p in A 1 and q in A 2 } What if operation were different? –Set union, set difference, symmetric difference

95. 95 Observation continued * Input –FSA M 1 = (Q 1,  , q 1,  , A 1 ) –FSA M 2 = (Q 2,  , q 2,  , A 2 ) Output –FSA M 3 = (Q 3,  , q 3,  , A 3 ) –What is A 3 ? Set intersection: A 3 = {(p, q) | p in A 1 and q in A 2 } Set union: A 3 = {(p, q) | p in A 1 or q in A 2 } Set difference: A 3 = {(p, q) | p in A 1 and q not in A 2 } Symmetric difference: A 3 = {(p, q) | (p in A 1 and q not in A 2 ) or (p not in A 1 and q in A 2 ) }

96. 96 Observation conclusion LFSA is closed under –set intersection –set union –set difference –symmetric difference The constructions used to prove these closure properties are essentially identical

97. 97 Comments * You should be able to execute this algorithm –Convert two FSA’s into a third FSA with the correct properties. You should understand the idea behind this algorithm –The third FSA essentially runs both input FSA’s simultaneously on any input string –How we set A 3 depending on the specific set operation You should understand how this algorithm can be used to simplify design of FSA’s You should be able to construct new algorithms for new closure property proofs

98. 98 Comparison * L 1 intersect L 2 L1L1 L2L2 LFSA M3M3 M1M1 M2M2 FSA’s LFSA REC FSA’s C++ Programs L L M P

99. 99 Module 18 NFA’s –nondeterministic transition functions computations are trees, not paths –L(M) and LNFA LFSA subset of LNFA

100. 100 Nondeterministic Finite State Automata NFA’s

101. 101 Change:  is a relation For an FSA M,  (q,a) results in one and only one state for all states q and characters a. –That is,  is a function For an NFA M,  (q,a) can result in a set of states –That is,  is now a relation –Next step is not determined (nondeterministic)

102. 102 Example NFA aaab a,b Why is this only an NFA and not an FSA? Identify as many reasons as you can.

103. 103 Computing with NFA’s Configurations: same as they are for FSA’s Computations are different –Initial configuration is identical –However, there may be several next configurations or there may be none. Computation is no longer a “path” but is now a “graph” (often a tree) rooted at the initial configuration –Definition of halting, accepting, and rejecting configurations is identical –Definition of acceptance must be modified

104. 104 Computation Graph (Tree) aaab a,b Input string aaaaba (1, aaaaba) (1, aaaba)(2, aaaba) (1, aaba)(2, aaba)(3, aaba)(1, aba)(2, aba)(3, aba)crash(1, ba)(2, ba)(3, ba) (1, a)(4, a) (1, )(2, )(5, )

105. 105 Definition of ├ unchanged * aaab a,b Input string aaaaba (1, aaaaba)├ (1, aaaba) (1, aaaaba)├ (2, aaaba) (1, aaaaba)├ 3 (1, aba) (1, aaaaba)├ 3 (3, aba) (1, aaaaba)├ * (2, aba) (1, aaaaba)├ * (3, aba) (1, aaaaba)├ * (1, ) (1, aaaaba)├ * (5, ) (1, aaaaba) (1, aaaba)(2, aaaba) (1, aaba)(2, aaba)(3, aaba) (1, aba)(2, aba)(3, aba)crash (1, ba)(2, ba)(3, ba) (1, a)(4, a) (1, )(2, )(5, )

106. 106 Acceptance and Rejection aaab a,b Input string aaaaba M accepts string x if one of the configurations reached is an accepting configuration (q 0, x) ├ * (f, ),f in A M rejects string x if all configurations reached are either not halting configurations or are rejecting configurations (1, aaaaba) (1, aaaba)(2, aaaba) (1, aaba)(2, aaba)(3, aaba) (1, aba)(2, aba)(3, aba)crash (1, ba)(2, ba)(3, ba) (1, a)(4, a) (1, )(2, )(5, )

107. 107 Comparison aaa a,b b b b b a FSA aaab a,b NFA

108. 108 Defining L(M) and LNFA M accepts string x if one of the configurations reached is an accepting configuration –(q 0, x) |- * (f, ),f in A M rejects string x if all configurations reached are either not halting configurations or are rejecting configurations L(M) (or Y(M)) N(M) LNFA –Language L is in language class LNFA iff

109. 109 Comparing language classes LFSA subset of LNFA

110. 110 LFSA subset LNFA Let L be an arbitrary language in LFSA Let M be the FSA such that L(M) = L –M exists by definition of L in LFSA Construct an NFA M’ such that L(M’) = L Argue L(M’) = L There exists an NFA M’ such that L(M’) = L L is in LNFA –By definition of L in LNFA

111. 111 Visualization LFSA LNFA FSA’s NFA’s L L M M’ Let L be an arbitrary language in LFSA Let M be an FSA such that L(M) = L M exists by definition of L in LFSA Construct NFA M’ from FSA M Argue L(M’) = L There exists an NFA M’ such that L(M’) = L L is in LNFA

112. 112 Construction We need to make M into an NFA M’ such that L(M’) = L(M) How do we accomplish this?

113. 113 Module 19 LNFA subset of LFSA –Theorem 4.1 on page 131 of Martin textbook –Compare with set closure proofs Main idea –A state in FSA represents a set of states in original NFA

114. 114 LNFA subset LFSA Let L be an arbitrary language in Let M be –M exists by definition of Construct an M’ such that L(M’) Argue L(M’) = There exists an M’ such that L(M’) = L is in –By definition of

115. 115 Visualization LNFA LFSA NFA’s FSA’s L L M M’ Let L be an arbitrary language in LNFA Let M be an NFA such that L(M) = L M exists by definition of L in LNFA Construct FSA M’ from NFA M Argue L(M’) = L There exists an FSA M’ such that L(M’) = L L is in LFSA

116. 116 Construction Specification We need an algorithm which does the following –Input: NFA M –Output: FSA M’ such that L(M’) = L(M)

117. 117 An NFA can be in several states after processing an input string x Difficulty * aaab a,b Input string aaaaba (1, aaaaba) (1, aaaba)(2, aaaba) (1, aaba)(2, aaba)(3, aaba) (1, aba)(2, aba)(3, aba)crash (1, ba)(2, ba)(3, ba) (1, a)(4, a) (1, )(2, )(5, )

118. 118 All strings which end up in the set of states {1,2,3} are indistinguishable with respect to L(M) Observation * aaab a,b Input string aaaaba (1, aaaaba) (1, aaaba)(2, aaaba) (1, aaba)(2, aaba)(3, aaba) (1, aba)(2, aba)(3, aba)crash (1, ba)(2, ba)(3, ba) (1, a)(4, a) (1, )(2, )(5, )

119. 119 Given an NFA M = (Q, ,q 0, ,A), the equivalent FSA M’ will have state set 2 Q (one state for each subset of Q) Example –In this case there are 5 states in Q –2 Q, the set of all subsets of Q, has 2 5 elements including {} and Q –The FSA M’ will have 2 5 states What strings end up in state {1,2,3} of M’? –The strings which end up in states 1, 2, and 3 of NFA M. –In this case, strings which do not contain aaba and end with aa such as aa, aaa, and aaaa. Idea aaab a,b Input string aaaaba

120. 120 Idea Illustrated aaab a,b Input string aaaaba (1,aaaaba)({1}, aaaaba) (1, aaaba)(2, aaaba)({1,2}, aaaba) (1, aaba)(2, aaba)(3, aaba) ({1,2,3}, aaba) (1, aba)(2, aba)(3, aba)({1,2,3}, aba) (1, ba)(2, ba)(3, ba)({1,2,3}, ba) (1, a)(4, a) ({1,4}, a) ({1,2,5}, ) (1, )(2, )(5, )

121. 121 Construction Input NFA M = (Q, , q 0, , A) Output FSA M’ = (Q’,  ’, q’,  ’, A’) –What is Q’? all subsets of Q including Q and {} In this case, Q’ = –What is  ’? We always make  ’ =  In this case,  ’ =  = {a,b} –What is q’? We always make q’ = {q 0 } In this case q’ = a,b aa NFA M 1 23

122. 122 Construction Input NFA M = (Q, , q 0, , A) Output FSA M’ = (Q’,  ’, q’,  ’, A’) –What is A’? Suppose a string x ends up in states 1 and 2 of the NFA M above. –Is x accepted by M? –Should {1,2} be an accepting state in FSA M’? Suppose a string x ends up in states 1 and 2 and 3 of the NFA M above. –Is x accepted by M? –Should {1,2,3} be an accepting state in FSA M’? Suppose p = {q 1, q 2, …, q k } where q 1, q 2, …, q k are in Q p is in A’ iff at least one of the states q 1, q 2, …, q k is in A In this case, A’ = a,b aa NFA M 1 23

123. 123 Construction Input NFA M = (Q, , q 0, , A) Output FSA M’ = (Q’,  ’, q’,  ’, A’) –What is  ’? If string x ends up in states 1 and 2 after being processed by the NFA above, where does string xa end up after being processed by the NFA above? Figuring out  ’(p,a) in general –Suppose p = {q 1, q 2, …, q k } where q 1, q 2, …, q k are in Q –Then  ’(p,a) =  (q 1,a) union  (q 2,a) union … union  (q k,a) »Similar to 2 FSA to 1 FSA construction –In this case »  ’({1,2},a) = a,b aa NFA M 1 23

124. 124 Construction Summary Input NFA M = (Q, , q 0, , A) Output FSA M’ = (Q’,  ’, q’,  ’, A’) –Q’ = all subsets of Q including Q and {} In this case, Q’ = {{}, {1}, {2}, {3}, {1,2}, {1,3}, {2,3}, {1,2,3}} –  ’ =  In this case,  ’ =  = {a,b} –q’ ={q 0 } In this case, q’ = {1} –A’ Suppose p = {q 1, q 2, …, q k } where q 1, q 2, …, q k are in Q p is in A’ iff at least one of the states q 1, q 2, …, q k is in A –  ’ Suppose p = {q 1, q 2, …, q k } where q 1, q 2, …, q k are in Q Then  ’(p,a) =  (q 1,a) union  (q 2,a) union … union  (q k,a) a,b aa NFA M 1 23

125. 125 Example Summary a,b aa NFA M 1 23 {1}{1,2}{1,2,3}{1,3} {}{2}{3}{2,3} a,b a a a a a b b b b b FSA M’

126. 126 Example Summary Continued a,b aa NFA M 1 23 {1}{1,2}{1,2,3}{1,3} {}{2}{3}{2,3} a,b a a a a a b b b b b FSA M’ These states cannot be reached from initial state and are unnecessary.

127. 127 Example Summary Continued a,b aa NFA M 1 23 {1}{1,2}{1,2,3}{1,3} a a a a b b b b Smaller FSA M’ By examination, we can see that state {1,3} is unnecessary. However, this is a case by case optimization. It is not a general technique or algorithm.

128. 128 Example 2 a,b ab Step 1: name the three states of NFA M ABC NFA M

129. 129 Step 2: transition table a,b ab ABC NFA M {A}{B}{} ab {B}{B,C}{B} {} {B,C}{B}{B,C}  ’({B,C},a) =  (B,a) U  (C,a) = {B} U {} = {B}  ’({B,C},b) =  (B,b) U  (C,b) = {B,C} U {} = {B,C}

130. 130 Step 3: accepting states a,b ab ABC NFA M {A}{B}{} ab {B} {B,C} {} {B,C}{B}{B,C} Which states should be accepting? Any state which includes an accepting state of M, in this case, C. A’ = {{B,C}}

131. 131 Step 4: Answer a,b ab ABC NFA M Initial state is {A} Set of final states A’ = {{B,C}} {A}{B}{} ab {B} {B,C} {} {B,C}{B}{B,C} This is sufficient. You do NOT need to turn this into a diagram.

132. 132 Step 5: Optional a,b ab ABC NFA M a ab {A}{B}{B,C} b a a,b {} b FSA M’

133. 133 Comments You should be able to execute this algorithm –You should be able to convert any NFA into an equivalent FSA. You should understand the idea behind this algorithm –For an FSA M’, strings which end up in the same state of M’ are indistinguishable wrt L(M’) –For an NFA M, strings which end up in the same set of states of M are indistinguishable wrt L(M)

134. 134 Comments You should understand the importance of this algorithm –Design tool We can design using NFA’s A computer will convert this NFA into an equivalent FSA –FSA’s can be executed by computers whereas NFA’s cannot (or at least cannot easily be run by computers) –Chaining together algorithms Perhaps it is easy to build NFA’s to accept L 1 and L 2 Use this algorithm to turn these NFA’s to FSA’s Use previous algorithm to build FSA to accept L 1 intersect L 2 You should be able to construct new algorithms for new closure property proofs

135. 135 Module 20 NFA’s with -transitions –NFA- ’s Formal definition Simplifies construction –LNFA- –Showing LNFA  is a subset of LNFA and therefore a subset of LFSA

136. 136 Defining NFA- ’s

137. 137 Change:  -transitions We now allow an NFA M to change state without reading input That is, we add the following categories of transitions to  –  (q  is allowed

138. 138 Example * a,b a aba b aab

139. 139 Defining L(M) and LNFA- M accepts string x if one of the configurations reached is an accepting configuration –(q 0, x) |- * (f, ),f e A M rejects string x if all configurations reached are either not halting configurations or are rejecting configurations L(M) or Y(M) N(M) LNFA- –Language L is in language class LNFA- iff

140. 140 LNFA- subset LFSA Recap of what we already know –Let M be any NFA –There exists an algorithm A 1 which constructs an FSA M’ such that L(M’) = L(M) New goal –Let M be any NFA- –There exists an algorithm A 2 which constructs an FSA M’ such that L(M’) = L(M)

141. 141 Visualization Goal –Let M be any NFA- –There exists an algorithm A 2 which constructs an FSA M’ such that L(M’) = L(M) NFA- M FSA M’ A2A2

142. 142 Modified Goal Can we use any existing algorithms to simplify the task of developing algorithm A 2 ? –Yes, we can use algorithm A 1 which converts an NFA M 1 into an FSA M’ such that L(M’) = L(M 1 ) NFA- M FSA M’A2A2 NFA- M FSA M’ Algorithm A 2 A1A1 NFA  M 1 A 2’

143. 143 New Goal Difficulty –NFA- M can make transitions on –How can the NFA M 1 simulate these -transitions? NFA- M NFA M 1 A 2’ a bb 234561

144. 144 Basic Idea For each state q of M and each character  of , figure out which states are reachable from q taking any number of -transitions and exactly one transition on that character . In the NFA-  M 1, directly connect q to each of these states using an arc labeled with . NFA- M NFA M 1 A 2’ a bb 234561 234561 b b b

145. 145 Process State 2 NFA- M NFA-  M 1 A 2’ a bb 234561 234561 b b b b b b

146. 146 Process State 3 NFA- M NFA-  M 1 A 2’ a bb 234561 234561 b b b b b b a a b

147. 147 Final Picture NFA- M NFA-  M 1 A 2’ a bb 234561 234561 b b b b b b a a b b b a a

148. 148 Construction Input NFA- M = (Q, , q 0, , A) Output NFA M 1 = (Q 1,  1, q 1,  1, A 1 ) –What is Q 1 ? Q 1 = Q In this case, Q 1 = {1,2,3,4,5,6} –What is  1 ?  1 =  In this case,  1 =  = {a,b} –What is q 1 ? We always make q 1 = q 0 In this case q 1 = 1 a bb 234561

149. 149 Construction Input NFA- M = (Q, , q 0, , A) Output NFA M 1 = (Q 1,  1, q 1,  1, A 1 ) –What is  1 ?  1 (q,a) = the set of states reachable from state q in M taking any number of -transitions and exactly one transition on the character a –More on this later In this case –  1 (1,a) = {} –  1 (1,b) = {3,4,5} –What is A 1 ? A 1 = A with one minor change –If an accepting state is reachable from q 0 using only  -transitions, then we make q 1 an element of A 1 In this case, using only -transitions, no accepting state is reachable from q 0, so A 1 = A a bb 234561

150. 150 Computing  1 (q,a)  1 (q,a) = the set of states reachable from state q in M taking 0 or more -transitions and exactly one transition on the character a –Break this down into three steps First compute all states reachable from q using 0 or more -transitions –We call this set of states  (q) Next, compute all states reachable from any element of  (q) using the character a –We can denote these states as  (  (q),a) Finally, compute all states reachable from states in  (  (q),a) using 0 or more -transitions –We denote these states as  (  (  (q),a)) –This is the desired answer a bb 234561

151. 151 Example  1 (1,b) = {3,4,5} –Compute  (1), all states reachable from state 1 using 0 or more - transitions  (1) = {1,2} –Compute  (  (1),b), all states reachable from any element  (1) of using the character b:  (  (1),b) =  ({1,2},b) =  (1,b) U  (2,b) = {} U {3} = {3} –Compute  (  (  (1),b)), all states reachable from states in  (  (1),b) using 0 or more -transitions  (  (  (1),b)) =  (3) = {3,4,5} a bb 234561

152. 152 Comments You should be able to execute this algorithm –Convert any NFA- into an equivalent NFA. You should understand the idea behind this algorithm –Why the transition function is computed the way it is –Why A 1 may need to include q 1 in some cases You should understand the importance of this algorithm –Compiler role again –Use in combination with previous algorithm for converting any NFA into an equivalent FSA to create a new algorithm for converting any NFA- into an equivalent FSA

153. 153 LNFA- = LFSA Implications –Lets us use the term LFSA to refer to this language class –Given a language L is in LFSA We know there exists an FSA M s.t. L(M) = L We know there exists an NFA M s.t. L(M) = L –To show a language L is in LFSA Show there exists an FSA M s.t. L(M) = L Show there exists an NFA- M s.t. L(M) = L

154. 154 Module 21 Closure Properties for LFSA using NFA’s –From now on, when I say NFA, I mean any NFA including an NFA- unless I add a specific restriction –union (second proof) –concatenation –Kleene closure

155. 155 LFSA closed under set union (again)

156. 156 LFSA closed under set union Let L 1 and L 2 be arbitrary languages in LFSA Let M 1 and M 2 be NFA’s s.t. L(M 1 ) = L 1, L(M 2 ) = L 2 –M 1 and M 2 exist by definition of L 1 and L 2 in LFSA and the fact that every FSA is an NFA Construct NFA M 3 from NFA’s M 1 and M 2 Argue L(M 3 ) = L 1 union L 2 There exists NFA M 3 s.t. L(M 3 ) = L 1 union L 2 L 1 union L 2 is in LFSA

157. 157 Visualization L 1 union L 2 L1L1 L2L2 LFSA M3M3 M1M1 M2M2 NFA’s Let L 1 and L 2 be arbitrary languages in LFSA Let M 1 and M 2 be NFA’s s.t. L(M 1 ) = L 1, L(M 2 ) = L 2 M 1 and M 2 exist by definition of L 1 and L 2 in LFSA and the fact that every FSA is an NFA Construct NFA M 3 from NFA’s M 1 and M 2 Argue L(M 3 ) = L 1 union L 2 There exists NFA M 3 s.t. L(M 3 ) = L 1 union L 2 L 1 union L 2 is in LFSA

158. 158 Algorithm Specification Input –Two NFA’s M 1 and M 2 Output –NFA M 3 such that L(M 3 ) = ? NFA M 1 NFA M 2 NFA M 3 A

159. 159 Use -transition NFA M 1 NFA M 2 NFA M 3 A a M1M1 a,b M2M2 a M3M3

160. 160 General Case * NFA M 1 NFA M 2 NFA M 3 A M1M1 M2M2 M3M3

161. 161 Construction * Input –NFA M 1 = (Q 1,  , q 1,  , A 1 ) –NFA M 2 = (Q 2,  , q 2,  , A 2 ) Output –NFA M 3 = (Q 3,  , q 3,  , A 3 ) –What is Q 3 ? Q 3 = –What is  3 ?  3 =  1 =  2 –What is q 3 ? q 3 = NFA M 1 NFA M 2 NFA M 3 A

162. 162 Construction Input –NFA M 1 = (Q 1,  , q 1,  , A 1 ) –NFA M 2 = (Q 2,  , q 2,  , A 2 ) Output –NFA M 3 = (Q 3,  , q 3,  , A 3 ) –What is A 3 ? A 3 = –What is  3 ?  3 = NFA M 1 NFA M 2 NFA M 3 A

163. 163 Comments You should be able to execute this algorithm You should understand the idea behind this algorithm You should understand how this algorithm can be used to simplify design You should be able to design new algorithms for new closure properties You should understand how this helps prove result that regular languages and LFSA are identical –In particular, you should understand how this is used to construct an NFA M from a regular expression r s.t. L(M) = L(r) –To be seen later

164. 164 LFSA closed under set concatenation

165. 165 LFSA closed under set concatenation Let L 1 and L 2 be arbitrary languages in LFSA Let M 1 and M 2 be NFA’s s.t. L(M 1 ) = L 1, L(M 2 ) = L 2 –M 1 and M 2 exist by definition of L 1 and L 2 in LFSA and the fact that every FSA is an NFA Construct NFA M 3 from NFA’s M 1 and M 2 Argue L(M 3 ) = L 1 concatenate L 2 There exists NFA M 3 s.t. L(M 3 ) = L 1 concatenate L 2 L 1 concatenate L 2 is in LFSA

166. 166 Visualization L 1 concatenate L 2 L1L1 L2L2 LFSA M3M3 M1M1 M2M2 NFA’s Let L 1 and L 2 be arbitrary languages in LFSA Let M 1 and M 2 be NFA’s s.t. L(M 1 ) = L 1, L(M 2 ) = L 2 –M 1 and M 2 exist by definition of L 1 and L 2 in LFSA and the fact that every FSA is an NFA Construct NFA M 3 from NFA’s M 1 and M 2 Argue L(M 3 ) = L 1 concatenate L 2 There exists NFA M 3 s.t. L(M 3 ) = L 1 concatenate L 2 L 1 concatenate L 2 is in LFSA

167. 167 Algorithm Specification Input –Two NFA’s M 1 and M 2 Output –NFA M 3 such that L(M 3 ) = NFA M 1 NFA M 2 NFA M 3 A

168. 168 Use -transition NFA M 1 NFA M 2 NFA M 3 A a M1M1 a,b M2M2 a M3M3

169. 169 General Case NFA M 1 NFA M 2 NFA M 3 A M1M1 M2M2 M3M3

170. 170 Construction Input –NFA M 1 = (Q 1,  , q 1,  , A 1 ) –NFA M 2 = (Q 2,  , q 2,  , A 2 ) Output –NFA M 3 = (Q 3,  , q 3,  , A 3 ) –What is Q 3 ? Q 3 = –What is  3 ?  3 =  1 =  2 –What is q 3 ? q 3 = NFA M 1 NFA M 2 NFA M 3 A

171. 171 Construction Input –NFA M 1 = (Q 1,  , q 1,  , A 1 ) –NFA M 2 = (Q 2,  , q 2,  , A 2 ) Output –NFA M 3 = (Q 3,  , q 3,  , A 3 ) –What is A 3 ? A 3 = –What is  3 ?  3 = NFA M 1 NFA M 2 NFA M 3 A

172. 172 Comments You should be able to execute this algorithm You should understand the idea behind this algorithm You should understand how this algorithm can be used to simplify design You should be able to design new algorithms for new closure properties You should understand how this helps prove result that regular languages and LFSA are identical –In particular, you should understand how this is used to construct an NFA M from a regular expression r s.t. L(M) = L(r) –To be seen later

173. 173 LFSA closed under Kleene Closure

174. 174 LFSA closed under Kleene Closure Let L be arbitrary language in LFSA Let M 1 be an NFA s.t. L(M 1 ) = L –M 1 exists by definition of L 1 in LFSA and the fact that every FSA is an NFA Construct NFA M 2 from NFA M 1 Argue L(M 2 ) = L 1 * There exists NFA M 2 s.t. L(M 2 ) = L 1 * L 1 * is in LFSA

175. 175 Visualization L1*L1* L1L1 LFSA NFA’s Let L be arbitrary language in LFSA Let M 1 be an NFA s.t. L(M 1 ) = L –M 1 exists by definition of L 1 in LFSA and the fact that every FSA is an NFA Construct NFA M 2 from NFA M 1 Argue L(M 2 ) = L 1 * There exists NFA M 2 s.t. L(M 2 ) = L 1 * L 1 * is in LFSA M1M1 M2M2

176. 176 Algorithm Specification Input –NFA M 1 Output –NFA M 2 such that L(M 2 ) = NFA M 1 NFA M 2 A

177. 177 Use -transition NFA M 1 NFA M 2 A a M1M1 a M2M2

178. 178 General Case * NFA M 1 NFA M 2 A M1M1 M2M2

179. 179 Construction Input –NFA M 1 = (Q 1,  , q 1,  , A 1 ) Output –NFA M 2 = (Q 2,  , q 2,  , A 2 ) –What is Q 2 ? Q 2 = –What is  2 ?  2 =  1 –What is q 2 ? q 2 = NFA M 1 NFA M 2 A

180. 180 Construction Input –NFA M 1 = (Q 1,  , q 1,  , A 1 ) Output –NFA M 2 = (Q 2,  , q 2,  , A 2 ) –What is A 2 ? A 2 = –What is  2 ?  2 = NFA M 1 NFA M 2 A

181. 181 Comments You should be able to execute this algorithm You should understand the idea behind this algorithm –Why do we need to make an extra state p? You should understand how this algorithm can be used to simplify design You should be able to design new algorithms for new closure properties You should understand how this helps prove result that regular languages and LFSA are identical –In particular, you should understand how this is used to construct an NFA M from a regular expression r s.t. L(M) = L(r) –To be seen later

182. 182 Module 22 Regular languages are a subset of LFSA –algorithm for converting any regular expression into an equivalent NFA –Builds on existing algorithms described in previous lectures

183. 183 Regular languages are a subset of LFSA

184. 184 Reg. Lang. subset LFSA Let L be an arbitrary Let R be the –R exists by definition of Construct an –M is constructed from Argue There exists an L is in –By definition of

185. 185 Visualization Regular Languages LFSA Regular Expressions NFA- ’s L L R M

186. 186 Algorithm Specification Input –Regular expression R Output –NFA M such that L(M) = Regular expression R NFA- M A

187. 187 Recursive Algorithm We have an inductive definition for regular languages and regular expressions Our algorithm for converting any regular expression into an equivalent NFA is recursive in nature –Base Case –Recursive or inductive Case

188. 188 Base Case Regular expression R has zero operators –No concatenation, union, Kleene closure –For any alphabet , only |  | + 2 regular languages can be depicted by any regular expression with zero operators The empty language  The language { } The |  | languages consisting of one string {a} for all a in 

189. 189 Table lookup Finite number of base cases means we can use table lookup to handle them a b 

190. 190 Recursive Case Regular expression R has at least one operator –This means R is built up from smaller regular expressions using the union, Kleene closure, or concatenation operators –More specifically, there are 3 cases: R = R 1 +R 2 R = R 1 R 2 R = R 1 *

191. 191 Recursive Calls The algorithm recursively calls itself to generate NFA’s M 1 and M 2 which accept L(R 1 ) and L(R 2 ) The algorithm applies the appropriate construction –union –concatenation –Kleene closure to NFA’s M 1 and M 2 to produce an NFA M such that L(M) = L(R) 1) R = R 1 + R 2 2) R = R 1 R 2 3) R = R 1 *

192. 192 Pseudocode Algorithm _____________ RegExptoNFA(_____________) { regular expression R 1, R 2 ; NFA M 1, M 2 ; Modify R by removing unnecessary enclosing parentheses /* Base Case */ If R = a, return (NFA for {a}) /* include here */ If R = , return (NFA for {}) /* Recursive Case */ Find “last operator O” of regular expression R Identify regular expressions R 1 (and R 2 if necessary) M 1 = RegExptoNFA(R 1 ) M 2 = RegExptoNFA(R 2 ) /* if necessary */ return (OP(M1, M2)) /* OP is chosen based on O */ }

193. 193 Example A: R = (b+a)a * Last operator is concatenation R 1 = (b+a) R 2 = a* Recursive call with R 1 = (b+a) B: R = (b+a) Extra parentheses stripped away Last operator is union R 1 = b R 2 = a Recursive call with R 1 = b

194. 194 Example Continued C: R = b Base case NFA for {b} returned B: return to this invocation of procedure Recursive call where R = R 2 = a D: R = a Base case NFA for {a} returned B: return to this invocation of procedure return UNION(NFA for {b}, NFA for {a}) A: return to this invocation of procedure Recursive call where R = R 2 = a*

195. 195 Example Finished E: R = a* Last operator is Kleene closure R 1 = a Recursive call where R = R 1 = a F: R = a Base case NFA for {a} returned E: return to this invocation of procedure return (KLEENE(NFA for {a})) A: return to this invocation of procedure return CONCAT(NFA for {b,a}, NFA for {a}*)

196. 196 concatenate Pictoral View (b|a)a* (b|a)a* union baba b a Kleene Closure aa a a b a

197. 197 Parse Tree We now present the “parse” tree for regular expression (b+a)a* concatenate unionKleene closure baa

198. 198 Module 23 Regular languages review –Several ways to define regular languages –Two main types of proofs/algorithms Relative power of two computational models proofs/constructions Closure property proofs/constructions –Language class hierarchy Applications of regular languages

199. 199 Defining regular languages

200. 200 Three definitions LFSA –A language L is in LFSA iff there exists an FSA M s.t. L(M) = L LNFA –A language L is in LNFA iff there exists an NFA M s.t. L(M) = L Regular languages –A language L is regular iff there exists a regular expression R s.t. L(R) = L Conclusion –All these language classes are equivalent –Any language which can be represented using any one of these models can be represented using either of the other two models

201. 201 Two types of proofs/constructions

202. 202 Relative power proofs These proofs work between two language classes and two computational models The crux of these proofs are algorithms which behave as follows: –Input: One program from the first computational model –Output: A program from the second computational model that is equivalent in function to the first program

203. 203 Closure property proofs These proofs work within a single language class and typically within a single computational model The crux of these proofs are algorithms which behave as follows: –Input: 1 or 2 programs from a given computational model –Output: A third program from the same computational model that accepts/describes a third language which is a combination of the languages accepted/described by the two input programs

204. 204 Comparison L 1 intersect L 2 L1L1 L2L2 LFSA M3M3 M1M1 M2M2 FSA’s LNFA LFSA NFA’s FSA’s L L M M’

205. 205 Language class hierarchy All languages over alphabet  RE REC regular H H ?

206. 206 Three remaining topics Myhill-Nerode Theorem –Provides technique for proving a language is not regular –Also represents fundamental understanding of what a regular language is Decision problems about regular languages –Most are solvable in contrast to problems about recursive languages Pumping lemma –Provides technique for proving a language is not regular

207. 207 Module 24 Myhill-Nerode Theorem –distinguishability –equivalence classes of strings –designing FSA’s –proving a language L is not regular

208. 208 Distinguishability

209. 209 Distinguishable and Indistinguishable String x is distinguishable from string y with respect to language L iff –there exists a string z such that xz is in L and yz is not in L OR xz is not in L and yz is in L String x is indistinguishable from string y with respect to language L iff –for all strings z, xz and yz are both in L OR xz and yz are both not in L

210. 210 Example Let EVEN-ODD be the set of strings over {a,b} with an even number of a’s and an odd number of b’s –Is the string aa distinguishable from the string bb with respect to EVEN-ODD? –Is the string aa distinguishable from the string ab with respect to EVEN-ODD?

211. 211 Equivalence classes of strings

212. 212 Definition of equivalence classes Every language L partitions  * into equivalence classes via indistinguishability –Two strings x and y belong to the same equivalence class defined by L iff x and y are indistinguishable w.r.t L –Two strings x and y belong to different equivalence classes defined by L iff x and y are distinguishable w.r.t. L

213. 213 Example How does EVEN-ODD partition {a,b}* into equivalence classes? Strings with an EVEN number of a’s and an EVEN number of b’s Strings with an EVEN number of a’s and an ODD number of b’s Strings with an ODD number of a’s and an EVEN number of b’s Strings with an ODD number of a’s and an ODD number of b’s

214. 214 Second Example Let 1MOD3 be the set of strings over {a,b} whose length mod 3 = 1. How does 1MOD3 partition {a,b}* into equivalence classes? Length mod 3 = 0 Length mod 3 = 1 Length mod 3 = 2

215. 215 Designing FSA’s

216. 216 Designing an FSA for EVEN-ODD Even Odd Even b ab a a b b a

217. 217 Designing an FSA for 1MOD3 Length mod 3 = 0 Length mod 3 = 1 Length mod 3 = 2 a aa a,b

218. 218 Proving a language is not regular

219. 219 Third Example Let EQUAL be the set of strings x over {a,b} s.t. the number of a’s in x = the number of b’s in x How does EQUAL partition {a,b}* into equivalence classes? How many equivalence classes are there? Can we construct a finite state automaton for EQUAL?

220. 220 Myhill-Nerode Theorem

221. 221 Theorem Statement Two part statement –If L is regular, then L partitions  * into a finite number of equivalence classes –If L partitions  * into a finite number of equivalence classes, then L is regular One part statement –L is regular iff L partitions  * into a finite number of equivalence classes

222. 222 Implication 1 Method for constructing FSA’s to accept a language L –Identify equivalence classes defined by L –Make a state for each equivalence class –Identify initial and accepting states –Add transitions between the states You can use a canonical element of each equivalence class to help with building the transition function 

223. 223 Implication 2 Method for proving a language L is not regular –Identify equivalence classes defined by L –Show there are an infinite number of such equivalence classes Table format may help, but it is only a way to help illustrate that there are an infinite number of equivalence classes defined by L

224. 224 Proving a language is not regular revisited

225. 225 Proving EQUAL is not regular Let EQUAL be the set of strings x over {a,b} s.t. the number of a’s in x = the number of b’s in x We want to show that EQUAL partitions {a,b}* into an infinite number of equivalence classes We will use a table that is somewhat reminiscent of the table used for diagonalization –Again, you must be able to identify the infinite number of equivalence classes being defined by the table. They ultimately represent the proof that EQUAL or whatever language you are working with is not regular.

226. 226 Table * a aa aaa aaaa aaaaa... b IN OUT... bb OUT IN OUT... bbb OUT OUT 1 IN OUT... bbbb OUT IN OUT... bbbbb OUT IN... ……………………………… The strings being distinguished are the rows. The tables entries indicate that the concatenation of the row string with the column string is in or not in EQUAL. Each complete column shows one row string is distinguishable from all the other row strings.

227. 227 Concluding EQUAL is nonregular * We have shown that EQUAL partitions {a,b}* into an infinite number of equivalence classes –In this case, we only identified some of the equivalence classes defined by EQUAL, but that is sufficient Thus, the Myhill-Nerode Theorem implies that EQUAL is nonregular

228. 228 Summary Myhill-Nerode Theorem and what it says –It does not say a language L is regular iff L is finite Many regular languages such as  * are not finite –It says that a language L is regular iff L partitions  * into a finite number of equivalence classes Provides method for designing FSA’s Provides method for proving a language L is not regular –Show that L partitions  * into an infinite number of equivalence classes

229. 229 Three Types of Problems Create a table that helps prove that a specific language L is not regular –You get to choose the “row” and “column” strings –I choose the “row” strings Identify the equivalence classes defined by L as highlighted by a given table

230. 230 Module 25 Decision problems about regular languages –Basic problems are solvable halting, accepting, and emptiness problems –Solvability of other problems answer-preserving input transformations to basic problems

231. 231 Programs In this unit, our programs are the following three types of objects –FSA’s –NFA’s –regular expressions Previously, they were C++ programs –Review those topics after mastering today’s examples

232. 232 Basic Decision Problems (and algorithms for solving them)

233. 233 Halting Problem Input –FSA M –Input string x to M Question –Does M halt on x? Give an algorithm for solving the FSA halting problem.

234. 234 Accepting Problem Input –FSA M –Input string x to M Question –Is x in L(M)? Give an algorithm ACCEPT for solving the accepting problem.

235. 235 Empty Language Problem Input –FSA M Question –Is L(M)={}? Give an algorithm for solving the empty language problem. –Don’t look ahead to the next slide.

236. 236 Algorithms for solving empty language problem Algorithm 1 –View FSA M as a directed graph (nodes, arcs) –See if any accepting node is reachable from the start node Algorithm 2 –Let n be the number of states in FSA M –Run ACCEPT(M,x) for all input strings of length < n –If any are accepted THEN no ELSE yes Why is algorithm 2 correct?

237. 237 Solving Other Problems (using answer-preserving input transformations)

238. 238 Complement Empty Problem Input –FSA M Question –Is (L(M)) c = {}? Show how to use an answer-preserving input transformation to help solve this problem –Show that the Complement Empty problem transforms to the Empty Language problem –Don’t look at next two slides

239. 239 Algorithm Description Convert input FSA M into an FSA M’ such that L(M’) = (L(M)) c –We do this by applying the algorithm which we used to show that LFSA is closed under complement Feed FSA M’ into algorithm which solves the empty language problem If that algorithm returns yes THEN yes ELSE no

240. 240 Input Transformation Illustrated Algorithm for solving empty language problem FSA M Complement Construction FSA M’ Yes/No Algorithm for complement empty problem The complement construction algorithm is the answer-pres. input transformation. If M is a yes input instance of CE, then M’ is a yes input instance of EL. If M is a no input instance of CE, then M’ is a no input instance of EL.

241. 241 NFA Empty Problem Input –NFA M Question –Is L(M)={}? Show how to use answer-preserving input transformations to help solve this problem –Show that the NFA Empty problem transforms to the Empty Language problem

242. 242 Input Transformation Yes/No Algorithm for NFA empty problem

243. 243 Equal Problem Input –FSA’s M 1 and M 2 Question –Is L(M 1 ) = L(M 2 )? Show how to use answer-preserving input transformations to solve this problem –Try and transform this problem to the empty language problem –If L(M 1 ) = L(M 2 ), then what combination of L(M 1 ) and L(M 2 ) must be empty?

244. 244 Input Transformation Illustrated Yes/No Algorithm for Equal problem

245. 245 Summary Decision problems with programs as inputs Basic problems –You need to develop algorithms from scratch based on properties of FSA’s Solving new problems –You need to figure out how to combine the various algorithms we have seen in this unit to solve the given problem