Regular Languages Chapter 1 Giorgi Japaridze Theory of Computability

How a finite automaton works 1.1.a Giorgi Japaridze Theory of Computability q2q2 q0q0 q1q

The language of a machine 1.1.b Giorgi Japaridze Theory of Computability q2q2 q0q0 q1q L(M), “the language of M”, or “the language recognized by M” --- the set of all strings that the machine M accepts What is the language recognized by our automaton A?

Formal definition of a finite automaton 1.1.c Giorgi Japaridze Theory of Computability A (deterministic) finite automaton (DFA) is a 5-tuple (Q, , , s, F), where: Q is a finite set whose elements are called the states  is a finite set called the alphabet  is a function of the type Q   Q called the transition function s is an element of Q called the start state F is a subset of Q called the set of accept states

Our automaton formalized 1.1.d Giorgi Japaridze Theory of Computability q2q2 q0q0 q1q Q:::s:F:Q:::s:F: 0 1 q 0 q 1 q 2 A = (Q, , , s, F)

Formal definition of computation 1.1.e Giorgi Japaridze Theory of Computability M = (Q, , , s, F) M accepts the string u 1 u 2 … u n iff there is a sequence r 1, r 2, …, r n, r n+1 of states such that: r 1 =s r i+1 =  (r i,u i ), for each i with 1  i  n r n+1  F q2q2 q0q0 q1q q 0 q 2 q 0 q 2 q1q1 u 1 u 2 … u n r 1, r 2, …, r n, r n+1 0

Designing finite automata 1.1.f Giorgi Japaridze Theory of Computability Task: Design an automaton that accepts a bit string iff it contains an even number of “1”s.

Designing finite automata 1.1.g Task: Design an automaton that accepts a bit string iff the number of “1”s that it contains is divisible by 3.

Designing finite automata 1.1.h Task: Let L2={w | w is a string of 0s whose length is divisible by 2} and L3={w | w is a string of 0s whose length is divisible by 3} Design an automaton that recognizes L2  L3

Designing finite automata 1.1.i Task: Let L2={w | w is a string of 0s whose length is divisible by 2} and L3={w | w is a string of 0s whose length is divisible by 3} Design an automaton that recognizes L2  L3

Designing finite automata 1.1.j Task: Design an automaton that recognizes the language X={w | w is a string of 0s whose length is divisible neither by 2 nor by 3} Definition: Let L be a language over an alphabet . The complement of L is the language {w | w is a string over  such that w  L}. X is the complement of what language?

NFAs (Nondeterministic Finite Automata) 1.2.a Giorgi Japaridze Theory of Computability q1q1 q2q2 q3q3 0, q1q1 q 1 q 2 q 1 q 3 q 1 q 2 q1q1 q 1 q

NFAs (Nondeterministic Finite Automata) 1.2.a Giorgi Japaridze Theory of Computability q1q1 q2q2 q3q3 0,1 1 What language does this NFA recognize?

Formal definition of a nondeterministic finite automaton 1.2.b Giorgi Japaridze Theory of Computability An NFA is a 5-tuple (Q, , , s, F), where: Q is a finite set whose elements are called the states  is a finite set called the alphabet  is a function of the type Q   P (Q) called the transition function s is an element of Q called the start state F is a subset of Q called the set of accept states

Example 1.2.c Giorgi Japaridze Theory of Computability Q:::s:F:Q:::s:F: a b A = (Q, , , s, F) 32 1 a a b a,b b

Formal definition of accepting 1.2.d Giorgi Japaridze Theory of Computability M = (Q, , , s, F) M accepts the string u 1 u 2 … u n iff there is a sequence r 1, r 2, …, r n, r n+1 of states such that: r 1 =s r i+1 =  (r i,u i ), for each i with 1  i  n r n+1  F When M is a DFA M accepts the string u 1 u 2 … u n iff there is a sequence r 1, r 2, …, r n, r n+1 of states such that: r 1 =s r i+1   (r i,u i ), for each i with 1  i  n r n+1  F When M is an NFA

What language does this NFA recognize? 1.2.e Giorgi Japaridze Theory of Computability

What language does this DFA recognize? 1.2.f Giorgi Japaridze Theory of Computability

Equivalence of NFAs and DFAs 1.2.g Giorgi Japaridze Theory of Computability Two machines are said to be equivalent if they recognize the same language. Theorem 1.39 Every NFA has an equivalent DFA. Proof. Consider an NFA N = (Q, , , s, F ) We need construct an equivalent DFA D = (Q’, ,  ’, s’, F’) using a procedure called the subset construction described on the next slide.

The subset construction 1.2.h Giorgi Japaridze Theory of Computability Constructing DFA D = (Q’, ,  ’, s’, F’) from NFA N = (Q, , , s, F) Q’ = P (Q)  ’(R,a) = {q | q=  (p,a) for some p  R} s’ = {s} F’= {R | R is a subset of Q containing an accept state of N} D obviously works correctly: at every step in the computation, it clearly enters a state that corresponds to the subset of states that N could be in at that point.

Example of applying the subset construction 1.2.i Giorgi Japaridze Theory of Computability Q’::’:s’:F’:Q’::’:s’:F’: a b  {1} {2} {3} {1,2} {1,3} {2,3} {1,2,3} N = (Q, , , s, F) 32 1 a a b a,b b Q’ = P (Q)  ’(R,a) = {q | q=  (p,a) for some p  R} s’ = {s} F’= {R | R is a subset of Q containing an accept state of N}

The resulting DFA 1.2.j Giorgi Japaridze Theory of Computability {3} {1} {1,3}  {2,3} {1,2}{1,2,3} {2} b a,b a a b a b a b b a b a D

Removing unreachable states 1.2.k Giorgi Japaridze Theory of Computability {3} {1}  {2,3} {1,2,3} b a,b a a b b a b a D

Testing in work 1.2.l Giorgi Japaridze Theory of Computability {3} {1}  {2,3} {1,2,3} b a,b a a b b a b a D 32 1 a a b b N b a a

Regular operations 1.3.a Giorgi Japaridze Theory of Computability Union: L1  L2 = {x | x  L1 or x  L2} {Good,Bad}  {Boy,Girl} = {0,00,000,…}  {1,11,111,…} = L  = Concatenation: L1  L2 = {xy | x  L1 and y  L2} {Good,Bad}  {Boy,Girl} = {0,00,000,…}  {1,11,111,…} = L   = Star: L * = {x 1 …x k | k  0 and each x i is in L} {Boy,Girl} * = {0,00,000,…} * =  * =

Regular expressions 1.3.b Giorgi Japaridze Theory of Computability We say that R is a regular expression (RE) iff R is one of the following: 1. a, where a is a symbol of the alphabet 2.  3.  4. (R1)  (R2), where R1 and R2 are RE 5. (R1)  (R2), where R1 and R2 are RE 6. (R1)*, where R1 is a RE What language is represented by the expression: {a} {  }  The union of the languages represented by R1 and R2 The concatenation of the languages represented by R1 and R2 The star of the language represented by R1 Conventions:  The symbol  is often omitted in RE  Some parentheses can be omitted. The precedence order for the operators is: * (highest),  (medium),  (lowest)

Regular languages 1.3.c Giorgi Japaridze Theory of Computability A language is said to be regular iff it can be represented by a regular expression. Language Expression {11} {Boy, Girl, Good, Bad} { ,0,00,000,0000,…} {0,00,000,0000,…} { ,01,0101,010101, ,…} {x | x = 0 k where k is a multiple of 2 or 3} {x | x is divisible by 8} {x | x MOD 4 = 3}

Exercising reading regular expressions 1.3.d Giorgi Japaridze Theory of Computability Expression Language 0*10* (Good  Bad)(Boy  Girl) (Tom  Bob)_is_(good  bad) {Name_is_adjective | Name is an uppercase letter followed by zero or more lowercase letters, and adjective is a lowercase letter followed by zero or more lowercase letters} (0  1)*101(0  1)* ( (0  1)(0  1) ) *

Regular languages and DFA-recognizable languages are the same 1.3.e Theorem 1.54* A language is regular if and only if some NFA (DFA) recognizes it. In other words, a) [The “only if” part] For every regular expression there is an NFA that recognizes exactly the language represented by that expression. b) [The “if” part] For every NFA there is a regular expression that represents exactly the language recognized by that NFA.

Constructing an NFA from a regular expression: Base cases 1.3.f Case of a, where a is a symbol of the alphabet. Case of  Case of 

Constructing an NFA from a regular expression: Case of union 1.3.g Case of (R1)  (R2), where R1 and R2 are RE First, construct NFAs N1 and N2 from R1 and R2: s1 N1 N2 s2 Then, combine them in the following way: s1 N1 N2 s2

Constructing an NFA from a regular expression: Case of concatenation 1.3.h Case of (R1)  (R2), where R1 and R2 are RE First, construct NFAs N1 and N2 from R1 and R2: N1 s2 N2 Then, combine them in the following way: N1 s2 N2 s1

Constructing an NFA from a regular expression: Case of star 1.3.i Case of (R1) *, where R1 is a RE First, construct an NFA N1 from R1: s1 N1 Then, extend it in the following way: s1 N1

Constructing an NFA from a regular expression: An example 1.3.j #(0  1)* (0  1)* # 0       0 1 # #011

GNFA 1.3.k great (great)* grand mother  father grand  g r e a t g r e a t g r e a t g r a n d f a t h e r  

About  -transitions 1.3.l great (great)* grand mother  father grand  Adding or removing  -transitions does not change the recognized language 

The same GNFA simplified 1.3.m great grand mother  father  

Ripping a state out 1.3.n mother  father grand (great)* 

Eliminating parallel transitions 1.3.o mother  father   (great)*grand

Again ripping out 1.3.p (   (great)*grand) (mother  father)

How, exactly, to do ripping out 1.3.q1 Assume, we are ripping out the state r from a GNFA that has no parallel transitions. Let L be the label of the loop from r to r (if there is no loop, then L=  ). L T R S

How, exactly, to do ripping out 1.3.q2 Assume, we are ripping out the state r from a GNFA that has no parallel transitions. Let L be the label of the loop from r to r (if there is no loop, then L=  ). 1. For every pair s 1,s 2 of states such that there is an E 1 -labeled transition from s 1 to r and an E 2 -labeled transition from r to s 2, add an R 1 L*R 2 -labeled transition from s 1 to s 2 ; L T R S RL*T SL*T

How, exactly, to do ripping out 1.3.q3 Assume, we are ripping out the state r from a GNFA that has no parallel transitions. Let L be the label of the loop from r to r (if there is no loop, then L=  ). 1. For every pair s 1,s 2 of states such that there is an E 1 -labeled transition from s 1 to r and an E 2 -labeled transition from r to s 2, add an R 1 L*R 2 -labeled transition from s 1 to s 2 ; 2. Delete r together with all its incoming and outgoing transitions. RL*T SL*T

How, exactly, to eliminate parallel transitions 1.3.r Whenever you see parallel transitions labeled with R1 and R2, Replace them by a transition labeled with R1  R2. R1 R2 R1  R2 Repeat until there are no parallel transitions remaining.

From NFA to RE 1.3.s a b b b

From NFA to RE: Step t Step 1: If there are incoming arrows to the start state, or the start state is an accept state, then add a new start state and connect it with an  -arrow to the old start state. a b b b  a

From NFA to RE: Step u a b b b  Step 2: If there are more than one, or no, accept states, or there is an accept state that has outgoing arrows, then add a new accept state, make all the old accept states non-accept states and connect each of them with an  -arrow to the new accept state.   a

From NFA to RE: Step v a b b b    Step 3: Eliminate all parallel transitions. a

From NFA to RE: Step w1 b b   Step 4: While there are internal states (states that are neither the start nor the accept state), do the following: Step 4.1: Select an internal state and rip it out; Step 4.2: Eliminate all parallel transitions. a b aa ab

From NFA to RE: Step w2 b b  aa   Step 4: While there are internal states (states that are neither the start nor the accept state), do the following: Step 4.1: Select an internal state and rip it out; Step 4.2: Eliminate all parallel transitions. a b ab

From NFA to RE: Step w3  Step 4: While there are internal states (states that are neither the start nor the accept state), do the following: Step 4.1: Select an internal state and rip it out; Step 4.2: Eliminate all parallel transitions. b a(b  aa)* b(b  aa)* b(b  aa)*ab a(b  aa)*ab

From NFA to RE: Step w4 Step 4: While there are internal states (states that are neither the start nor the accept state), do the following: Step 4.1: Select an internal state and rip it out; Step 4.2: Eliminate all parallel transitions. a(b  aa)*   b(b  aa)* b(b  aa)*ab b  a(b  aa)*ab

From NFA to RE: Step w5 Step 4: While there are internal states (states that are neither the start nor the accept state), do the following: Step 4.1: Select an internal state and rip it out; Step 4.2: Eliminate all parallel transitions. a(b  aa)* ( b  a(b  aa)*ab ) ( b(b  aa)*ab ) * (   b(b  aa)* )

From NFA to RE: Step w6 Step 4: While there are internal states (states that are neither the start nor the accept state), do the following: Step 4.1: Select an internal state and rip it out; Step 4.2: Eliminate all parallel transitions. ( ( b  a(b  aa)*ab ) ( b(b  aa)*ab ) * (   b(b  aa)* )  ( a(b  aa)* )

From NFA to RE: Step x Step 5: Return the label of the only remaining arrow (if there is no arrow, return  ). Claim: The resulting RE represents exactly the language recognized by the original NFA. This completes the proof of Theorem ( ( b  a(b  aa)*ab ) ( b(b  aa)*ab ) * (   b(b  aa)* )  ( a(b  aa)* )

Pumping Lemma 1.4.a Pumping Lemma (preliminary version): Suppose L is the language recognized by a DFA with p states, and w is a string in L of length at least p. Then w may be divided into three parts, w = xyz, so that: 1. For each i  0, xy i z  L; 2. |y| > 0; 3. |xy|  p. Example: Suppose a DFA with 6 states accepts Villanova. Then Villanova = xyz, e.g., with x = Vi, y = lla, z = nova, V i l l a n o v a so that the DFA also accepts Vinova (xy 0 z) Villanova (xy 1 z) Villallanova (xy 2 z) Villallallanova (xy 3 z) Villallallallanova (xy 4 z) … We also have: |lla| > 0, |Villa|  6.

Proof idea 1.4.b V i l l n o v V i l l a n o v a a

An equivalent formulation of the pumping lemma 1.4.c Pumping Lemma (Theorem 1.70): Suppose L is a regular language. Then there is a number p, called the pumping length, such that any string w from L of length at least p can be divided into three parts, w = xyz, so that: 1. For each i  0, xy i z  L; 2. |y| > 0; 3. |xy|  p. Proof: Suppose L is a regular language. Then we know that there must be a DFA recognizing L. Take the pumping length p to be the number of states of this DFA. The rest of the theorem is then a repetition of the previous formulation of the pumping lemma.

Using the pumping lemma for proving nonregularity: Example d Example 1.73: Show the nonregularity of B = {0 n 1 n | n  0} Proof by contradiction: Assume B is regular. Let then p be its pumping length. Select w  B with |w|  p. By the pumping lemma, w=xyz and y can be pumped, so that we must also have xyyz  B. Case 1: y only has 0s. But then xyyz has more 0s than 1s and  B. Case 2: y only has 1s. But then xyyz has more 1s than 0s and  B. Case 3: y has both 0s and 1s. But then xyyz has a 1 followed by a 0 and  B. The assumption that B is regular took us to a contradiction. Hence the assumption was wrong, i.e. B is not regular.

Using the pumping lemma for proving nonregularity: Example e Example 1.75: Show the nonregularity of F = { ww | w  {0,1}* } Proof by contradiction: Assume F is regular. Let then p be its pumping length. Observe that 0 p 10 p 1  F. By the pumping lemma, 0 p 10 p 1 =xyz and y can be pumped. By Condition 3, |xy|  p. Therefore, y is entirely in the first 0 p. Pumping y would produce a string that has only 0s is the first half, and two 1s in the second half. Obviously this string cannot be in F, which contradicts with the pumping lemma. The assumption that F is regular took us to a contradiction. Hence the assumption was wrong, i.e. F is not regular.