CS 154, Lecture 4: Limitations on DFAs (I), Pumping Lemma, Minimizing DFAs

Non-Regular Languages Regular or Not? D = { w | w has equal number of occurrences of 01 and 10 } REGULAR! C = { w | w has equal number of 1s and 0s} NOT REGULAR! How can we prove that there is no DFA for a particular language? Surprising Algorithms (even in restricted models) are routinely being discovered

The Pumping Lemma: Structure in Regular Languages Let L be a regular language Then there is a positive integer P s.t. for all strings w  L with |w| ≥ P there is a way to write w = xyz, where: 1. |y| > 0 (that is, y  ε) |xy| ≤ P For all i ≥ 0, xyiz  L Why is it called the pumping lemma? The word w gets pumped into longer and longer strings…

Let w be a string where w  L and |w| ≥ P Proof: Let M be a DFA that recognizes L Let P be the number of states in M Let w be a string where w  L and |w| ≥ P We show: w = xyz |y| > 0 |xy| ≤ P xyiz  L for all i ≥ 0 y x w z … q0 qj qk q|w| Claim: There must exist j and k such that 0 ≤ j < k ≤ P, and qj = qk

Applying the Pumping Lemma Let’s prove that EQ={ w | #1s = #0s} is not regular. By contradiction. Assume EQ is regular. Let P be as in pumping lemma. Let w = 0P1P∈ EQ.  Can write w = xyz, with |y| > 0, |xy| ≤ P, such that for all i ≥ 0, xyiz is also in EQ Claim: The string y must be all zeroes. Why? Because |xy| ≤ P and w = xyz = 0P1P But then xyyz has more 0s than 1s Contradiction!

Applying the Pumping Lemma Prove: SQ = {0n2 | n ≥ 0} is not regular Assume SQ is regular. Let w = 0P2  Can write w = xyz, with |y| > 0, |xy| ≤ P, such that for all i ≥ 0, xyiz is also in EQ So xyyz ∊ SQ. Note that xyyz = 0P2+|y| Note that 0 < |y| ≤ P So |xyyz| = P2 + |y| ≤ P2 + P < P2 + 2P + 1 = (P+1)2 and P2 < |xyyz| < (P+1)2 Therefore |xyyz| is not a perfect square! Hence 0P2+|y| = xyyz ∉ SQ, so our assumption must be false.  SQ is not regular!

Does this DFA have a minimal number of states?

Is this minimal? How can we tell in general?

Theorem: For every regular language L, there is a unique (up to re-labeling of the states) minimal-state DFA M* such that L = L(M*). Furthermore, there is an efficient algorithm which, given any DFA M, will output this unique M*. If these were true for more general models of computation, that would be an engineering breakthrough

Note: There isn’t a uniquely minimal NFA

Extending transition function  to strings Given DFA M = (Q, Σ, , q0, F), we extend  to a function  : Q  Σ* → Q as follows: (q, ε) = q (q, ) = (q, ) (q, 1 …k+1 ) = ( (q, 1 …k ), k+1 ) (q, w) = the state of M reached after reading in w, starting from state q Note: (q0, w)  F  M accepts w Def. w  Σ* distinguishes states q1 and q2 if exactly one of (q1, w), (q2, w) is a final state Def. w  Σ* distinguishes states q1 and q2 if (q1, w)  F  (q2, w)  F

Distinguishing two states Def. w  Σ* distinguishes states q1 and q2 if exactly one of (q1, w), (q2, w) is a final state I’m in q1 or q2, but which? How can I tell? Ok, I’m accepting! Must have been q1 Here… read this W

Fix M = (Q, Σ, , q0, F) and let p, q  Q Definitions: State p is distinguishable from state q if there is w  Σ* that distinguishes p and q ( there is w  Σ* so that exactly one of (p, w), (q, w) is a final state) State p is indistinguishable from state q if p is not distinguishable from q ( for all w  Σ*, (p, w)  F  (q, w)  F) Pairs of indistinguishable states are redundant…

ε distinguishes all final states from non-final states Which pairs of states are distinguishable here? ε distinguishes all final states from non-final states

The string 10 distinguishes q0 and q3 Which pairs of states are distinguishable here? The string 10 distinguishes q0 and q3

The string 0 distinguishes q1 and q2 Which pairs of states are distinguishable here? The string 0 distinguishes q1 and q2

Fix M = (Q, Σ, , q0, F) and let p, q, r  Q Define a binary relation ∼ on the states of M: p ∼ q iff p is indistinguishable from q p ≁ q iff p is distinguishable from q Proposition: ∼ is an equivalence relation p ∼ p (reflexive) p ∼ q  q ∼ p (symmetric) p ∼ q and q ∼ r  p ∼ r (transitive) Proof?

Fix M = (Q, Σ, , q0, F) and let p, q, r  Q Therefore, the relation ∼ partitions Q into disjoint equivalence classes Proposition: ∼ is an equivalence relation [q] := { p | p ∼ q }

Algorithm: MINIMIZE-DFA Input: DFA M Output: DFA MMIN such that: L(M) = L(MMIN) MMIN has no inaccessible states MMIN is irreducible || For all states p  q of MMIN, p and q are distinguishable Theorem: MMIN is the unique minimal DFA that is equivalent to M

The states of MMIN will be the equivalence classes of states of M Intuition: The states of MMIN will be the equivalence classes of states of M We’ll uncover these equivalent states with a dynamic programming algorithm

The Table-Filling Algorithm Input: DFA M = (Q, Σ, , q0, F) Output: (1) DM = { (p, q) | p, q  Q and p ≁ q } (2) EQUIVM = { [q] | q  Q } We know how to find those pairs of states that the string ε distinguishes… Use this and iteration to find those pairs distinguishable with longer strings The pairs of states left over will be indistinguishable High-Level Idea:

The Table-Filling Algorithm Input: DFA M = (Q, Σ, , q0, F) Output: (1) DM = { (p, q) | p, q  Q and p ≁ q } (2) EQUIVM = { [q] | q  Q } Base Case: For all (p, q) such that p accepts and q rejects  p ≁ q

The Table-Filling Algorithm Input: DFA M = (Q, Σ, , q0, F) Output: (1) DM = { (p, q) | p, q  Q and p ≁ q } (2) EQUIVM = { [q] | q  Q } Base Case: For all (p, q) such that p accepts and q rejects  p ≁ q Iterate: If there are states p, q and symbol σ  Σ satisfying: D D  (p, ) = p mark p ≁ q ≁   (q, ) = q D Repeat until no more D’s can be added

D D D D D D

D D D D

Suppose (p, q) is marked D at a later point. Claim: If (p, q) is marked D by the Table-Filling algorithm, then p ≁ q Proof: By induction on the number of steps in the algorithm before (p,q) is marked D If (p, q) is marked D at the start, then one state’s in F and the other isn’t, so ε distinguishes p and q Suppose (p, q) is marked D at a later point. Then there are states p, q such that: 1. (p, q) are marked D  p ≁ q (by induction) So there’s a string w s.t. (p, w)  F  (q, w)  F 2. p = (p,) and q = (q,), where   Σ The string w distinguishes p and q!

Proof (by contradiction): Claim: If (p, q) is not marked D by the Table-Filling algorithm, then p ∼ q Proof (by contradiction): Suppose the pair (p, q) is not marked D by the algorithm, yet p ≁ q (call this a “bad pair”) Then there is a string w such that |w| > 0 and: (p, w)  F and (q, w)  F (Why is |w| > 0?) We have w = w, for some string w’ and some   Σ Of all such bad pairs, let p, q be a pair with the shortest distinguishing string w Let p = (p,) and q = (q,) Then (p, q) is also a bad pair, but with a SHORTER distinguishing string, w !

Output: Equivalent minimal-state DFA MMIN Algorithm MINIMIZE Input: DFA M Output: Equivalent minimal-state DFA MMIN 1. Remove all inaccessible states from M 2. Run Table-Filling algorithm on M to get: EQUIVM = { [q] | q is an accessible state of M } 3. Define: MMIN = (QMIN, Σ, MIN, q0 MIN, FMIN) QMIN = EQUIVM, q0 MIN = [q0], FMIN = { [q] | q  F } MIN( [q],  ) = [ ( q,  ) ] Claim: L(MMIN) = L(M)

D D D D D D D D D

D D D D D D D D D

Thm: MMIN is the unique minimal DFA equivalent to M Claim: If L(M)=L(MMIN) and M has no inaccessible states and M’ is irreducible  there is an isomorphism between M and MMIN Suppose for now the Claim is true. If M’ is a minimal DFA, then M’ has no inaccessible states and is irreducible (why?) So the Claim implies: Let M’ be a minimal DFA for M.  there is an isomorphism between M’ and the DFA MMIN that is output by MINIMIZE(M).  The Thm holds!

Thm: MMIN is the unique minimal DFA equivalent to M Claim: If L(M)=L(MMIN) and M has no inaccessible states and M’ is irreducible  there is an isomorphism between M and MMIN Proof: We recursively construct a map from the states of MMIN to the states of M Base Case: q0 MIN  q0 Recursive Step: If p  p   Then q  q q q

q q Base Case: q0 MIN  q0 Recursive Step: If p  p  Then q  q

q q Base Case: q0 MIN  q0 Recursive Step: If p  p  Then q  q We need to prove: The map is defined everywhere The map is well defined The map is a bijection The map preserves all transitions: If p  p then MIN(p, )  ’(p’, ) (this follows from the definition of the map!)

The map is defined everywhere q q Base Case: q0 MIN  q0 Recursive Step: If p  p  Then q  q The map is defined everywhere That is, for all states q of MMIN there is a state q of M such that q  q If q  MMIN, there is a string w such that MIN(q0 MIN,w) = q Let q = (q0,w). Then q  q

q q Base Case: q0 MIN  q0 Recursive Step: If p  p  Then q  q The map is well defined Proof by contradiction. Suppose there are states q and q such that q  q and q  q We show that q and q are indistinguishable, so it must be that q = q

MMIN M Suppose there are states q and q such that q  q and q  q Suppose q and q are distinguishable MMIN M u w u w Accept Accept q q0 MIN q q0 Contradiction! v w v w q Reject q Reject q0 MIN q0

q q Base Case: q0 MIN  q0 Recursive Step: If p  p  Then q  q The map is onto Want to show: For all states q of M there is a state q of MMIN such that q  q For every q there is a string w such that M’ reaches state q’ after reading in w Let q be the state of MMIN after reading in w Claim: q  q

MMIN M The map is one-to-one Proof by contradiction. Suppose there are states p q such that p  q and q  q If p  q, then p and q are distinguishable MMIN M u w u w Accept Accept p q q0 MIN q0 Contradiction! v w v w Reject q Reject q q0 MIN q0

How can we prove that two regular expressions are equivalent?

Parting thoughts: Pumping for contradictions DFAs can’t count DFAs can be optimized Later: Can DFAs be learned? Questions?