1. CS 154, Lecture 4: Limitations on DFAs (I),
Pumping Lemma, Minimizing DFAs

2. Non-Regular Languages
Regular or Not?
D = { w | w has equal number of occurrences of 01 and 10 }
REGULAR!
C = { w | w has equal number of 1s and 0s}
NOT REGULAR!
How can we prove that there is no DFA for a particular language?
Surprising Algorithms (even in restricted models) are routinely being discovered

3. The Pumping Lemma: Structure in Regular Languages
Let L be a regular language
Then there is a positive integer P s.t.
for all strings w  L with |w| ≥ P there is a way to write w = xyz, where:
1. |y| > 0 (that is, y  ε)
|xy| ≤ P
For all i ≥ 0, xyiz  L
Why is it called the pumping lemma? The word w gets pumped into longer and longer strings…

4. Let w be a string where w  L and |w| ≥ P
Proof: Let M be a DFA that recognizes L
Let P be the number of states in M
Let w be a string where w  L and |w| ≥ P
We show: w = xyz
|y| > 0
|xy| ≤ P
xyiz  L for all i ≥ 0 y x w z … q0 qj qk
q|w|
Claim: There must exist j and k such that 0 ≤ j < k ≤ P, and qj = qk

5. Applying the Pumping Lemma
Let’s prove that EQ={ w | #1s = #0s} is not regular.
By contradiction. Assume EQ is regular.
Let P be as in pumping lemma. Let w = 0P1P∈ EQ.
 Can write w = xyz, with |y| > 0, |xy| ≤ P, such that for all i ≥ 0, xyiz is also in EQ
Claim: The string y must be all zeroes.
Why? Because |xy| ≤ P and w = xyz = 0P1P
But then xyyz has more 0s than 1s
Contradiction!

6. Applying the Pumping Lemma
Prove: SQ = {0n2 | n ≥ 0} is not regular
Assume SQ is regular. Let w = 0P2
 Can write w = xyz, with |y| > 0, |xy| ≤ P, such that for all i ≥ 0, xyiz is also in EQ
So xyyz ∊ SQ. Note that xyyz = 0P2+|y|
Note that 0 < |y| ≤ P
So |xyyz| = P2 + |y| ≤ P2 + P < P2 + 2P + 1 = (P+1)2
and P2 < |xyyz| < (P+1)2
Therefore |xyyz| is not a perfect square!
Hence 0P2+|y| = xyyz ∉ SQ, so our assumption must be false.
 SQ is not regular!

7. Does this DFA have a minimal number of states?

9. Is this minimal?
How can we tell in general?

10. Theorem:
For every regular language L, there is a unique (up to re-labeling of the states) minimal-state DFA M* such that L = L(M*).
Furthermore, there is an efficient algorithm which, given any DFA M, will output this unique M*.
If these were true for more general models of computation, that would be an engineering breakthrough

11. Note: There isn’t a uniquely minimal NFA

12. Extending transition function  to strings
Given DFA M = (Q, Σ, , q0, F), we extend  to a function  : Q  Σ* → Q as follows:
(q, ε) = q
(q, ) =
(q, )
(q, 1 …k+1 ) = ( (q, 1 …k ), k+1 )
(q, w) = the state of M reached after reading in w, starting from state q
Note: (q0, w)  F  M accepts w
Def. w  Σ* distinguishes states q1 and q2 if exactly one of (q1, w), (q2, w) is a final state
Def. w  Σ* distinguishes states q1 and q2 if (q1, w)  F  (q2, w)  F

13. Distinguishing two states
Def. w  Σ* distinguishes states q1 and q2 if
exactly one of (q1, w), (q2, w) is a final state
I’m in q1 or q2, but which? How can I tell?
Ok, I’m accepting!
Must have been q1
Here… read this W

14. Fix M = (Q, Σ, , q0, F) and let p, q  Q
Definitions:
State p is distinguishable from state q
if there is w  Σ* that distinguishes p and q ( there is w  Σ* so that exactly one of (p, w), (q, w) is a final state)
State p is indistinguishable from state q
if p is not distinguishable from q
( for all w  Σ*, (p, w)  F  (q, w)  F)
Pairs of indistinguishable states are redundant…

15. ε distinguishes all final states from non-final states
Which pairs of states are distinguishable here?
ε distinguishes all final states from non-final states

16. The string 10 distinguishes q0 and q3
Which pairs of states are distinguishable here?
The string 10 distinguishes q0 and q3

17. The string 0 distinguishes q1 and q2
Which pairs of states are distinguishable here?
The string 0 distinguishes q1 and q2

18. Fix M = (Q, Σ, , q0, F) and let p, q, r  Q
Define a binary relation ∼ on the states of M:
p ∼ q iff p is indistinguishable from q
p ≁ q iff p is distinguishable from q
Proposition: ∼ is an equivalence relation
p ∼ p (reflexive)
p ∼ q  q ∼ p (symmetric)
p ∼ q and q ∼ r  p ∼ r (transitive)
Proof?

19. Fix M = (Q, Σ, , q0, F) and let p, q, r  Q
Therefore, the relation ∼ partitions Q into disjoint equivalence classes
Proposition: ∼ is an equivalence relation
[q] := { p | p ∼ q }

21. Algorithm: MINIMIZE-DFA
Input: DFA M
Output: DFA MMIN such that:
L(M) = L(MMIN)
MMIN has no inaccessible states
MMIN is irreducible ||
For all states p  q of MMIN, p and q are distinguishable
Theorem: MMIN is the unique minimal DFA that is equivalent to M

22. The states of MMIN will be the equivalence classes of states of M
Intuition:
The states of MMIN will be the
equivalence classes of states of M
We’ll uncover these equivalent states with
a dynamic programming algorithm

23. The Table-Filling Algorithm
Input: DFA M = (Q, Σ, , q0, F)
Output:
(1) DM = { (p, q) | p, q  Q and p ≁ q }
(2) EQUIVM = { [q] | q  Q }
We know how to find those pairs of states that the string ε distinguishes…
Use this and iteration to find those pairs distinguishable with longer strings
The pairs of states left over will be indistinguishable
High-Level Idea:

24. The Table-Filling Algorithm
Input: DFA M = (Q, Σ, , q0, F)
Output:
(1) DM = { (p, q) | p, q  Q and p ≁ q }
(2) EQUIVM = { [q] | q  Q }
Base Case: For all (p, q) such that p accepts and q rejects  p ≁ q

25. The Table-Filling Algorithm
Input: DFA M = (Q, Σ, , q0, F)
Output:
(1) DM = { (p, q) | p, q  Q and p ≁ q }
(2) EQUIVM = { [q] | q  Q }
Base Case: For all (p, q) such that p accepts and q rejects  p ≁ q
Iterate: If there are states p, q and symbol σ  Σ satisfying: D D
 (p, ) = p
mark p ≁ q ≁ 
 (q, ) = q D
Repeat until no more D’s can be added

26. D D D D D D

27. D D D D

28. Suppose (p, q) is marked D at a later point.
Claim: If (p, q) is marked D by the Table-Filling algorithm, then p ≁ q
Proof: By induction on the number of steps in the algorithm before (p,q) is marked D
If (p, q) is marked D at the start, then one state’s in F and the other isn’t, so ε distinguishes p and q
Suppose (p, q) is marked D at a later point.
Then there are states p, q such that:
1. (p, q) are marked D  p ≁ q (by induction)
So there’s a string w s.t. (p, w)  F  (q, w)  F
2. p = (p,) and q = (q,), where   Σ
The string w distinguishes p and q!

29. Proof (by contradiction):
Claim: If (p, q) is not marked D by the Table-Filling algorithm, then p ∼ q
Proof (by contradiction):
Suppose the pair (p, q) is not marked D by the algorithm, yet p ≁ q (call this a “bad pair”)
Then there is a string w such that |w| > 0 and:
(p, w)  F and (q, w)  F
(Why is |w| > 0?)
We have w = w, for some string w’ and some   Σ
Of all such bad pairs, let p, q be a pair with the shortest distinguishing string w
Let p = (p,) and q = (q,)
Then (p, q) is also a bad pair, but with a SHORTER distinguishing string, w !

30. Output: Equivalent minimal-state DFA MMIN
Algorithm MINIMIZE
Input: DFA M
Output: Equivalent minimal-state DFA MMIN
1. Remove all inaccessible states from M
2. Run Table-Filling algorithm on M to get:
EQUIVM = { [q] | q is an accessible state of M }
3. Define: MMIN = (QMIN, Σ, MIN, q0 MIN, FMIN)
QMIN = EQUIVM, q0 MIN = [q0], FMIN = { [q] | q  F }
MIN( [q],  ) = [ ( q,  ) ]
Claim: L(MMIN) = L(M)

31. D D D D D D D D D

32. D D D D D D D D D

33. Thm: MMIN is the unique minimal DFA equivalent to M
Claim: If L(M)=L(MMIN) and M has no inaccessible states and M’ is irreducible  there is an isomorphism between M and MMIN
Suppose for now the Claim is true.
If M’ is a minimal DFA, then M’ has no inaccessible states and is irreducible (why?) So the Claim implies:
Let M’ be a minimal DFA for M.
 there is an isomorphism between M’ and the DFA MMIN that is output by MINIMIZE(M).
 The Thm holds!

34. Thm: MMIN is the unique minimal DFA equivalent to M
Claim: If L(M)=L(MMIN) and M has no inaccessible states and M’ is irreducible  there is an isomorphism between M and MMIN
Proof: We recursively construct a map from the states of MMIN to the states of M
Base Case: q0 MIN  q0
Recursive Step:
If p  p  
Then q  q q q

35. q q
Base Case: q0 MIN  q0
Recursive Step:
If p  p 
Then q  q

36. q q
Base Case: q0 MIN  q0
Recursive Step:
If p  p 
Then q  q
We need to prove:
The map is defined everywhere
The map is well defined
The map is a bijection
The map preserves all transitions: If p  p then MIN(p, )  ’(p’, )
(this follows from the definition of the map!)

37. The map is defined everywhereq q
Base Case: q0 MIN  q0
Recursive Step:
If p  p 
Then q  q
The map is defined everywhere
That is, for all states q of MMIN
there is a state q of M such that q  q
If q  MMIN, there is a string w such that MIN(q0 MIN,w) = q
Let q = (q0,w). Then q  q

38. q q
Base Case: q0 MIN  q0
Recursive Step:
If p  p 
Then q  q
The map is well defined
Proof by contradiction. Suppose there are states q and q such that
q  q and q  q
We show that q and q are indistinguishable,
so it must be that q = q

39. MMIN M Suppose there are states q and q such that
q  q and q  q
Suppose q and q are distinguishable
MMIN M u w u w
Accept
Accept q
q0 MIN q
q0
Contradiction! v w v w q
Reject
q
Reject
q0 MIN
q0

40. q q
Base Case: q0 MIN  q0
Recursive Step:
If p  p 
Then q  q
The map is onto
Want to show: For all states q of M there is a state q of MMIN such that q  q
For every q there is a string w such that M’ reaches state q’ after reading in w
Let q be the state of MMIN after reading in w Claim: q  q

41. MMIN M The map is one-to-one
Proof by contradiction. Suppose there are states p q such that p  q and q  q
If p  q, then p and q are distinguishable
MMIN M u w u w
Accept
Accept p q
q0 MIN
q0
Contradiction! v w v w
Reject q
Reject q
q0 MIN
q0

42. How can we prove that two regular expressions are equivalent?

43. Parting thoughts:
Pumping for contradictions
DFAs can’t count
DFAs can be optimized
Later: Can DFAs be learned?
Questions?