1. CS21 Decidability and Tractability
Lecture 6
January 18, 2019
January 18, 2019
CS21 Lecture 6

2. Outline Context-Free Grammars and Languages
equivalence of NPDAs and CFGs
non context-free languages
January 18, 2019
CS21 Lecture 6

3. Context-Free Grammars
A → 0A1
A → B
B → #
start symbol
terminal symbols
non-terminal symbols
production
January 18, 2019
CS21 Lecture 6

4. CFG example Balanced parentheses:
( )
( ( ) ( ( ( ) ( ) ) ) )
a string w in Σ* = { (, ) }* is balanced iff:
# “(”s equals # “)”s, and
for any prefix of w, # “(”s ≥ # “)”s
Exercise: design a CFG for balanced parentheses.
January 18, 2019
CS21 Lecture 6

5. CFG example Arithmetic expressions over {+,*,(,),a}
(a + a) * a
a * a + a + a + a + a
A CFG generating this language:
<expr> → <expr> * <expr>
<expr> → <expr> + <expr>
<expr> → (<expr>) | a
January 18, 2019
CS21 Lecture 6

6. CFG example A derivation of the string: a+a*a
<expr> → <expr> * <expr>
<expr> → <expr> + <expr>
<expr> → (<expr>) | a
A derivation of the string: a+a*a
<expr> ⇒ <expr> * <expr>
⇒ <expr> + <expr> * <expr>
⇒ a + <expr> * <expr>
⇒ a + a * <expr>
⇒ a + a * a
January 18, 2019
CS21 Lecture 6

7. Parse Trees Easier way to picture derivation: parse tree
grammar encodes grouping information; this is captured in the parse tree.
<expr>
<expr> *
<expr>
<expr> +
<expr> a a a
January 18, 2019
CS21 Lecture 6

8. CFGs and parse trees
<expr> → <expr> * <expr>
<expr> → <expr> + <expr>
<expr> → (<expr>) | a
Is this a good grammar for arithmetic expressions?
can group wrong way (+ precedence over *)
January 18, 2019
CS21 Lecture 6

9. Solution to problem forces correct precedence in parse tree grouping
<expr> → <expr> + <term> | <term>
<term> → <term> * <factor> | <factor>
<factor> → <term> * <factor>
<factor> → (<expr>) | a
forces correct precedence in parse tree grouping
within parentheses, * cannot occur as ancestor of + in the parse tree.
January 18, 2019
CS21 Lecture 6

10. Parse Trees parse tree for a + a * a in new grammar: <expr> +
<expr> → <expr> + <term> | <term>
<term> → <term> * <factor> | <factor>
<factor> → <term> * <factor>
<factor> → (<expr>) | a
<expr> +
<expr>
<term>
<term>
<term> *
<factor>
<factor>
<factor> a a a
January 18, 2019
CS21 Lecture 6

11. Some facts about CFLs CFLs are closed under
union (proof?)
concatenation (proof?)
star (proof?)
Every regular language is a CFL
proof?
January 18, 2019
CS21 Lecture 6

12. NPDA, CFG equivalence
Theorem: a language L is recognized by a NPDA iff L is described by a CFG.
Must prove two directions:
(⇒) L is recognized by a NPDA implies L is described by a CFG.
(⇐) L is described by a CFG implies L is recognized by a NPDA.
January 18, 2019
CS21 Lecture 6

13. NPDA, CFG equivalence
Proof of (⇐): L is described by a CFG implies L is recognized by a NPDA. # 1 # 1 # 1 q1 q2 q3
an idea: A # #
A → 0A1
A → # 1 1 1 $ $ $ : : :
January 18, 2019
CS21 Lecture 6

14. what is wrong with this approach?
NPDA, CFG equivalence
we’d like to non-deterministically guess the derivation, forming it on the stack
then scan the input, popping matching symbol off the stack at each step
accept if we get to the bottom of the stack at the end of the input.
what is wrong with this approach?
January 18, 2019
CS21 Lecture 6

15. NPDA, CFG equivalence q1 q2 q3 only have access to top of stack
combine steps 1 and 2:
allow to match stack terminals with tape during the process of producing the derivation on the stack # 1 # 1 # 1 q1 q2 q3 A #
A → 0A1
A → # A 1 1 1 $ $ $
January 18, 2019
CS21 Lecture 6

16. NPDA, CFG equivalence informal description of construction:
place $ and start symbol S on the stack
repeat:
if the top of the stack is a non-terminal A, pick a production with A on the lhs and substitute the rhs for A on the stack
if the top of the stack is a terminal b, read b from the tape, and pop b from the stack.
if the top of the stack is $, enter the accept state.
January 18, 2019
CS21 Lecture 6

17. NPDA, CFG equivalence one transition for each production A → w
ε, A → w = w1w2…wk q r
ε, A → w
shorthand for:
b, b → ε
ε, ε → wk-1
ε, $ → ε
one transition for each terminal b q2 … qk
ε, ε → w1
ε, A → wk q1 q r
January 18, 2019
CS21 Lecture 6

18. NPDA, CFG equivalence
Proof of (⇒): L is recognized by a NPDA implies L is described by a CFG.
harder direction
first step: convert NPDA into “normal form”:
single accept state
empties stack before accepting
each transition either pushes or pops a symbol
January 18, 2019
CS21 Lecture 6

19. NPDA, CFG equivalence
main idea: non-terminal Ap,q generates exactly the strings that take the NPDA from state p (w/ empty stack) to state q (w/ empty stack)
then Astart, accept generates all of the strings in the language recognized by the NPDA.
January 18, 2019
CS21 Lecture 6

20. NPDA, CFG equivalence Two possibilities to get from state p to q:
generated by Ap,r
generated by Ar,q
stack height p q r
abcabbacacbacbacabacabbabbabaacabbbababaacaccaccccc
input
string taking NPDA from p to q
January 18, 2019
CS21 Lecture 6

21. NPDA, CFG equivalence NPDA P = (Q, Σ, Γ, δ, start, {accept}) CFG G:
non-terminals V = {Ap,q : p, q ∈ Q}
start variable Astart, accept
productions:
for every p, r, q ∈ Q, add the rule
Ap,q → Ap,rAr,q
January 18, 2019
CS21 Lecture 6

22. NPDA, CFG equivalence Two possibilities to get from state p to q:
generated by Ar,s
stack height r s p
push d
pop d q
abcabbacacbacbacabacabbabbabaacabbbababaacaccaccccc
input
string taking NPDA from p to q
January 18, 2019
CS21 Lecture 6

23. NPDA, CFG equivalence NPDA P = (Q, Σ, Γ, δ, start, {accept}) CFG G:
from state p, read a, push d, move to state r
NPDA P = (Q, Σ, Γ, δ, start, {accept})
CFG G:
non-terminals V = {Ap,q : p, q ∈ Q}
start variable Astart, accept
productions:
for every p, r, s, q ∈ Q, d ∈Γ and a, b ∈ (Σ ∪ {ε})
if (r, d) ∈ δ(p, a, ε), and
(q, ε) ∈ δ(s, b, d), add the rule
Ap,q → aAr,sb
from state s, read b, pop d, move to state q
January 18, 2019
CS21 Lecture 6

24. NPDA, CFG equivalence NPDA P = (Q, Σ, Γ, δ, start, {accept}) CFG G:
non-terminals V = {Ap,q : p, q ∈ Q}
start variable Astart, accept
productions:
for every p ∈ Q, add the rule
Ap,p → ε
January 18, 2019
CS21 Lecture 6

25. NPDA, CFG equivalence two claims to verify correctness:
if Ap,q generates string x, then x can take NPDA P from state p (w/ empty stack) to q (w/ empty stack)
if x can take NPDA P from state p (w/ empty stack) to q (w/ empty stack), then Ap,q generates string x
January 18, 2019
CS21 Lecture 6

26. NPDA, CFG equivalence
1. if Ap,q generates string x, then x can take NPDA P from state p (w/ empty stack) to q (w/ empty stack)
induction on length of derivation of x.
base case: 1 step derivation. must have only terminals on rhs. In G, must be production of form Ap,p → ε.
January 18, 2019
CS21 Lecture 6

27. NPDA, CFG equivalence
1. if Ap,q generates string x, then x can take NPDA P from state p (w/ empty stack) to q (w/ empty stack)
assume true for derivations of length at most k, prove for length k+1.
verify case: Ap,q → Ap,rAr,q →k x = yz
verify case: Ap,q → aAr,sb →k x = ayb
January 18, 2019
CS21 Lecture 6

28. NPDA, CFG equivalence
2. if x can take NPDA P from state p (w/ empty stack) to q (w/ empty stack), then Ap,q generates string x
induction on # of steps in P’s computation
base case: 0 steps. starts and ends at same state p. only has time to read empty string ε.
G contains Ap,p → ε.
January 18, 2019
CS21 Lecture 6

29. NPDA, CFG equivalence
2. if x can take NPDA P from state p (w/ empty stack) to q (w/ empty stack), then Ap,q generates string x
induction step. assume true for computations of length at most k, prove for length k+1.
if stack becomes empty sometime in the middle of the computation (at state r)
y is read going from state p to r (Ap,r→* y)
z is read going from state r to q (Ar,q→* z)
conclude: Ap,q → Ap,rAr,q →* yz = x
January 18, 2019
CS21 Lecture 6

30. NPDA, CFG equivalence
2. if x can take NPDA P from state p (w/ empty stack) to q (w/ empty stack), then Ap,q generates string x
if stack becomes empty only at beginning and end of computation.
first step: state p to r, read a, push d
go from state r to s, read string y (Ar,s→* y)
last step: state s to q, read b, pop d
conclude: Ap,q → aAr,sb →* ayb = x
January 18, 2019
CS21 Lecture 6

31. NPDA, CFG equivalence
2. if x can take NPDA P from state p (w/ empty stack) to q (w/ empty stack), then Ap,q generates string x
if stack becomes empty only at beginning and end of computation.
first step: state p to r, read a, push d
go from state r to s, read string y (Ar,s→* y)
last step: state s to q, read b, pop d
conclude: Ap,q → aAr,sb →* ayb = x
January 18, 2019
CS21 Lecture 6

32. Pumping Lemma for CFLs
CFL Pumping Lemma: Let L be a CFL. There exists an integer p (“pumping length”) for which every w ∈ L with |w| ≥ p can be written as
w = uvxyz such that
for every i ≥ 0, uvixyiz ∈ L , and
|vy| > 0, and
|vxy| ≤ p.
January 18, 2019
CS21 Lecture 6

33. CFL Pumping Lemma Example
Theorem: the following language is not context-free:
L = {anbncn : n ≥ 0}.
Proof:
let p be the pumping length for L
choose w = apbpcp
w = aaaa…abbbb…bcccc…c
w = uvxyz, with |vy| > 0 and |vxy| ≤ p.
January 18, 2019
CS21 Lecture 6

34. CFL Pumping Lemma Example
possibilities:
w = aaaa…aaabbb…bbcccc…c
(if v, y each contain only one type of symbol, then pumping on them produces a string not in the language) u v y z x
January 18, 2019
CS21 Lecture 6

35. CFL Pumping Lemma Example
possibilities:
w = aaaa…abbbb…bccccc…c
(if v or y contain more than one type of symbol, then pumping on them might produce a string with equal numbers of a’s, b’s, and c’s – if vy contains equal numbers of a’s, b’s, and c’s. But they will be out of order.) u v y z x
January 18, 2019
CS21 Lecture 6

36. CFL Pumping Lemma Example
Theorem: the following language is not context-free:
L = {xx : x ∈ {0,1}*}.
Proof:
let p be the pumping length for L
try w = 0p10p1
can this be pumped?
January 18, 2019
CS21 Lecture 6

37. CFL Pumping Lemma Example
L = {xx : x ∈ {0,1}*}.
try w = 02p12p02p12p
w = uvxyz, with |vy| > 0 and |vxy| ≤ p.
case: vxy in first half.
then uv2xy2z = 0??...?1??...?
case: vxy in second half.
then uv2xy2z = ??...?0??...?1
case: vxy straddles midpoint
then uv0xy0z = uxz = 02p1i0j12p with i ≠ 2p or j ≠ 2p
January 18, 2019
CS21 Lecture 6

38. some non-terminal must repeat on long path
CFL Pumping Lemma
Proof: consider a parse tree for a very long string w ∈ L:
long path S
. . . A B C
. . .
. . . A D S C S A A B a A C b A D D C B A a b b S S b b B A b b b
some non-terminal must repeat on long path a a b a
January 18, 2019
CS21 Lecture 6

39. CFL Pumping Lemma Schematic proof: S S A S A A A A u v y z u v x y z A
January 18, 2019
CS21 Lecture 6

40. CFL Pumping Lemma Schematic proof: S S A S A A A u z u v x y z x u z
January 18, 2019
CS21 Lecture 6

41. CFL Pumping Lemma how large should pumping length p be?
need to ensure other conditions:
|vy| > |vxy| ≤ p
b = max # symbols on rhs of any production (assume b ≥ 2)
if parse tree has height ≤ h, then string generated has length ≤ bh (so length > bh implies height > h)
January 18, 2019
CS21 Lecture 6

42. CFL Pumping Lemma let m be the # of nonterminals in the grammar
to ensure path of length at least m+2, require
|w| ≥ p = bm+2
since |w| > bm+1, any parse tree for w has height > m+1
let T be the smallest parse tree for w
longest root-leaf path must consist of ≥ m+1 non-terminals and 1 terminal.
January 18, 2019
CS21 Lecture 6

43. CFL Pumping Lemma is |vy| > 0 ? is |vxy| ≤ p? S
must be a repeated non-terminal A on long path
select a repetition among the lowest m+1 non-terminals on path.
pictures show that for every i ≥ 0, uvixyiz ∈ L A A u v x y z
is |vy| > 0 ?
smallest parse tree T ensures
is |vxy| ≤ p?
red path has length ≤ m+2, so ≤ bm+2 = p leaves
January 18, 2019
CS21 Lecture 6