1. FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY
15-453
FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY

2. Some Homework 1 Statistics:
Each question was graded out of 20 points.
Max: 100
Mean: 78
Median: 79.02
Std Dev: 15.65

3. Your 1st Project Report is (New Homework on the Web)
NOTE :
Your 1st Project Report is
Due Tuesday!
Turn in your Homework!
(New Homework on the Web)

4. PDAs ARE EQUIVALENT TO CFGs
THURSDAY Jan 31

5. string
pop
push
ε,ε → $
0,ε → 0
1,0 → ε
ε,$ → ε
1,0 → ε

6. CONTEXT-FREE GRAMMARS
Production rules
terminals
variable
A → 0A1
A → ε
 (VΣ)* A
 0A1
 00A11
 000A111 
(yields)
A *
(derives)

7. A Language L is generated by a CFG
L is recognized by a PDA

8. Suppose L is generated by a CFG G = (V, Σ, R, S)
Construct P = (Q, Σ, Γ, , q, F) that recognizes L
A Language L is generated by a CFG 
L is recognized by a PDA

9. Suppose L is generated by a CFG G = (V, Σ, R, S)
Construct P = (Q, Σ, Γ, , q, F) that recognizes L
ε,ε → S$
For each rule 'A → w’  R:
ε,A → wR
For each terminal a  Σ:
a,a → ε
Here’s a “sketch” of what the PDA P will look like.
Start: Push $ on the stack, then push start variable S.
When $ is at the top of the stack again, we accept iff there’s no more input. --
We add transitions of the following kind.
The top of the stack will be either a variable or a terminal.
For each variable A, and production rule involving A,
if we pop A we’ll push the symbols in w on the stack, in reverse.
(Why reverse? Last in, first out.)
For each terminal a,
if it matches the current input symbol, we pop it, otherwise reject.
ε,$ → ε

10. S → aTb T → Ta | ε ε,ε → $ ε,ε → T ε,S → b ε,ε → T ε,ε → S ε,T → a
What language does this grammar generate? {a^m b^n | m, n >= 1}
First we push $, then S.
If we pop S, then we push aTb on the stack, in reverse.
If we pop T, we have two choices: we can push Ta, or push nothing.
We pop a letter if it corresponds to the input symbol.
Anything else, we reject.
We accept if we reach $ again, and there is no more input.
ε,$ → ε
ε,ε → a
ε,T → ε
a,a → ε
b,b → ε

11. S → aTb S * ab (derives) T → Ta | ε ε,ε → $ ε,ε → T ε,S → b ε,ε → T
Let’s look at what the PDA does on ab.
(Clearly the grammar generates ab)
Here is one accepting computation for it.
We push b, T, a onto the stack.
We pop a, read input a.
We pop T, read “epsilon”
We pop b, read input b.
Then we pop $.
ε,$ → ε
ε,ε → a
ε,T → ε
a,a → ε
b,b → ε

12. Suppose L is generated by a CFG G = (V, Σ, R, S)
Describe P = (Q, Σ, Γ, , q, F) that recognizes L :
(1) Push $ and then S on the stack
(2) Repeat the following steps forever:
(a) Pop the stack, call the result X.
(b) If X is a variable A, guess a rule that matches A and push result into the stack
Rather than giving a big formal description,
let’s describe the PDA in pseudocode
(This is going to be necessary in many cases in the future, too.)
(c) If X is a terminal, read next symbol from input and compare it to terminal.
If they’re different, reject.
(d) If X is $: then accept iff no more input

13. A Language L is generated by a CFG
L is recognized by a PDA

14. A Language L is generated by a CFG  L is recognized by a PDA
Given PDA P = (Q, Σ, Γ, , q, F)
Construct a CFG G = (V, Σ, R, S) such that
L(G) = L(P)
First, simplify P to have the following form:
(1) It has a unique accept state, qacc
To make it so that each transition either pops or pushes, but not both, we make two states for each one we have in the PDA: one for popping and one for pushing.
(If it’s an epsilon-transition that doesn’t push or pop, then we can just have it push a dummy symbol on the stack, then pop the dummy symbol in another state.)
(2) It empties the stack before accepting
(3) Each transition either pushes a symbol or pops a symbol, but not both at the same time

15. SIMPLIFY ε,ε → $ q0 q1 0,ε → 0 1,0 → ε ε,$ → ε q3 q2 1,0 → ε
Recall this PDA. What does it recognize?
How would we simplify this PDA?

16. SIMPLIFY ε,ε → E ε,ε → $ Q q0 q1 0,ε → 0 1,0 → ε ε,ε → ε ε,$ → ε q3 q2
First we make a new start state, that epsilon-transitions to the old one, and pushes “E”.
(“E” is a new symbol, will always be a the bottom of the stack.)
Then we have epsilon-transitions to a unique final state---that’s just like with NFAs.
Now to make sure the stack is empty, we add “popping” epsilon-transitions,
and we only accept if E is popped last. q4
ε,E → ε
ε,σ → ε q5

17. SIMPLIFY ε,ε → E ε,ε → $ Q q0 q1 0,ε → 0 ε,ε → D 1,0 → ε q’0 ε,$ → ε
Finally, to make sure that there is always a symbol popped or pushed onto the stack…
we add an additional state that pushes and pops a dummy symbol
q’3
ε,D → ε q4
ε,E → ε q5
ε,σ → ε

18. V = {Apq | p,qQ } S = Aq0qacc
Idea For Our Grammar G:
For every pair of states p and q in PDA P,
G will have a variable Apq whose production rules will generate all strings x that can take:
P from p with an empty stack
to q with an empty stack
So if we can generate a string from the start variable,
this means the string will take P from start state with an empty stack
to final state with an empty stack!
I.e. it accepts!
V = {Apq | p,qQ }
S = Aq0qacc

19. ε,ε → E Aq0q1 generates?  Aq1q2 generates? {0n1n | n > 0}
ε,ε → $ Q q0 q1
0,ε → 0
ε,ε → D
1,0 → ε
q’0
ε,$ → ε q3 q2
1,0 → ε
ε,ε → D
q’3
Aq0q1 generates? 
ε,D → ε
What set of strings would A_q0 q1 generate?
Why is this set for A_{q1 q2}? We want an empty stack!
Aq1q2 generates? q4
{0n1n | n > 0}
ε,E → ε q5
Aq1q3 generates? 
ε,σ → ε
WANT: Apq generates all strings that take p with an empty stack to q with empty stack

20. P’s first move on x must be a push P’s last move on x must be a pop
WANT: Apq generates all strings that take p with an empty stack to q with empty stack
Let x be such a string
P’s first move on x must be a push
P’s last move on x must be a pop
Two possibilities:
1. The symbol popped at the end is exactly the one pushed at the beginning
1: So, P empties its stack right at the end, never in the middle.
2: That is, P emptied its stack at some point in the middle, and then started pushing symbols on it again.
2. The symbol popped at the end is not the one pushed at the beginning

21. x = ayb takes p with empty stack to q with empty stack
1. The symbol t popped at the end is exactly the one pushed at the beginning
stack
height r s a b
input
string
push t
pop t
Suppose we have a string x, that begins with letter a, ends with letter b.
We can chart the height of the stack over time as a graph. Each symbol of the input is on the x-axis.
Suppose we run the PDA from state p, and read x, then some computation path ends with q in empty stack.
Suppose after we read input a, we push t and go to state r.
Then after reading y, we are in some state s, and the stack is the same as before (it just has t on it).
Finally, popping t and reading b, we are in q.
How can we capture this behavior with a production rule? p
─ ─ ─ ─ x ─ ─ ─ ─ q
δ(p, a, ε) → (r, t)
Apq → aArsb
δ(s, b, t) → (q, ε)

22. 2. The symbol popped at the end is not the one pushed at the beginning
stack
height
input
string p r q
Let r be the state the PDA is in when it emptied the stack completely.
Then we’d have Apq -> Apr Arq
From p with an empty stack, we get to r with an empty stack,
then from r with an empty stack we get to q with an empty stack.
This is just like going from p to q with empty stacks.
Apq → AprArq

23. For every p, q, r, s  Q, t  Γ and a, b  Σε
Formally:
V = {Apq | p, qQ }
S = Aq0qacc
For every p, q, r, s  Q, t  Γ and a, b  Σε
If (r, t)  (p, a, ε) and (q, ε)  (s, b, t)
Then add the rule Apq → aArsb
For every p, q, r  Q,
add the rule Apq → AprArq
For every p  Q,
add the rule App → ε

24. x can bring P from p with an empty stack to q with an empty stack 
For all x,
Apq generates x 
x can bring P from p with an empty stack
to q with an empty stack 
Proof (by induction on the number of steps in the derivation of x from Apq):
Inductive Step:
Base Case:
The derivation has 1 step:
App  ε
Assume true for derivations of length ≤ k and prove true for derivations of length k+1:
In the base case: Only one possible derivation.
Clearly input eps takes P from p with empty stack to state p with empty stack…
The first step in the derivation is one of two possibilities:
Case 1: x = yz. A_{pr} ->* y, A_{rq} ->* z.
Both of these strings are generated in at most k steps.
So y can bring P from p to r, and z can bring r to q, by IH.
Thus x brings P from p to q.
Case 2: x=ayb
if P starts in p then after reading a, it can go to r and push t.
By IH, y can bring P from r to s, with t at the top of the stack.
Then from s reading b, it can pop t and end in state q.
Apq * x in k+1 steps
Apq → AprArq
Apq → aArsb
(r,t)  (p,a,ε) and (q, ε)  (s,b,t)
state
push
state
alphabet
pop

25. x can bring P from p with an empty stack to q with an empty stack 
For all x,
Apq generates x 
x can bring P from p with an empty stack
to q with an empty stack 
Proof (by induction on the number of steps in the computation of P from p to q with empty stacks, on input x):
Base Case:
The computation has 0 steps
So it starts and end in the same state:
We must show that App * x
But it must be that x = ε, so we are done

26. Write x as ayb. Ars * y by I.H., use Apq → aArsb
Inductive Step:
Assume true for computations of length ≤ k, we’ll prove true for computations of length k+1
Suppose that P has a computation where x brings p to q with empty stacks in k+1 steps
Two cases:
1. The stack is empty only at the beginning and the end of this computation
1. The symbol t pushed at the beginning is the symbol popped at the end.
Let a be input symbol read at beginning, b read at end.
So x = ayb.
Let r be the state after the first step, let s be the state before the last step.
y can bring P from r with an empty stack to s with an empty stack
==> A_{rs} generates y, by IH.
With A_{pq} -> aA_{rs}b, A_{pq} generates x.
2. x = yz.
Let r be a state in which the stack becomes empty in the middle.
Let y be the input read to that point, z be input read after.
Then y brings p to r => A_{pr} generates y
Then y brings r to q => A_{rq} generates z
by IH.
Thus A_{pq} generates x from the rule: A_{pq} -> A_{pr} A_{rq}
Write x as ayb. Ars * y by I.H., use Apq → aArsb
2. The stack is empty somewhere in the middle of the computation
Write x as yz. Apr * y, Arq * z by I.H., Apq → AprArq

27. A Language L is generated by a CFG
L is recognized by a PDA

28. Corollary: Every regular language is context-free
Why does this follow?