1. Context-Free Languages
Theory of Computation
Context-Free Languages

2. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
Last Time
Is L = {0ix | i ≥ 0, x {0,1}* and |x| ≤ i} regular?
Is L = {0i | i is prime} regular?
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

3. Context-Free Languages
Context-Free Languages (CFL) are described using Context-Free Grammars (CFG).
A CFG is a simple recursive method of specifying grammar rules which can generate strings in a language – these languages are the CFL’s.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

4. Context-Free Grammars
The following is an example of a CFG, call it G1:
A → 1A (A and B = variables)
A → B
B → # (0, 1 and # = terminals)
A grammar consists of a collection of substitution rules (projections).
A is start variable in this case – usually occurs on left hand side of topmost rule.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

5. Context-Free Grammars
Use grammar to describe a language by generating each string of language:
Write down start variable.
Find a variable and a rule which starts with that variable. Replace written variable with the right hand side of this rule.
Repeat the second step until no variables remain.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

6. Context-Free Grammars
G1 generates the string 1111#0000 using the following sequences:
A → 1A0 → 11A00 → 111A000 → 1111A0000 → 1111B0000 → 1111#0000
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

7. Context-Free Grammars
All strings generated in this manner constitute the language of the grammar.
L(G1) = language of grammar G1.
Can show that L(G1) is {1n#0n | n ≥ 0}.
Any language that can be generated by some context-free grammar is called a context-free language.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

8. Context-Free Grammars
Note:
For convenience if a variable has several rules they are often abbreviated:
A → 1A0 and A → B may be represented as: A → 1A0 | B, where “|” represents “or”.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

9. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
Definition of a CFG
A context-free grammar is a 4-tuple (V, Σ , R, S), where
V are the variables (finite set)
Σ are the terminal states (finite set)
R is the set of rules
S is the start variable, S V
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

10. Context-Free Grammars
In grammar G1, V = {A, B}, Σ = {0, 1, #}, S = A, and R is the collections of the rules:
A → 1A0
A → B
B → #
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

11. Context-Free Grammars
Consider G3 = ({S}, {a,b}, R, S). The set of rules R, is
S → aSb | SS | ε
This grammar generates strings such as ab, abab, aababb and aaabbb.
Note that L(G3) is the language of all strings of properly nested parnetheses.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

12. Context-Free Grammars
Consider the language of palindromes:
L = {w {a,b}* | w = wr}
This is not regular, but the language can be generated by the following rules:
S → aSa S → bSb
S → a S → b
S → ε
V = {S}, S = S, Σ = {a,b} and R are rules above.
Check to ensure if produces palindromes.
What about a CFG for the non-palindromes?
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

13. Derivation Trees Consider the same example grammar:
And a derivation of :
Fall 2006
Costas Busch - RPI

14. yield
Costas Busch - RPI
Fall 2006

15. yield
Costas Busch - RPI
Fall 2006

16. yield
Costas Busch - RPI
Fall 2006

17. yield
Costas Busch - RPI
Fall 2006

18. Derivation Tree
(parse tree)
yield
Costas Busch - RPI
Fall 2006

19. Sometimes, derivation order doesn’t matter
Leftmost derivation:
Rightmost derivation:
Give same
derivation tree
Costas Busch - RPI
Fall 2006

20. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
Designing CFG’s
Many CFL’s are the union of simpler ones.
Construct the smaller simpler CFG’s and then construct them to give the larger CFG for the CFL.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

21. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
Designing CFG’s
Construct a grammar for the language: {0n1n | n ≥ 0} {1n0n | n ≥ 0}.
Firstly construct the grammar
S1 → 0 S11 | ε for the language {0n1n | n ≥ 0}, and the grammar
S2 → 1 S20 | ε for the language {1n0n | n ≥ 0}, and then add the rule S → S1| S2 to give the grammar:
S → S1| S2
S1 → 0 S11 | ε
S2 → 1 S20 | ε
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

22. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
Designing CFG’s
We can construct a CFG for a regular language by first constructing a DFA for the language.
A DFA may be converted into an equivalent CFG as follows:
Make a variable Ri for each state qi of the DFA.
Add the rule Ri → aRj to the CFG if δ(qi, a) = qj is a transition in the DFA.
Add the rule Ri → ε if qi is an accept state of the DFA.
Make R0 the start state of the grammar where q0 is the start state of the machine.
Verify that it works!!
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

23. Union, concatenation and closure of CFG’s
Theorem: If L1 and L2 are CFL’s then L1 L2, L1L2 and L*1 are also CFL’s.
That is, the context-free languages are closed under union, concatenation and Kleene-closure.
The proofs are constructive.
Begin with two grammars: G1 = (V1, Σ , R1, S1) and G2 = (V2, Σ , R2, S2), generating CFL’s L1 and L2 respectively.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

24. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
Union of CFG’s
The new CFG Gx is made as:
Σ remains the same
Sx is the new start variable
Vx = V V2 {Sx}
Rx = R1 R {Sx → S1|S2}
Explanation: All we have done is augment the variable set with a new start state and then allowed the new start state to map to either of the two grammars. So, we’ll generate strings from either L1 or L2, i.e. L1 L2
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

25. Concatenation of CFG’s
The new CFG Gy is made as:
Σ remains the same
Sy is the new start variable
Vy = V V {Sy}
Ry = R1 R {Sx → S1S2}
Explanation: Again, all we have done is to augment the variable set with a new start state, and then allowed the new start state to map to the concatenation of the two original start symbols. So, we will generate strings that begin with strings from L1 and end with strings from L2, i.e. L1L2
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

26. Kleene-Closure of CFG’s
The new CFG Gz is made as:
Σ remains the same
Sz is the new start variable
Vz = V {Sz}
Rz = R {Sz → S1Sz | ε}
Explanation: Again we have augmented the variable set with a new start state, and then allowed the new start state to map to either S1Sz or ε. This means we can generate strings with zero or more strings made from expanding the variable S1, i.e. L*1
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

27. Fall 2006
Ambiguity
Costas Busch - RPI

28. Grammar for mathematical expressions
Example strings:
Denotes any number
Fall 2006
Costas Busch - RPI

29. A leftmost derivation
for
Costas Busch - RPI
Fall 2006

30. Another
leftmost derivation
for
Costas Busch - RPI
Fall 2006

31. Two derivation trees
for
Costas Busch - RPI
Fall 2006

32. take
Costas Busch - RPI
Fall 2006

33. Compute expression result using the tree
Good Tree
Bad Tree
Compute expression result
using the tree
Fall 2006
Costas Busch - RPI

34. Two different derivation trees
may cause problems in applications which
use the derivation trees:
Evaluating expressions
In general, in compilers
for programming languages
Fall 2006
Costas Busch - RPI

35. Ambiguous Grammar: (Two different derivation trees give two
A context-free grammar is ambiguous
if there is a string which has:
two different derivation trees or
two leftmost derivations
(Two different derivation trees give two
different leftmost derivations and vice-versa)
Fall 2006
Costas Busch - RPI

36. this grammar is ambiguous since
Example:
this grammar is ambiguous since
string
has two derivation trees
Costas Busch - RPI
Fall 2006

37. this grammar is ambiguous also because
string
has two leftmost derivations
Costas Busch - RPI
Fall 2006

38. Another ambiguous grammar:
IF_STMT
if EXPR then STMT
if EXPR then STMT else STMT
Variables
Terminals
Very common piece of grammar
in programming languages
Fall 2006
Costas Busch - RPI

39. IF_STMT if expr1 then STMT if expr2 then stmt1 else stmt2 IF_STMT if
If expr1 then if expr2 then stmt1 else stmt2
IF_STMT if
expr1
then
STMT if
expr2
then
stmt1
else
stmt2
Two derivation trees
IF_STMT if
expr1
then
STMT
else
stmt2 if
expr2
then
stmt1
Fall 2006
Costas Busch - RPI

40. In general, ambiguity is bad and we want to remove it
Sometimes it is possible to find
a non-ambiguous grammar for a language
But, in general we cannot do so
Fall 2006
Costas Busch - RPI

41. A successful example: Equivalent Ambiguous Grammar Non-Ambiguous
generates the same
language
Costas Busch - RPI
Fall 2006

42. Unique
derivation tree
for
Costas Busch - RPI
Fall 2006

43. An un-successful example:
is inherently ambiguous:
every grammar that generates this
language is ambiguous
Fall 2006
Costas Busch - RPI

44. Example (ambiguous) grammar for :
Costas Busch - RPI
Fall 2006

45. has always two different derivation trees (for any grammar)
The string
has always two different derivation trees
(for any grammar)
For example
Fall 2006
Costas Busch - RPI

46. Pushdown Automata (PDA)
Pushdown Automata are similar to nondeterministic finite automata but have an extra element – stack.
This stack provided extra memory space.
Also allows pushdown automata to recognise some nonregular languages.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

47. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
PDA
Finite Automata
Pushdown Automata ……
state
control
state
control
stack
input
input
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

48. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
PDA
Why is a stack useful?
It can hold an unlimited amount of information.
Remember that a FA was unable to recognise the language {0n1n | n ≥ 0} because it can’t store large numbers.
However, a PDA does not have this problem, due to the presence of a stack: it can use the stack to store how many 0’s it has seen.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

49. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
PDA
A PDA can write symbols on the stack and read them back later.
Writing a symbol “pushes down” all the other symbols on the stack.
Only the top symbol in the stack can ever be read – once read it is removed.
Writing a symbol is known as “pushing” and reading a symbol known as “popping”.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

50. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
PDA
The PDA has no way of checking for an empty stack.
Gets around this by placing a special character, $, on the stack initially.
Then if it ever sees the $ again it knows that the stack is effectively empty.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

51. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
PDA
Important:
Deterministic and non-deterministic PDA’s are not equivalent in power.
Non deterministic PDA’s recognise certain languages which no deterministic pushdown automata can recognise.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

52. Formal Definition of PDA’s
The formal definition of a PDA is similar to that of a FA, except for the stack.
The stack contains symbols drawn from some alphabet.
The machine may use different alphabets for its input and the stack
We need to specify an input alphabet Σ and a stack alphabet Γ
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

53. Formal Definition of PDA’s
In order to formally define a PDA we need to determine the transition function. Recall:
Σε = Σ {ε} and Γε = Γ {ε}
The domain of the transition function is Q × Σε × Γε
Therefore, the current state, next input symbol read and top state of the stack determine the next move of the PDA.
Note that either symbol may be ε meaning that the machine may move without reading a symbol from the input or the stack.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

54. Formal Definition of PDA’s
What is the range of the transition function?
The machine may enter some new state and possibly write to the top of the stack.
The function δ can represent this by returning a member of Q along with a member of Γε, i.e. a member of Q × Γε
A number of legal next moves may be allowed
The transition function incorporates this nondeterminism in the usual way – i.e. returning a set of members of Q × Γε, that is, a member of P(Q × Γε).
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

55. Formal Definition of PDA’s
A pushdown automata is a 6-tuple (Q,Σ,Γ,δ,q0,F), where Q, Σ, Γ, and F are all finite sets, and
Q is the set of states,
Σ is the input alphabet,
Γ is the stack alphabet,
δ: Q×Σε×Γε → P(Q × Γε) is the transition function,
q0 Q is the start state, and
F Q is the set of accept states.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

56. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
PDA
The computation of a PDA, M, is as follows:
It accepts input w if w can be written as w = w1w2…wm where each w1 Σε and sequences of states r0,r1,…,rm Q and strings s0, s1,…, sm Γ* exist that satisfy the following:
r0 = q0 and s0 = ε: M starts out properly, in start state and empty stack.
For i = 0, 1,.., m-1, we have (ri+1, b) δ(ri, wi+1, a), where si = at and si+1 = bt for some a,b Γε and t Γ*: M moves properly according to the state, stack and next input symbols.
rm F: Accept states occurs when the input end is reached.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

57. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
PDA
In a state transition we write “a, b → c” to signify that when machine reads input a it may replace symbol b on top of the stack with a c.
Any of a, b, c can be ε.
If a is ε, the machine may make this transition without reading any input symbol.
If b is ε the machine performs transition without reading and popping any stack symbol.
If c is ε machine does not write any symbol to stack.
Can we design a PDA to recognise the language: {0n1n | n ≥ 0} ?
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

58. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
Context Free
A language is context free if and only if some pushdown automata recognises it.
Every regular language is recognised by a finite automaton and every finite automaton is automatically a pushdown automaton that ignores the stack, we can note that every regular language is also a context-free language.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

59. Regular and Context-Free Languages
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

60. Non-Context Free Languages
Recall that we looked at the pumping lemma previously for showing that certain languages are not regular.
We will look at a similar lemma for context-free languages.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

61. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
Pumping Lemma
The pumping lemma states that every context-free language has a special value called the pumping length such that all longer strings in the language can be “pumped”.
Pumped, this time, means that the string can be divided into five parts such that the 2nd and 4th parts of the string may be repeated together any number of times and the resulting string still be part of the language.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

62. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
Pumping Lemma
If A is a context-free language, then there is a number p (the pumping length) where, if s is a string in A of length at least p, then s may be divided into 5 parts, s = uvxyz such that:
for each i ≥ 0, uvixyiz A
|vy| > 0, and
|vxy| ≤ p.
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth

63. Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth
Pumping Lemma
Let us look at some examples:
Use the Pumping Lemma to show that the following languages are not context-free:
B = {anbncn | n ≥ 0}
C = {aibjck | 0 ≤ i ≤ j ≤ k}
D = {ww | w {0,1}* }
E = {0n1n0n1n | n ≥ 0}
Dr. A. Mooney, Dept. of Computer Science, NUI Maynooth