1. Fall 2017-2018 Compiler Principles Context-free Grammars Refresher
Roman Manevich
Ben-Gurion University of the Negev

2. Example grammar S  S ; S S  id := E S  print (L) E  id E  num
E  E + E
L  E
L  L, E
shorthand for Statement
shorthand for Expression
shorthand for List (of expressions)

3. CFG terminology S  S ; S S  id := E S  print (L) E  id E  num
E  E + E
L  E
L  L, E
Symbols: Terminals (tokens): ; := ( ) id num print Non-terminals: S E L
Start non-terminal: S Convention: the non-terminal appearing in the first derivation rule
Grammar productions (rules) N  α

4. More definitions
Sentential form: a sequence of symbols, terminals (tokens) and non-terminals
Sentence: a sequence of terminals (tokens)
Derivation step: given a sentential form αNβ and rule N  µ a step is the transition αNβ  αµβ
Derivation sequence: a sequence of derivation steps 1 …  k such that i  i+1 is the result of applying one production and k is a sentence

5. Language of a CFG
A word ω is in L(G) (valid program) if there exists a corresponding derivation sequence
Start the start symbol
Repeatedly replace one of the non-terminals by a right-hand side of a production
Stop when the sentence contains only terminals
ω is in L(G) if S * ω
Leftmost derivation
Rightmost derivation

6. Leftmost derivation S  S ; S S  id := E S  print (L) E  id E  num
a := ; b := S
S  S ; S
S  id := E
S  print (L)
E  id
E  num
E  E + E
L  E
L  L, E 1
=> S ; S 2
=> id := E ; S 3
=> id := num ; S 4 5
=> id := num ; id := E 6
=> id := num ; id := E + E 7
=> id := num ; id := num + E 8
=> id := num ; id := num + num
id := num ; id := num + num

7. Rightmost derivation S  S ; S S  id := E S  print (L) E  id
a := ; b := S
S  S ; S
S  id := E
S  print (L)
E  id
E  num
E  E + E
L  E
L  L, E 1
=> S ; S 2
=> S ; id := E 3
=> S ; id := E + E 4 5
=> S ; id := E + num 6
=> S ; id := num + num 7
=> id := E ; id := num + num 8
=> id := num ; id := num + num
id := num ; id := num + num

8. Canonical derivations
Leftmost/rightmost derivations may not be unique but they allow describing a derivation by the sequence of production rules taken (since non-terminal is already known)
Leftmost derivation example: 1, 2, 5, 2, 6, 5, 5
Rightmost derivation example: 1, 2, 6, 5, 5, 2, 5

9. Parse trees Tree nodes are symbols, children ordered left-to-right
Each internal node is non-terminal and its children correspond to one of its productions
N  µ1 … µk
Root is start non-terminal
Leaves are tokens
Yield of parse tree: left-to-right walk over leaves N µ1 … µk

10. Parse tree exercise S  S ; S S  id := E S  print (L) E  id E  num
L  L, E
Draw parse tree for expression id :=
num ; id :=
num +
num

11. Parse tree exercise S  S ; S S  id := E S  print (L) E  id E  num
Order-independent representation S
S  S ; S
S  id := E
S  print (L)
E  id
E  num
E  E + E
L  E
L  L, E S S E E E E id :=
num ; id :=
num +
num
Equivalently add parentheses labeled by non-terminal names
(S(Sa := (E56)E)S ; (Sb := (E(E7)E (E3)E)E)S)S

12. Capabilities and limitations of CFGs
CFGs naturally express
Hierarchical structure
A program is a list of classes, A Class is a list of definition…
Alternatives
A definition is either a field definition or a method definition
Beginning-end type of constraints
Balanced parentheses S  (S)S | ε
Cannot express
Correlations between unbounded strings (identifiers)
For example: variables are declared before use: ω S ω
Handled by semantic analysis (attribute grammars)
p. 173

13. Badly-formed grammars
By Oren neu dag (Own work) [CC-BY-SA-3.0 ( via Wikimedia Commons

14. Badly-formed grammars
A non-terminal N is reachable if S * αNβ
A non-terminal N is generating if N * ω
A grammar G is badly-formed if it either contains unreachable non-terminals or non-generating non-terminals
G1 = S  x N  y
G2 = S  x | N N  a N b N
Exercise: algorithm to test whether a grammar is badly-formed
Theorem: for every grammar G there exists an equivalent well-formed grammar G’ ( that is, L(G)=L(G’) ) Proof: exercise
From now on, we will only handle well-formed grammars

15. Ambiguity in Context-free grammars

16. Sometimes there are two parse trees
Arithmetic expressions: E  id
E  num
E  E + E E  E * E E  ( E )
1 + (2 + 3)
(1 + 2) + 3 E E E E E E E E E E
num(1) +
num(2) +
num(3)
num(1) +
num(2) +
num(3)
Leftmost derivation E E + E num + E num + E + E num + num + E
num + num + num
Rightmost derivation E E + E E + num E + E + num E + num + num num + num + num

17. Is ambiguity a problem for compilers?
Arithmetic expressions: E  id
E  num
E  E + E E  E * E E  ( E )
1 + (2 + 3)
(1 + 2) + 3
Depends on semantics E E E E E E E E E E
num(1) +
num(2) +
num(3)
= 6
num(1) +
num(2) +
num(3)
= 6
Leftmost derivation E E + E num + E num + E + E num + num + E
num + num + num
Rightmost derivation E E + E E + num E + E + num E + num + num num + num + num

18. Problematic ambiguity example
Arithmetic expressions: E  id
E  num
E  E + E E  E * E E  ( E )
1 + 2 * 3
1 + (2 * 3)
(1 + 2) * 3
This is what we usually want: * has precedence over + E E E E E E E E E E
num(1) +
num(2) *
num(3)
= 7
num(1) +
num(2) *
num(3)
= 9
Leftmost derivation E E + E num + E num + E * E num + num * E
num + num * num
Rightmost derivation E E * E E * num E + E * num E + num * num num + num * num

19. Ambiguous grammars
A grammar is ambiguous if there exists a word for which there are
Two different leftmost derivations
Two different rightmost derivations
Two different parse trees
Property of grammars, not languages

20. Facts about ambiguous grammars
Some languages are inherently ambiguous – no unambiguous grammars exist [Parikh 1961]
Checking whether an arbitrary grammar is ambiguous is undecidable [Hopcroft, Motwani, Ullman, 2001]