1. Bernd Fischer bfischer@cs.sun.ac.za RW713: Compiler and Software Language Engineering

2. Bottom-Up Parsing

3. Top-down vs. bottom-up parsing

4. Ex +Nat* Ex Ex  Nat | (Ex) | Ex + Ex | Ex * Ex Matched input string  success ! Corresponds to a Leftmost derivation  hence LL Ex Ex + Ex Nat + Ex Nat + Ex * Ex Nat + Nat * Ex Nat + Nat * Nat

5. Top-down vs. bottom-up parsing Nat + Nat * Nat Ex + Nat * Nat Ex + Ex * Nat Ex + Ex * Ex Ex + Ex Ex +Nat* Ex Ex  Nat | (Ex) | Ex + Ex | Ex * Ex Reached start symbol  success ! Corresponds to a Rightmost derivation (in reverse)!  hence LR

6. Shift-Reduce Parsing

7. REMINDER Use a parse stack to represent the derivation: initialize s = S if x = ε –if s = ε then accept else reject if tos ∈ T –if x i = tos then pop; skip x i else reject if tos ∈ N –pick a production tos → α in P; pop; push(α) Top-down parsing searches for the (leftmost) derivation using a stack. The parser stack can be explicit or implicit. symbol by symbol, in reverse order.

8. Shift-reduce parsing searches for the (rightmost) derivation using a stack. Use a parse stack to represent the derivation: initialize s = ε if x = ε –if s = S then accept else reject shift: push(x i ) reduce: if s = αX 1 X 2... X n –pick a production A → X 1 X 2... X n in P; pop n ; push(A); The parser stack is typically explicit. tos

9. Shift-reduce parsing searches for the (rightmost) derivation using a stack.

12. Use a parse stack to represent the derivation: initialize s = ε if x = ε –if s = S then accept else reject shift: push(x i ) reduce: if s = αX 1 X 2... X n –pick a production A → X 1 X 2... X n in P; pop n ; push(A); shift or reduce? which production? The parser stack is typically explicit.

13. Shift-reduce parsing searches for the (rightmost) derivation using a stack. Schematic syntax tree with α ∈ (N ∪ T)*, x, y ∈ T*, a ∈ T, and start symbol S read pointer stack “shift a”“reduce with A → γ” ? ? need to constrain choice

14. Shift-reduce parsing maintains a viable prefix on the stack. Definition: Let G = (N, T, P, S) be a context-free grammar and S ⇒ r * βAy ⇒ r βγy. Then γ is called a handle or redex of the right-sentential form βγy. Each prefix of βγ is called a viable prefix of G. Shift-reduce parsing invariants: The parser stack is a viable prefix. S ⇒ r * sy

15. Shift-reduce parsing maintains a viable prefix on the stack. Definition: Let G = (N, T, P, S) be a context-free grammar and S ⇒ r * βAy ⇒ r βγy. Then γ is called a handle or redex of the right-sentential form βγy. Each prefix of βγ is called a viable prefix of G. Theorem: The language of viable prefixes of a grammar G is regular. Corollary: We can build and use a DFA to recognize viable prefixes...... and so constrain the choice of a shift-reduce parser.

16. LR(0) Parsing

17. LR(0) items

18. Parsing with an NFA over LR(0) items.

25. Constructing the LR(0) NFA Let G = (N, T, P, S) be a context-free grammar. For each nonterminal A ∈ N, construct the item automaton. Build union of item automata: Start state is the start state of item automaton for S, final states are final states of item automata. Add transitions from each state which contains the dot in front of a nonterminal A to the starting state of the item automaton of A. Theorem: The automaton obtained in this way exactly accepts the language of viable prefixes of G if all states are declared to be final.

26. Constructing the LR(0) NFA

27. Constructing the LR(0) DFA

29. Direct Construction of the LR(0) DFA Needs closure operation on itemsets

30. Direct Construction of the LR(0) DFA Needs closure operation on itemsets

31. Direct Construction of the LR(0) DFA Needs closure operation on itemsets

32. Direct Construction of the LR(0) DFA Needs goto operation to represent transition relation

33. Direct Construction of the LR(0) DFA Example:

34. Direct Construction of the LR(0) DFA Example:

35. Direct Construction of the LR(0) DFA Example:

36. Direct Construction of the LR(0) DFA Example:

37. Direct Construction of the LR(0) DFA Example:

38. Direct Construction of the LR(0) DFA Example:

39. Direct Construction of the LR(0) DFA Example:

40. Direct Construction of the LR(0) DFA Example:

41. Direct Construction of the LR(0) DFA Example:

42. Direct Construction of the LR(0) DFA Example:

43. Direct Construction of the LR(0) DFA Example:

44. Parsing with the LR(0) DFA In principle the LR(0) DFA can be used for parsing: run DFA over sentential form until accepting state is reached apply accepting rule from itemset to reduce tail of viable prefix re-run DFA over new sent. form until accepting state is reached apply accepting rule from itemset to reduce tail of viable prefix... ⇒ instead: use pushdown automaton viable prefix redex

45. LR(0) Pushdown Automata Basic ideas: states == itemsets (conceptionally) uses two stacks: –states –grammar elements uses four kinds of actions per state –shift – push current input symbol –reduce(rule) – reduce with rule –accept –error – default uses goto-table: state x grammar symbol → state usually ignored

46. LR(0) Pushdown Automata Basic loop: state contains shift item A → α ● aβ –check that x i = a; syntax error if not –push a on symbol stack –push goto[tos, a] on stack state contains reduce item A → α ● –pop |α| elements off symbol stack –pop |α| elements off stack –push A on symbol stack –push goto[tos, A] on stack –accept if A = S and x = ε “dot before terminal” “dot at end”

47. Parsing with the LR(0) PDA

59. Does this always work...? shift/reduce conflict reduce/reduce conflict

60. Recess Refresher

61. Pop-Quiz... Remember: Check whether Γ 5 is LR(0)!

62. SLR(1)-Parsing

63. Does this always work...? reduce/reduce conflict shift/reduce conflict Grammar tells us to… … reduce to A if next input is a … reduce to B if next input is b … shift if next input is c follow sets

64. SLR(1) parsing uses follow sets to resolve conflicts in an LR(0) state. follow(A) = {a} follow(B) = {b} follow(A) ∩ follow(B) = ∅ ⇒ use next token to pick rule ⇒ resolves reduce/reduce conflict c ∉ follow(A) ∪ follow(B) ⇒ resolves shift/reduce conflict

65. SLR(1) Grammars Definition: Let G = (N, T, P, S) be a context-free grammar and I be a state of the LR(0) DFA for G. I has an SLR(1) conflict iff I contains two different reduce items A → α ● and B → β ● such that follow(A) ∩ follow(B) ≠ ∅ ; or two items A → α ● and B → β ● aγ such that a ∈ follow(A). G is an SLR(1) grammar if there is no SLR(1) conflict.

66. LR(0) vs. SLR(1) LR(0): uses sets of LR(0) items as states uses GOTO[state, grammar symbol] as transitions actions depend on state only SLR(1) uses sets of LR(0) items as states uses GOTO[state, grammar symbol] as transitions actions depend on state and next input token

67. xLR(1) parsing tables Different LR(1) parsing variants use the same tables:

68. xLR(1) parsing tables Different LR(1) parsing variants use the same tables: empty entry == syntax error empty entry == can’t happen SLR(1) tables can easily be constructed from the LR(0) DFA via the SLR(1) definition.

69. Construction of SLR(1) tables

70. Does this always work...? follow(A) = {a,b} follow(B) = {b}

71. Does this always work...? b ∈ follow(A) a ∈ follow(A) follow(A) = {a,b} follow(B) = {b} The follow-sets... are a global approximation of possible continuations ignore (left) derivation context

72. Canonical LR(1)-Parsing

73. LR(1) items are pairs of LR(0) items and a look-ahead symbol.

74. LR(1) item computation 4. LR(1) property same as SLR(1) property (but uses lookaheads from LR(1) items)

75. LR(1) DFA construction - example

80. ● ● ●

81. LR(1) DFA construction - example same LR(0) item in different LR(1) states ⇒ different look-aheads reflect different derivation contexts ⇒ state-splitting removes SLR(1) conflict

82. LALR(1)-Parsing

83. LALR(1) DFA construction - conceptual LALR(1) tables can be constructed directly, without going via the LR(1) states. [ASLU, 4.7.5] union of lookahead sets

84. LALR(1) DFA construction - conceptual 4. LALR(1) property same as SLR(1) property (but uses lookaheads from merged LR(1) items)

85. LALR(1) DFA construction - example

95. conflicts are rare, though

96. “Hacking” xLR(1) tables

97. Handling precedence and associativity

98. Idea: remove conflicting parse table entries

99. Handling precedence and associativity Idea: remove conflicting parse table entries

100. Handling precedence and associativity Idea: remove conflicting parse table entries

101. Error handling Idea: add specific error handlers into ACTION table: default: tell legal tokens (i.e., have non-error entry)

102. xLR comparison

103. LR vs LL

104. LR(0) vs SLR(1) vs LALR(1) vs LR(1) method of choice (yacc and friends)

105. LR(0) vs SLR(1) vs LALR(1) vs LR(1)

106. Formal Language Theory

107. Classes of Grammars

111. Classes of Languages