Top-Down Parsing

Relationship between parser types

Recursive descent Recursive descent parsers simply try to build a top-down parse tree. It would be better if we always knew the correct action to take. It would be better if we could avoid recursive procedure calls during parsing.

Predictive parsers A predictive parser always knows which production to use, ( to avoid backtracking ) Example: for the productions stmt -> if ( expr ) stmt else stmt | while ( expr ) stmt | for ( stmt expr stmt ) stmt a recursive descent parser would always know which production to use, depending on the input token.

Transition diagrams Transition diagrams can describe recursive parsers, just like they can describe lexical analyzers, (but the diagrams are slightly different.) Construction: Eliminate left recursion from G Left factor G For each non-terminal A, do Create an initial and final (return) state For each production A -> X1 X2 … Xn, create a path from the initial to the final state with edges X1 X2 … Xn.

Example transition diagrams An expression grammar with left recursion With ambiguity E -> E+T | T T -> T*F | F F -> (E) | id Corresponding transition diagrams: Eliminating the ambiguity E -> T E’ E’ -> + T E’ | ε T -> F T’ T’ -> * F T’ | ε F -> ( E ) | id

The parsing table and parsing program The table is a 2D array M[A,a] where A is a nonterminal symbol and a is a terminal or $. At each step, the parser considers the top-of-stack symbol X and input symbol a: If both are $, accept If they are the same (nonterminals), pop X, advance input If X is a nonterminal, consult M[X,a]. If M[X,a] is “ERROR” call an error recovery routine. Otherwise, if M[X,a] is a production of the grammar X -> UVW, replace X on the stack with WVU (U on top)

Predictive parsing without recursion To get rid of the recursive procedure calls, we maintain our own stack.

Example Use the table-driven predictive parser to parse id + id * id Assuming parsing table E -> T E’ E’ -> + T E’ | ε T -> F T’ T’ -> * F T’ | ε F -> ( E ) | id Initial stack is $E Initial input is id + id * id $

Building a predictive parse table The construction requires two functions: 1. FIRST 2. FOLLOW

For First For a string of grammar symbols α, FIRST(α) is the set of terminals that begin all possible strings derived from α. If α =*> ε, then ε is also in FIRST(α). E -> T E’ E’ -> + T E’ | ε T -> F T’ T’ -> * F T’ | ε F -> ( E ) | id FIRST(E) = FIRST (T) = FIRST (F) = {( , id } FIRST(E’) = {+ , e} FIRST(T) = {( , id} FIRST(T’) = { *, e} FIRST(F) = {( , id }

For Follow Follow (E) = { ) , $ } Follow (E’) = Follow (E)= { ) ,$ } FOLLOW(A) for non terminal A is the set of terminals that can appear immediately to the right of A in some sentential form. If A can be the last symbol in a sentential form, then $ is also in FOLLOW(A). E -> T E’ E’ -> + T E’ | ε T -> F T’ T’ -> * F T’ | ε F -> ( E ) | id Follow (E) = { ) , $ } Follow (E’) = Follow (E)= { ) ,$ } Follow (T) = { +, Follow (E)}= {+ , ) , $} Follow (T’) = {+, ) ,$} Follow ( F) = {*, +, ), $ }

How to compute FIRST(α) If X is a terminal, FIRST(X) = X. Otherwise (X is a nonterminal), 1. If X -> ε is a production, add ε to FIRST(X) 2. If X -> Y1 … Yk is a production, then place a in FIRST(X) if for some i, a is in FIRST(Yi) and Y1…Yi-1 =*> ε. Given FIRST(X) for all single symbols X, Let FIRST(X1…Xn) = FIRST(X1) If ε ∈ FIRST(X1), then add FIRST(X2), and so on…

How to compute FOLLOW(A) Place $ in FOLLOW(S) (for S the start symbol) If A -> α B β, then FIRST(β)-ε is placed in FOLLOW(B) If there is a production A -> α B or a production A -> α B β where β =*> ε, then everything in FOLLOW(A) is in FOLLOW(B). Repeatedly apply these rules until no FOLLOW set changes.

Example FIRST and FOLLOW For our favorite grammar: E -> TE’ E’ -> +TE | ε T -> FT’ T’ -> *FT’ | ε F -> (E) | id What is FIRST() and FOLLOW() for all nonterminals?

Parse table construction with FIRST/FOLLOW Basic idea: if A -> α and a is in FIRST(α), then we expand A to α any time the current input is a and the top of stack is A. Algorithm: For each production A -> α in G, do: For each terminal a in FIRST(α) add A -> α to M[A,a] If ε ∈ FIRST(α), for each terminal b in FOLLOW(A), do: add A -> α to M[A,b] If ε ∈ FIRST(α) and $ is in FOLLOW(A), add A -> α to M[A,$] Make each undefined entry in M[ ] an ERROR

Example predictive parse table construction For our favorite grammar: E -> TE’ E’ -> +TE | ε T -> FT’ T’ -> *FT’ | ε F -> (E) | id What the predictive parsing table?

LL(1) grammars The predictive parser algorithm can be applied to ANY grammar. But sometimes, M[ ] might have multiply defined entries. Example: for if-else statements and left factoring: stmt -> if ( expr ) stmt optelse optelse -> else stmt | ε When we have “optelse” on the stack and “else” in the input, we have a choice of how to expand optelse (“else” is in FOLLOW(optelse) so either rule is possible)

LL(1) grammars If the predictive parsing construction for G leads to a parse table M[ ] WITHOUT multiply defined entries, we say “G is LL(1)” 1 symbol of lookahead Leftmost derivation Left-to-right scan of the input

LL(1) grammars Necessary and sufficient conditions for G to be LL(1): If A -> α | β There does not exist a terminal a such that a ∈ FIRST(α) and a ∈ FIRST(β) At most one of α and β derive ε If β =*> ε, then FIRST(α) does not intersect with FOLLOW(β). This is the same as saying the predictive parser always knows what to do!

Model of a non recursive predictive parser. a + b $ X Y Z $ Input buffer stack Predictive parsing program/driver Parsing Table M Model of a non recursive predictive parser.

Moves made by predictive parser on input id + id * id STACK INPUT OUTPUT $E $E' T $E' T' F $E' T' id $E' T' $E' $E' T + $E' T' F * $ id + id * id$ + id * id$ id * id$ * id$ id$ E  T E' T  F T' F  id T'   E'  + T E' T'  * F T' E'   Moves made by predictive parser on input id + id * id

Nonrecursive Predictive Parsing 1. If X = a = $, the parser halts and announces successful completion of parsing. 2. If X = a  $, the parser pops X off the stack and advances the input pointer to the next input symbol. 3. If X is a nonterminal, the program consults entry M[X, a] of the parsing table M. This entry will be either an X-production of the grammar or an error entry. If, for example, M[X, a] = {X  UVW}, the parser replaces X on top of the stack by WVU (with U on top). As output, we shall assume that the parser just prints the production used; any other code could be executed here. If M[X, a] = error, the parser calls an error recovery routine.

Parsing table M for grammar NONTER-MINAL INPUT SYMBOL Id + * ( ) $ E E' T T' F E  TE' T  FT' F  id E'  +TE' T'   T'  *FT' F  (E) E'   Parsing table M for grammar

Top-down parsing recap RECURSIVE DESCENT parsers are easy to build, but inefficient, and might require backtracking. TRANSITION DIAGRAMS help us build recursive descent parsers. For LL(1) grammars, it is possible to build PREDICTIVE PARSERS with no recursion automatically. Compute FIRST() and FOLLOW() for all nonterminals Fill in the predictive parsing table Use the table-driven predictive parsing algorithm