YANGYANG 1 Chap 5 LL(1) Parsing LL(1) left-to-right scanning leftmost derivation 1-token lookahead parser generator: Parsing becomes the easiest! Modifying parsers is also convenient.

YANGYANG 2 Chap 5 LL(1) Parsing Given the productions A   1 A   A   n During a (leftmost) derivation,... A... ...  1... or ...  2... or ...  n... Which route should we choose? (Try-and-error is not a good idea.) » Use the lookahead symbols.

YANGYANG 3 Chap 5 LL(1) Parsing Consider the situation: We are about to expand a nonterminal A and there are several productions whose LHS are A: A   1 A   A   n We choose one of the productions based on the lookahead token. Which one should we choose? Consider First(  1 ) First(  2 ) First(  n ) and if  i , then consider also Follow(A). *

YANGYANG 4 Chap 5 LL(1) Parsing Define predict(A   ) =First(  )  (if  First(  ) then Follow(A)) If the lookahead token a  predict(A  ) then we use the production A  to expand A. What if a  predict(A   1 ) and a  predict(A   2 )? What if a  predict(A  ) for all productions A  whose LHS are A?

YANGYANG 5 Chap 5 LL(1) Parsing Property of LL(1) grammars: If a grammar is LL(1), then for any two productions A  A  First(  Follow(A))  First(  Follow(A)) = 

YANGYANG 6 Chap 5 LL(1) Parsing Given the FIRST and FOLLOW sets in Fig. 5-2 and 5-3, calculate the predict set for each production.

YANGYANG 7 Chap 5 LL(1) Parsing §5.2 LL(1) Parse Table The predict() function may be represented as an LL(1) parse table. T: Vn * Vt  P  {error} a b A 3 B error.... T[A, a] = A  if a  predict(A  ) = error otherwise A grammar is LL(1) iff all entries in the parse table contain a unique production or the error flag.

YANGYANG 8 Chap 5 LL(1) Parsing Figure 5.5 The LL(1) table for Micro

YANGYANG 9 Chap 5 LL(1) Parsing 5.3 LL(1) parsers Similar to scanners, there are two kinds of parsers: 1. built-in: recursive descent 2. table-driven

YANGYANG 10 Chap 5 LL(1) Parsing 1. built-in stmt() { token = next_token(); switch(token) { case ID: /*production 5:stmt-->ID:= ;*/ match(ID); match(ASSIGN); exp(); match(SEMICOLON); break; case READ: /*production 6*/... case WRITE: /*production 7*/... default: syntax_error(....); }

YANGYANG 11 Chap 5 LL(1) Parsing It is obvious that these recursive descent parsing procedures can be generated automatically from the grammar. grammar LL(1) table parser generator recursive descent parser However, it is difficult for the parser generator to integrate the semantic routines into the (generated) recursive descent parser automatically.

YANGYANG 12 Chap 5 LL(1) Parsing 2. table-driven parser (+) generic driver Only the LL(1) table needs to be changed when the grammar is modified. (+) non-recursive (faster) Parser maintains a stack itself. No recursive calls.

YANGYANG 13 Chap 5 LL(1) Parsing lldriver() { push( START_SYMBOL ); a := next_token; while stack is not empty do { X := symbol on stack top if ( X is a nondeterminal && T[X, a] == X  Y 1  Y m ) ’ ) pop(1); push Y m, Y m-1, , Y 1 else if ( x == a ) pop(1); a := next_token(); else if ( x is an action symbol ) pop(1); call correspond routine else sntax_error(); }

YANGYANG 14 Chap 5 LL(1) Parsing Ex. begin A := B A; end $ a = begin X = parse stack Trace the action of the parser on this example.

YANGYANG 15 Chap 5 LL(1) Parsing 5.5 Action symbols Action symbols may be processed by the parser in a similar way. 1. in recursive descent parsers Ex. gen_action( “ ID:= #assign ” ); ” ) will generate the following code: match(ID); match(ASSIGN); exp(); assign(); match(semicolon); Parameters are transmitted through a semantic stack. Semantic stack is a stack of semantic records. Parser stack is a stack of grammar (and action) symbols.

YANGYANG 16 Chap 5 LL(1) Parsing 2. in LL(1) driver Action symbols are pushed into the parse stack in the same way as grammar symbols. When action symbols are on stack top, the driver calls corresponding semantic routines. See previous slide for lldriver. Parameters are also transmitted through semantic stack.

YANGYANG 17 Chap 5 LL(1) Parsing §5.6 Making grammars LL(1) Not all grammars are LL(1). However, some non-LL(1) grammars can be made LL(1) by simple modifications. When is a grammar not LL(1)? When there is an entry in the parse table that contains more than one productions. Ex ID ,5.... This is called a conflict, which means we do not know which production to use when is on stack top and ID is the next input token.

YANGYANG 18 Chap 5 LL(1) Parsing Conflicts are classfied into two categories: 1. common prefix 2. left recursion Common prefix Ex.  if then  if then else Consider when is on stack top,  ‘ if ’ is the next input token. We cannot choose which production to use at this time. In general, if we have two productions A  A  and First(  )  First(  )  , then we have a conflict.

YANGYANG 19 Chap 5 LL(1) Parsing Solution: factor out the common prefix Ex.  if then   else

YANGYANG 20 Chap 5 LL(1) Parsing 2. left recursion: productions of the form: A  A  grammar with left-recursive productions are not LL(1) because we may have A  A  A  same lookahead

YANGYANG 21 Chap 5 LL(1) Parsing Problem: left recursion A  A  A  A  Intuition: all the strings derivable from A have the form: , , ,  , , ,  Solution: replace the productions So we may use the following productions instead: A  T A  T T  T  T

YANGYANG 22 Chap 5 LL(1) Parsing Ex. Given the left-recursive grammar: E  E + T E  T T  T * P T  P P  ID After eliminating left recursion, we get E  T A A  A  + T A T  P B B  B  * P B P  ID

YANGYANG 23 Chap 5 LL(1) Parsing 3. more general solution ex.   ID :   ID := ; We cannot decide which production to use when is on the stack top and ID is the next token: ? lookahead ID ID

YANGYANG 24 Chap 5 LL(1) Parsing Solution: use the following productions (which essentially look ahead 2 tokens)  ID  :  := ;  ID := ; Try two examples: A: B := C ; B := C ;

YANGYANG 25 Chap 5 LL(1) Parsing 4. For more difficult cases, we use semantic routines to help parsing. Ex. In Ada, we may declare arrays as A: array(I.. J, BOOLEAN) A straightforward grammar is (for array bound) ..  ID  … and ID  First( ) This grammar is not LL(1) because we cannot make a decision when is on stack top and ID is the next token.

YANGYANG 26 Chap 5 LL(1) Parsing Solution:   .. All grammars can be transformed into Greibach Normal Form, in which a production has the form: A  a  terminal So given a grammar G, we can do G  GNF  no common prefix no left recursion but still NOT LL(1)! Ex. S  a A a S  b A b a A  b A  consider A is on stacktop; b is next token.

YANGYANG 27 Chap 5 LL(1) Parsing §5.7 The dangling-else problem Consider if a then if b then x := 1 else x := 2 Two possibilities: a a T T F b b T F T x := 2 x := 1 x := 2 x := 1 The problem is which ‘  if ’  the ‘  else ’ belong to. In essence, we are trying to find an LL(1) grammar for the set { [ i ] j | i  j  0} But is it possible?

YANGYANG 28 Chap 5 LL(1) Parsing 1st attempt: G1 S  [ S C S  C  ] C  This grammar is ambiguous. Consider [ [ ] S S [ S C [ S C [ S C [ S C ] ]

YANGYANG 29 Chap 5 LL(1) Parsing 2nd attempt: we can make ] be associated with the nearest unpaired [ as follows: S  [ S S  T T  [ T ] T  This grammar is not ambiguous. Consider [ [ ] S [ S [ T ] However, this grammar is not LL(1), either. Consider the case when S is on stack top and [ is the next input token. [  First( [ S ) [  First( T ) This grammar can be parsed with a bottom-up parser, but not a top-down parser.

YANGYANG 30 Chap 5 LL(1) Parsing Solution: conflicts + special rules 1. G  S ; 2. S  if S E 3. S  other 4. E  else S 5. E  The parse table if else other ; G 1 1 S 2 3 E 4,5 5 conflicts We can enforce that T[E, else] = 4th rule. This essentially forces ‘ else ’ to be matched with the nearest unpaired ‘ if ’.

YANGYANG 31 Chap 5 LL(1) Parsing Alternative solution: change the language. Add ‘ end if ’ at the end of every ‘ if ’. S  if S E S  other E  else S end if E  end if

YANGYANG 32 Chap 5 LL(1) Parsing §5.9 Properties of LL(1) parsers: A correct leftmost parse is guaranteed. All LL(1) grammars are un-ambiguous. linear time and linear space

YANGYANG 33 Chap 5 LL(1) Parsing § llgen Page 776 of the book output from llgen *define decrtn 1 ifprocess 2

YANGYANG 34 Chap 5 LL(1) Parsing § LL(k) parsing Recall a grammar is LL(1) only if for any two productions A  and A , First(  Follow(A))  First(  Follow(A)) =  To generalize, we write for any two productions A  and A  First k (  Follow k (A))  First k (  Follow k (A)) =  if G is strong LL(k). The word ‘ strong ’ means G imposes too strong a condition.

YANGYANG 35 Chap 5 LL(1) Parsing Consider G  S $ S  a A a S  b A b a A  b A  – This grammar is not LL(1) When A is on stack top and b is next token, we cannot choose between A  b and A . stack input b..... A Does it help if we can look ahead two tokens? NO! if the next two tokens are bb then we should choose A  b. if the next two tokens are ba then we cannot make a choice.

YANGYANG 36 Chap 5 LL(1) Parsing case 1. input is aba a A A S a a G $ $ $ lookahead match lookahead ab a ba at this point, we should choose A  b case 2. input is bba b A A b b S a a G $ $ $ lookahead match lookahead bb b ba at this point, we should choose A 

YANGYANG 37 Chap 5 LL(1) Parsing So the problem is not the limited number of lookahead tokens. The problem is in the ‘ context ’.

YANGYANG 38 Chap 5 LL(1) Parsing Therefore, the grammar is not strong LL(1). Actually, we can verify that the grammar is not strong LL(k) for all k  1 by verify that First k ( ba$ )  First k ( bFollow k (A) )  First k ( Follow k (A) ) for all k  1

YANGYANG 39 Chap 5 LL(1) Parsing However, it is possible to parse the language of the grammar under the following conditions: 1. look ahead two tokens 2. from left to right 3. using the left context We call such grammars LL(2), rather than strong LL(2). Note that LL(2)  strong LL(2) LL(1) = strong LL(1)