Programming Language Syntax 5 http://flic.kr/p/zCyMp

What’s a context-free grammar (CFG)? What distinguishes it from other grammars? What is a context-free language? What is a parser?

What’s a context-free grammar (CFG)? Rules for how to generate sentences in some lang. What distinguishes it from other grammars? The types of rules that can be expressed What is a context-free language? A lang. that can be generated with a CFG What is a parser? A program that recognizes sentences in a lang.

What’s does it mean to derive a string from a CFG? What is a parse tree? What is a left-most derivation? What is a right-most derivation?

CFG: String to derive: Derivation:

Parse Tree: Graphical Representation of Derivation

Right-most versus Left-most derivation

What’s does it mean to derive a string from a CFG? Iteratively expand CFG rules to yield string What is a parse tree? Graphical representation of a derivation Also, an output of a parser What is a left-most derivation? Expand left-most non-terminal first What is a right-most derivation? Expand right-most non-terminal first

What is an LL parser? What is an LR parser? What is an LL grammar? LR grammar? Why do we care about LL/LR parsers?

What is an LL parser? Parser that reads input left-to-right and produces left-most derivation What is an LR parser? Same as LL parser, except produces right-most derivation What is an LL grammar? LR grammar? Grammar that can be parsed by an LL/LR parser Why do we care about LL/LR parsers? More efficient implementations – O(n) vs O(n3)

Let’s delve deeper into LL parsers (ANTLR is one) Recall this example…

Start with the start rule (id_list) id(A) , id(B) , id(C) ; Start with the start rule (id_list)

Predict replacement for id_list id(A) , id(B) , id(C) ; Predict replacement for id_list

id(A) , id(B) , id(C) ; ? Only one choice

Read token and match terminal What if terminal doesn’t match? id(A) , id(B) , id(C) ; Read token and match terminal What if terminal doesn’t match?

Predict replacement for id_list_tail id(A) , id(B) , id(C) ; Predict replacement for id_list_tail

id(A) , id(B) , id(C) ; Can’t decide? Peek!

id(A) , id(B) , id(C) ; ?

Read token, match terminal id(A) , id(B) , id(C) ; ? Read token, match terminal

Read token, match terminal id(A) , id(B) , id(C) ; Read token, match terminal

Predict replacement for id_list_tail id(A) , id(B) , id(C) ; Predict replacement for id_list_tail

id(A) , id(B) , id(C) ; Can’t decide? Peek!

id(A) , id(B) , id(C) ; ?

Read token, match terminal id(A) , id(B) , id(C) ; ? Read token, match terminal

Read token, match terminal id(A) , id(B) , id(C) ; Read token, match terminal

Predict replacement for id_list_tail id(A) , id(B) , id(C) ; Predict replacement for id_list_tail

id(A) , id(B) , id(C) ; Can’t decide? Peek!

id(A) , id(B) , id(C) ;

Read token, match terminal id(A) , id(B) , id(C) ; Read token, match terminal

Done! No more non-terminals. No more input. id(A) , id(B) , id(C) ; Done! No more non-terminals. No more input.

Recall that peeking may be restricted 0 tokens of look-ahead (predict without peeking) LL(1) Up to 1 token of look-ahead (like the last example) LL(2) Up to 2 tokens of look-ahead LL(*) Unlimited tokens of look-ahead (ANTLR 3)

If you’re writing a grammar from which to generate an LL(1) parser, you must have an LL(1) grammar Otherwise, you get conflict errors…

Consider what happens if an LL(1) parser tries to parse this non-LL(1) grammar

Can we predict the next replacement? Yes! id_list_prefix ; id(A) , id(B) , id(C) ; id_list Can we predict the next replacement?

Can we predict the next replacement? id(A) , id(B) , id(C) ; id_list id_list_prefix ; Can we predict the next replacement? Nope. Let’s peek.

Can we predict the next replacement? id(A) , id(B) , id(C) ; id_list id_list_prefix ; Can we predict the next replacement? Hm. Still can’t. The problem is the left recursion in id_list_prefix—both its alternatives start with id!

But how do we know if our grammar is an LL(1) grammar But how do we know if our grammar is an LL(1) grammar? To find out, let’s look in more detail at how to implement a parser

Consider this LL(1) grammar

Here is a recursive descent parser that recognizes the grammar *See handout

Observe the pattern… One procedure for each production … …

Each procedure has a case stmt that considers the next input token Observe the pattern… Each procedure has a case stmt that considers the next input token … …

Observe the pattern… One case per right-hand side (+error) … …

Each case tries to predict the right-hand side to expand Observe the pattern… Each case tries to predict the right-hand side to expand … …

Observe the pattern… … … Based on prediction: Call appropriate procedure for non-terminals Call match() for terminals … …

Activity: Execute recursive-descent parser on each of these inputs Create a parse tree as you go Each time a procedure is called, hang a node off the calling procedure Be sure to keep track of the call stack

Solution

Next time: How to predict … …