5-Jan-16 Recursive descent parsing

Some notes on recursive descent The starter code that I gave you did not exactly fit the grammar that I gave you Both work, both are correct Many things can be coded either recursively or iteratively I gave you an iterative grammar and recursive code

Recursive rule for ::= public boolean term() { if (!factor()) return false; if (!multiplyOperator()) return true; if (!term()) error("No term after '*' or '/' "); return true; }

Iterative rule for ::= { } public boolean term() { if (!factor()) return false; while (multiplyOperator()) { if (!factor()) error("No factor after '*' or '/' "); } return true; }

Parse trees Every construct (statement, expression, etc.) in a programming language can be represented as a parse tree

Recognizers and parsers A recognizer tells whether a given string “belongs to” (is correctly described by) a grammar A parser uses a grammar to construct a parse tree from a given string One kind of parser is a recursive descent parser Recursive descent parsers have some disadvantages: They are not as fast as some other methods It is difficult to provide really good error messages They cannot do parses that require arbitrarily long lookaheads And some advantages: They are exceptionally simple They can be constructed from recognizers simply by doing some extra work—specifically, building a parse tree Recursive descent parsers are great for “quick and dirty” parsing jobs

The Stack One easy way to do recursive descent parsing is to have each parse method take the tokens it needs, build a parse tree, and put the parse tree on a global stack Write a parse method for each nonterminal in the grammar Each parse method should get the tokens it needs, and only those tokens Those tokens (usually) go on the stack Each parse method may call other parse methods, and expect those methods to leave their results on the stack Each (successful) parse method should leave one result on the stack

Example: while statement ::= “while” The parse method for a while statement: Calls the Tokenizer, which returns a “while” token Makes the “while” into a tree node, which it puts on the stack Calls the parser for, which parses a condition and puts it on the stack Stack now contains: “while” (“top” is on right) Calls the parser for, which parses a block and puts it on the stack Stack now contains: “while” Pops the top three things from the stack, assembles them into a tree, and pushes this tree onto the stack

Recognizing a while statement // ::= “while” private boolean whileStatement() { if (keyword("while")) { if (condition()) { if (block()) { return true; } else error("Missing '{' "); } else error("Missing "); } return false; } Why do you suppose I named this method whileStatement() instead of while() ?

Parsing a while statement // ::= “while” private boolean whileStatement() { if (keyword("while")) { if (condition()) { if (block()) { makeTree(3, 2, 1); return true; } else error("Missing '{' "); } else error("Missing "); } return false; } This code assumes that keyword(String), condition(), and block() all leave their results on a stack On the stack, while = 3, = 2, = 1

Alternative code public boolean whileStatement () { if (keyword("while") && condition() && block()) { makeTree(3, 2, 1); return true; } return false; } No room for an error condition in this code

Alternative code with one message public boolean whileStatement() { if (keyword("while")) { if (condition()) && (block()) { makeTree(3, 2, 1); return true; } error("Error in \"while\" statement"); } return false; }

Simple makeTree() method After recognizing a while loop, the stack looks like this: And I could have written code like this: private void makeTree() { Tree right = stack.pop(); Tree left = stack.pop(); Tree root = stack.pop(); root.addChild(left); root.addChild(right); stack.push(root); } while This code assumes that the root is the third item down, etc., and that isn’t always the case I found it more convenient to write more flexible methods while

More general makeTree method private void makeTree(int keyword, int left, int right) { Tree root = getStackItem(keyword); Tree leftChild = getStackItem(left); Tree rightChild = getStackItem(right); stack.pop(); stack.pop(); stack.pop(); root.addChild(leftChild); root.addChild(rightChild); stack.push(root); }

Parser methods In the BNF, I have one long definition for ::= | "turn" | "take" | "drop"... In my code, I broke that into multiple methods ::= | | |...

My command() method public boolean command() { if (move()) return true; if (turn()) return true; if (take()) return true; if (drop()) return true;... return false; }

Helper methods I wrote a number of helper methods for the Parser and for the ParserTest classes It’s helpful to have methods to build trees quickly and easily private makeTree(String op) private Tree makeTree(String op, Tree left, Tree right) Java 5 allows a variable number of arguments, so you could write private Tree makeTree(String op, Tree... children) Another useful method is assertStackTop, which is just private void assertStackTop(Tree expected) { assertEquals(expected, parser.stack.peek()); } Example: Tree condition = makeTree("=", "2", "2"); assertStackTop(makeTree("if", condition, "list"));

Final comments You are welcome to use any of this code, but......my code is never the last word on anything! Code can always be improved While I think my code is generally useful, you always have to understand it well enough to adapt it to your particular needs

The End