Introduction to JJTree Cheng-Chia Chen

JJTree a preprocessor for JavaCC that inserts parse tree building actions at various places in the JavaCC source. By default, generates code to construct parse tree nodes for each nonterminal in the language. can be modified so that some nonterminals do not have nodes generated, or a node is generated for a part of a production's expansion. Defines a Java interface Node that all parse tree nodes must implement. The interface provides methods for getting/setting/adding parent/children nodes.

JJTree operates in one of two modes: simple and multi (for want of better terms). In simple mode: each parse tree node is of concrete type SimpleNode; in multi mode: the type of the parse tree node is derived from the name of the node. If you don't provide implementations for the node classes JJTree will generate sample implementations based on SimpleNode for you. You can then modify the implementations to suit.

How JJTree construct Parse Tree JJTree constructs the parse tree from the bottom up by using a stack to store nodes: Usage of the stack: pushes nodes after they have been created. When it finds a parent for them, it pops the children from the stack and adds them to the parent, and finally pushes the new parent node itself. The stack is open: you can push, pop and otherwise manipulate its contents however you feel appropriate.

Node Construction Decorations A definite node is constructed with a specific number of children: #ADefiniteNode( integer) Means that many nodes are popped from the stack and made the children of the newly created node, which then is pushed into the stack. Integer can be any integer expression.

Decorated by :#ConditionalNode(booleanExpr) A conditional node Decorated by :#ConditionalNode(booleanExpr) If booleanExpr is evaluated to ture: Construct a node containing as children all nodes that were pushed on the stack within its node scope. O/w the node is not constructed, and all of the children remain on the node stack. Special cases: Indefinite nodes: #IndefiniteNode is short for #IndefiniteNode(true) use #mynode(true) instead of #mynode to avoid ambiguity. Greater-than nodes: #GTNode(>1) is short for #GTNode(jjtree.arity() > 1)

By default JJTree treats each nonterminal as an indefinite node and derives the name of the node from the name of its production. You can give it a different name with the following syntax: void P1() #MyNode : { ... } { ... } When the parser recognizes a P1 nonterminal it begins an indefinite node. It marks the stack, so that any parse tree nodes created and pushed on the stack by nonterminals in the expansion for P1 will be popped off and made children of the node MyNode. Can use void P1() #void : { ... } { ... } to suppress the creation of a node for P1. Now any parse tree nodes pushed by nonterminals in the expansion of P1 will remain on the stack, to be popped and made children of a production further up the tree. You can make #void the default behavior for non-decorated nodes by using the NODE_DEFAULT_VOID option.

Examples void P3() : {} { P4() ( P5() )+ P6() } an indefinite node P3 is begun, marking the stack scope, and then a P4 node, one or more P5 nodes and a P6 node are parsed. Any nodes that they push are popped and made the children of P3. Customize the generated tree: void P3() : {} { P4() ( P5() )+ #ListOfP5s P6() } => P3 node have 1 p4 node, 1 ListOfP5s node and one P6 node. #NodeName acts as a postfix operator; its scope is the immediate preceding expansion unit.

Node Scopes and User Actions Each node has a node scope. The scope of a indefinite node in the rule body is the preceding expansion unit. The scope of an indefinite node for a production is the entire right hand side of the production including its declaration block. User actions within this scope can access the node under construction by using the special identifier jjtThis to refer to the node. This identifier is implicitly declared to be of the correct type for the node, so any fields and methods that the node has can be easily accessed.

Node Scopes and User Actions User actions other than the final one can only access the children on the stack. They have not yet been added to the node, so they aren't available via the node's methods. Codes within final user actions can only accessed the node children through method such as jjtGetChild() since all children have been popped and made children of the node. Can use nodeCreated() to determine if the node is created for conditional node.

Exception handling An exception thrown by an expansion within a node scope that is not caught within the node scope is caught by JJTree itself. When this occurs, any nodes that have been pushed on to the node stack within the node scope are popped and thrown away. Then the exception is rethrown. The intention is to make it possible for parsers to implement error recovery and continue with the node stack in a known state. WARNING: JJTree currently cannot detect whether exceptions are thrown from user actions within a node scope. Such an exception will probably be handled incorrectly.

Node Scope Hooks If the NODE_SCOPE_HOOK option is set to true, JJTree generates calls to two user-defined parser methods on the entry and exit of every node scope. The method signature: void jjtreeOpenNodeScope(Node n) void jjtreeCloseNodeScope(Node n) If the parser is STATIC then these methods will have to be declared as static as well. They are both called with the current node as a parameter.

Possible use of Hook methods Used to store the node's first and last tokens so that the input can be easily reproduced again. For example: void jjtreeOpenNodeScope(Node n) { ((MySimpleNode)n).first_token = getToken(1); } void jjtreeCloseNodeScope(Node n) { ((MySimpleNode)n).last_token = getToken(0); } where MySimpleNode is based on SimpleNode and has the following additional fields: Token first_token, last_token; Another use might be to store the parser object itself in the node so that state that should be shared by all nodes produced by that parser can be provided. For example, the parser might maintain a symbol table.

The Life Cycle of a Node the node's constructor is called with its unique integer id as a parameter. jjtThis.jjtOpen() is called if the option NODE_SCOPE_HOOK is set, parser.openNodeScope() is called if an unhandled exception is thrown while the node is being parsed then the node is abandoned. if the node is conditional and its conditional expression evaluates to false then the node is abandoned. All nodes pushed on t o the stack within the scope are popped and made children of the node jjtThis.jjtClose() is called jjtThis is push into the stack parser.closeNodeScope() is called If jjtThis not the root, it will eventually be poped from the stack and made a child of another node.

Visitor Support If the VISITOR option is set to true JJTree will insert an jjtAccept() method into all of the node classes it generates, and also generate a visitor interface that can be implemented and passed to the nodes to accept. The name of the visitor interface is <parser>Visitor(…) where parser is the name of the parser. The interface is regenerated every time that JJTree is run, so that it accurately represents the set of nodes used by the parser.

JJTree Options BUILD_NODE_FILES (true) : Generate sample implementations for SimpleNode and any other nodes used in the grammar. MULTI ( false) : Generate a multi mode parse tree. The default is generating a simple mode parse tree. NODE_DEFAULT_VOID ( false) Do not produce an infinite node for each non-decorated production. NODE_FACTORY ( false) Use a factory method static jjtCreate(int id) to construct nodes: NODE_PACKAGE ("") : The package to generate the node classes into. The default is the parser package. NODE_PREFIX ("AST") :Prefix used to construct node class names from node identifiers in multi mode.

NODE_SCOPE_HOOK (default: false) NODE_USES_PARSER (default: false) JJTree will use an alternate form of the node construction routines where it passes the parser object in. For example, public static Node MyNode.jjtCreate(MyParser p, int id); MyNode(MyParser p, int id); STATIC (default: true) VISITOR (default: false) Insert a jjtAccept() method in the node classes, and generate a visitor implementation with an entry for every node type used in the grammar. VISITOR_EXCEPTION (default: "") If this option is set, it is used in the signature of the generated jjtAccept() methods and the visit() methods. Note: this option will be removed in a later version of JJTree. Don't use it if that bothers you.

JJTree state JJTree keeps its state in a parser class field called jjtree with the type JJTreeState: final class JJTreeState { void reset(); // reinitialize the node stack. Node rootNode(); //* Return the root node of the AST. boolean nodeCreated(); //* Determine whether the current node was actually closed and pushed */ int arity(); //* Return the number of nodes currently pushed on the node stack in the current node scope. */ void pushNode(Node n); //* Push a node on to the stack. */ Node popNode(); //* Return the node on the top of the stack, and remove it from the stack. */ Node peekNode(); ///* Return the node currently on the top of the stack. */}

Node Objects public interface Node { /** This method is called after the node has been made the current node. It indicates that child nodes can now be added to it. */ public void jjtOpen(); public void jjtClose(); public void jjtSetParent(Node n); public Node jjtGetParent(); public void jjtAddChild(Node n, int i); public Node jjtGetChild(int i); int jjtGetNumChildren(); }

SmipleNode implements Node automatically generated by JJTree if it doesn't already exist. provides a method for recursively dumping the node and its children. You might use this is in action like this: { ((SimpleNode)jjtree.rootNode()).dump(">"); } “>” is used as padding to indicate the tree hierarchy. Another utility method is generated if the VISITOR options is set: { public void childrenAccept(MyParserVisitor visitor); } This walks over the node's children in turn, asking them to accept the visitor. This can be useful when implementing preorder and postorder traversals.