Compilers: Parse Tree/9 1 Compiler Structures Objective – –extend the expressions language compiler to generate a parse tree for the input program, and then evaluate it , Semester 1, Creating and Evaluating a Parse Tree

Compilers: Parse Tree/9 2 Overview 1. The Expressions Grammar 2. exprParse2.c 3. Parse Tree Data Structures 4. Revised Parse Functions 5.Tree Building 6.Tree Printing 7.Tree Evaluation

Compilers: Parse Tree/9 3 In this lecture Source Program Target Lang. Prog. Semantic Analyzer Syntax Analyzer Lexical Analyzer Front End Code Optimizer Target Code Generator Back End Int. Code Generator Intermediate Code concentrating on parse tree generation and evaluation

Compilers: Parse Tree/ The Expressions Grammar It's LL(1) grammar: Stats => ( [ Stat ] \n )* Stat => let ID = Expr | Expr Expr => Term ( (+ | - ) Term )* Term => Fact ( (* | / ) Fact ) * Fact => '(' Expr ')' | Int | Id

Compilers: Parse Tree/9 5 An Expressions Program (test3.txt) let x = ( (x*y)/2) // comments // y let x = 5 let y = x /0 // comments

Compilers: Parse Tree/ exprParse2.c A recursive descent parser using the expressions language. This version of the parser differs from exprParse1.c by having the parse functions (e.g. statements(), statement()) create a parse tree as they execute. continued

Compilers: Parse Tree/9 7 There's a new printTree() function which prints the final tree, and evalTree() which evaluates it. Usage: $ gcc -Wall -o exprParse2 exprParse2.c $./exprParse2 < test1.txt

Compilers: Parse Tree/9 8 Output for test1.txt > exprParse2 < test1.txt \n NULL = x 2 = y + 3 x continued printed tree; same as \n NULL = x2 = y + 3x let x = 2 let y = 3 + x

Compilers: Parse Tree/9 9 x being declared x = 2 == 2 y being declared y = 5 == 5 > evaluation of the parse tree

Compilers: Parse Tree/ Parse Tree Data Structures typedef struct TreeNode { Token operTok; union { char *id; int value; struct {struct TreeNode *left, *right;} branches; } u; } Tree; A tree is made from TreeNodes.

Compilers: Parse Tree/9 11 Graphically operTok one of ID, INT, NEWLINE, ASSIGNOP, PLUSOP, MINUSOP, MULTOP, DIVOP id variable name (for ID) OR value integer (for INT) OR branches children pointers of this node (used by NEWLINE, ASSIGNOP, PLUSOP, MINUSOP, MULTOP, DIVOP) TreeNode a union, u leftright

Compilers: Parse Tree/9 12 Macros for Using TreeNode Fields #define TreeOper(t) ((t)->operTok) #define TreeID(t) ((t)->u.id) #define TreeValue(t) ((t)->u.value) #define TreeLeft(t) ((t)->u.branches.left) #define TreeRight(t) ((t)->u.branches.right)

Compilers: Parse Tree/ Revised Parse Functions The parse functions have the same 'shape' as the ones in exprParse0.c, but now call tree building functions, and return a Tree result. Functions: – –main(), statements(), statement(), expression(), term(), factor()

Compilers: Parse Tree/9 14 int main(void) { nextToken(); statements(); match(SCANEOF); return 0; } int main(void) // parse, then print and evaluate the resulting tree { Tree *t; nextToken(); t = statements(); match(SCANEOF); printTree(t, 0); printf("\n\n"); evalTree(t); return 0; } main() Before and After

Compilers: Parse Tree/9 15 statements() Before and After void statements(void) // statements ::= { [ statement] '\n' } { dprint("Parsing statements\n"); while (currToken != SCANEOF) { if (currToken != NEWLINE) statement(); match(NEWLINE); } } // end of statements() with no semantic actions

Compilers: Parse Tree/9 16 Tree *statements(void) { Tree *t, *left, *statTree; left = NULL; dprint("Parsing statements\n"); while (currToken != SCANEOF) { if (currToken != NEWLINE) statTree = statement(); else statTree = NULL; match(NEWLINE); if (statTree != NULL) { t = makeTreeNode(NEWLINE, left, statTree); left = t; } return left; } // end of statements()

Compilers: Parse Tree/9 17 Tree Structure for statements A statements sequence: s1 \n 1 s2 \n 2 s3 \n 3 becomes: \n 3 \n 2 \n 1 NULL s1 s2 s3

Compilers: Parse Tree/9 18 statement() Before and After void statement(void) // statement ::= ( 'let' ID '=' EXPR ) | EXPR { if (currToken == LET) { match(LET); match(ID); match(ASSIGNOP); expression(); } else expression(); } // end of statement() with no semantic actions

Compilers: Parse Tree/9 19 Tree *statement(void) { Tree *t, *idTree, *exprTree; dprint("Parsing statement\n"); if (currToken == LET) { match(LET); idTree = matchId(); // build tree node, not symbol table entry match(ASSIGNOP); exprTree = expression(); t = makeTreeNode(ASSIGNOP, idTree, exprTree); } else // expression t = expression(); return t; } // end of statement()

Compilers: Parse Tree/9 20 Tree Structures for statement = ID node expr tree or expr tree

Compilers: Parse Tree/9 21 expression() Before and After void expression(void) // expression ::= term ( ('+'|'-') term )* { term(); while((currToken == PLUSOP) || (currToken == MINUSOP)) { match(currToken); term(); } } // end of expression() with no semantic actions

Compilers: Parse Tree/9 22 Tree *expression(void) { Tree *t, *left, *right; int isAddOp; dprint("Parsing expression\n"); left = term(); while((currToken == PLUSOP)||(currToken == MINUSOP)) { isAddOp = (currToken == PLUSOP) ? 1 : 0; nextToken(); right = term(); if (isAddOp == 1) // addition t = makeTreeNode(PLUSOP, left, right); else // subtraction t = makeTreeNode(MINUSOP, left, right); left = t; } return left; } // end of expression()

Compilers: Parse Tree/9 23 Tree Structure for expression An expression sequence: t1 + 1 t2 - t3 + 2 t4 becomes: t1 t2 t3 t4

Compilers: Parse Tree/9 24 term() Before and After void term(void) // term ::= factor ( ('*'|'/') factor )* { factor(); while((currToken == MULTOP) || (currToken == DIVOP)) { match(currToken); factor(); } } // end of term() with no semantic actions

Compilers: Parse Tree/9 25 Tree *term(void) { Tree *t, *left, *right; int isMultOp; dprint("Parsing term\n"); left = factor(); while((currToken == MULTOP) || (currToken == DIVOP)) { isMultOp = (currToken == MULTOP) ? 1 : 0; nextToken(); right = factor(); if (isMultOp == 1) // multiplication t = makeTreeNode(MULTOP, left, right); else // division t = makeTreeNode(DIVOP, left, right); left = t; } return left; } // end of term()

Compilers: Parse Tree/9 26 Tree Structure for term An term sequence: f1 * 1 f2 / f3 * 2 f4 becomes: *2*2 / *1*1 f1 f2 f3 f4

Compilers: Parse Tree/9 27 factor() Before and After void factor(void) // factor ::= '(' expression ')' | INT | ID { if(currToken == LPAREN) { match(LPAREN); expression(); match(RPAREN); } else if(currToken == INT) match(INT); else if (currToken == ID) match(ID); else syntax_error(currToken); } // end of factor() with no semantic actions

Compilers: Parse Tree/9 28 Tree *factor(void) { Tree *t = NULL; dprint("Parsing factor\n"); if(currToken == LPAREN) { match(LPAREN); t = expression(); match(RPAREN); } else if(currToken == INT) { t = makeIntLeaf(currTokValue); match(INT); } else if (currToken == ID) t = matchId(); // do not access symbol table else syntax_error(currToken); return t; } // end of factor()

Compilers: Parse Tree/9 29 Match an ID (Extended) Tree *matchId(void) { Tree *t; if (currToken == ID) t = makeIDLeaf(tokString); match(ID); return t; } // end of matchID()

Compilers: Parse Tree/9 30 Tree Structure for factor There are three possible nodes: ID node or INT node or tree node

Compilers: Parse Tree/ Tree Building The nodes in a parse tree are connected by the parse functions. A tree node can have three different shapes: operTok id OR value OR branches TreeNode a union leftright

Compilers: Parse Tree/9 32 Making a Tree Node Tree *treeMalloc(void) // a tree node with no fields specified { Tree *t; t = (Tree *) malloc( sizeof(Tree) ); if(t == NULL) {/* out of memory? */ perror("Tree Node not made; out of memory"); exit(1); } return t; } // end of treeMalloc()

Compilers: Parse Tree/9 33 Making an ID Node Tree *makeIDLeaf(char *id) { Tree *t; t = treeMalloc(); TreeOper(t) = ID; TreeID(t) = (char *) malloc(strlen(id)+1); strcpy(TreeID(t), id); return t; } // end of makeIDLeaf() no symbol table entry created yet ID operTok "id str" id

Compilers: Parse Tree/9 34 Making an INT Node Tree *makeIntLeaf(int value) { Tree *t; t = treeMalloc(); TreeOper(t) = INT; TreeValue(t) = value; return t; } // end of makeIntLeaf() INT operTok integer value

Compilers: Parse Tree/9 35 Making a Node with Children Tree *makeTreeNode(Token op, Tree *left, Tree *right) /* Build an internal tree node, which contains an operator and points to two subtrees.*/ { Tree *t; t = treeMalloc(); TreeOper(t) = op; TreeLeft(t) = left; TreeRight(t) = right; return t; } // end of makeTreeNode() op operTok branches left right

Compilers: Parse Tree/ Tree Printing The printTree() function recurses over the tree, and does three different things depending on the three possible 'shapes' for a tree node. It includes an indent counter, which is used to print spaces (indents) in front of the node information.

Compilers: Parse Tree/9 37 void printTree(Tree *t, int indent) // print a tree, indenting by indent spaces { printIndent(indent); if (t == NULL) { printf("NULL\n"); return; } : continued

Compilers: Parse Tree/9 38 Token tok = TreeOper(t); if (tok == INT) printf("%d\n", TreeValue(t)); else if (tok == ID) printf("%s\n", TreeID(t)); else { // operator if (tok == NEWLINE) printf("\\n\n"); // show the \n else printf("%s\n", tokSyms[tok]); printTree(TreeLeft(t), indent+2); printTree(TreeRight(t), indent+2); } } // end of printTree()

Compilers: Parse Tree/9 39 void printIndent(int n) { int spaces; for(spaces = 0; spaces != n; spaces++) putchar(' '); } // end of printIndent()

Compilers: Parse Tree/9 40 Tree Printing Examples > exprParse2 < test2.txt \n NULL = x56 2 = bing_BONG - * 27 2 x56 * 5 / 67 3 let x56 = 2 let bing_BONG = (27 * 2) - x56 5 * (67 / 3)

Compilers: Parse Tree/9 41 Graphically \n NULL = x562 = bing_ BONG - * 272 x56 * / S1 S2 S3

Compilers: Parse Tree/9 42 test3.txt let x = ( (x*y)/2) // comments // y let x = 5 let y = x /0 // comments

Compilers: Parse Tree/9 43 > exprParse2 < test3.txt \n NULL = x / * x y 2 = x 5 = y / x 0

Compilers: Parse Tree/ Tree Evaluation Tree evaluation works in two stages: – –evalTree() searches over the tree looking for subtrees which start with an operator which is not NEWLINE – –these subtrees are evaluated by eval(), using the operators in their nodes

Compilers: Parse Tree/9 45 Finding non-NEWLINEs \n NULL = x562 = bing_ BONG - * 272 x56 * / evalTree() used here eval() used here

Compilers: Parse Tree/9 46 Code void evalTree(Tree *t) { if (t == NULL) return; Token tok = TreeOper(t); if (tok == NEWLINE) { evalTree( TreeLeft(t) ); evalTree( TreeRight(t) ); } else printf("== %d\n", eval(t)); } // end of evalTree()

Compilers: Parse Tree/9 47 int eval(Tree *t) { SymbolInfo *si; if (t == NULL) return 0; Token tok = TreeOper(t); if (tok == ID) { si = getIDEntry( TreeID(t) ); // lookup ID in symbol table return si->value; } : The operator can be one of ID, INT, ASSIGNOP, PLUSOP, MINUSOP, MULTOP, DIVOP continued 7 possibilities

Compilers: Parse Tree/9 48 else if (tok == INT) return TreeValue(t); else if (tok == ASSIGNOP) { // id = expr si = evalID(TreeLeft(t)); //add ID to sym. table int result = eval(TreeRight(t)); si->value = result; printf("%s = %d\n", si->id, result); return result; } else if (tok == PLUSOP) return eval(TreeLeft(t)) + eval(TreeRight(t)); else if (tok == MINUSOP) return eval(TreeLeft(t)) - eval(TreeRight(t)); :

Compilers: Parse Tree/9 49 else if (tok == MULTOP) return eval(TreeLeft(t)) * eval(TreeRight(t)); else if (tok == DIVOP) { int right = eval(TreeRight(t)); if (right == 0) { printf("Error: Div by 0; using 1 instead\n"); return eval(TreeLeft(t)); } else return eval(TreeLeft(t)) / right; } return 0; // shouldn't reach here } // end of eval()

Compilers: Parse Tree/9 50 SymbolInfo *evalID(Tree *t) { char *id = TreeID(t); return getIDEntry(id); // create sym. table entry for id } // end of evalID() this function finds or creates a symbol table entry for the id, and return a pointer to the entry (same as in exprParse1.c)

Compilers: Parse Tree/9 51 Evaluation Examples $./exprParse2 < test1.txt : x declared x = 2 == 2 y declared y = 5 == 5 let x = 2 let y = 3 + x

Compilers: Parse Tree/9 52 $./exprParse2 < test2.txt : x56 declared x56 = 2 == 2 bing_BONG declared bing_BONG = 52 == 52 == 110 // test2.txt example let x56 = 2 let bing_BONG = (27 * 2) - x56 5 * (67 / 3)

Compilers: Parse Tree/9 53 $./exprParse2 < test3.txt : == 11 x declared x = 2 == 2 y declared == 3 x = 5 == 5 Error: Division by zero; using 1 instead y = 5 == let x = ( (x*y)/2) // comments // y let x = 5 let y = x /0