1. 241-437 Compilers: Attr. Grammars/8 1 Compiler Structures Objective – –describe semantic analysis with attribute grammars, as applied in yacc and recursive descent parsers 241-437, Semester 1, 2011-2012 8. Attribute Grammars

2. 241-437 Compilers: Attr. Grammars/8 2 Overview 1. What is an Attribute Grammar? 2. Parse Tree Evaluation 3. Attributes 4. Attribute Grammars and yacc 5. A Grid Grammar 6.Recursive Descent and Attributes

3. 241-437 Compilers: Attr. Grammars/8 3 In this lecture Source Program Target Lang. Prog. Semantic Analyzer Syntax Analyzer Lexical Analyzer Front End Code Optimizer Target Code Generator Back End Int. Code Generator Intermediate Code concentrating on attribute grammars

4. 241-437 Compilers: Attr. Grammars/8 4 1. What is an Attribute Grammar? An attribute grammar is a context free grammar with semantic actions attached to some of the productions – –semantic = meaning An action specifies the meaning of a production in terms of its body terminals and nonterminals.

5. 241-437 Compilers: Attr. Grammars/8 5 Example Attribute Grammar L  E E  E + T E  T T  T * F T  F F  ( E ) F  num printf(E body.val) E.val := E body.val + T body.val E.val := T body.val T.val := T body.val * F body.val T.val := F body.val F.val := E body.val F.val := value(num) ProductionSemantic Action

6. 241-437 Compilers: Attr. Grammars/8 6 2. Parse Tree Evaluation One way of understanding semantic actions is as extra information (attributes) attached to the nodes of the parse tree for the input. The semantic action specifies the parent node attribute in terms of the attributes of its children.

7. 241-437 Compilers: Attr. Grammars/8 7 Basic Parse Tree Input: 9 * 5 + 2 L  E E  E + T E  T T  T * F T  F F  ( E ) F  num L E T E+ * T F 9 F 5 F 2 T

8. 241-437 Compilers: Attr. Grammars/8 8 Adding Meaning to the Tree What is the meaning of "9 * 5 + 2"? – –the answer is to evaluate it, to get 47 Add attributes to the tree, starting from the leaves and working up to the root – –use the semantic actions to get the attribute values

9. 241-437 Compilers: Attr. Grammars/8 9 Parse Tree with Actions L E T E+ * T F 9 F 5 F 2 T printf(E body.val) E.val := E body.val + T body.val E.val := T body.val T.val := T body.val * F body.val T.val := F body.val F.val := E body.val F.val := value(num) 9 9 45 47 printf 2 2 evaluate bottom-up 5

10. 241-437 Compilers: Attr. Grammars/8 10 3. Attributes Attribute values can be – –numbers, strings, any data structures, code, assembly language instructions It's not always necessary to build a parse tree in order to evaluate the grammar's action.

11. 241-437 Compilers: Attr. Grammars/8 11 Kinds of Attribute There are two main kinds of attribute evaluation: – –synthesized and inherited attributes The value of a synthesized attribute is calculated by using its body values – –as in the previous example

12. 241-437 Compilers: Attr. Grammars/8 12 Synthesized Attributes in a Tree Example: ProductionSemantic Action T  T * FT.val := T body.val * F body.val * T F T 9 45 5 evaluate bottom-up

13. 241-437 Compilers: Attr. Grammars/8 13 Inherited Attributes An inherited attribute for a body symbol (i.e. terminal, non-terminal) gets its value from the other body symbols and the parent value – –often used for evaluating more complex programming language features

14. 241-437 Compilers: Attr. Grammars/8 14 Inherited Attributes in a Tree X.x := function(A.a, Y.y) Y.y := function(A.a, X.x) A.a X.xY.y A.a X.xY.y Direction of evaluation Two examples:

15. 241-437 Compilers: Attr. Grammars/8 15 4. Attribute Grammars and yacc yacc supports (synthesized) attribute grammars – –yacc actions are semantic actions – –no parse tree is needed, since yacc evaluates the actions using the parser's built-in stack

16. 241-437 Compilers: Attr. Grammars/8 16 expr.y Again %token NUMBER % exprs: expr '\n' { printf("Value = %d\n", $1); } | exprs expr '\n' { printf("Value = %d\n", $2); } ; expr: expr '+' term { $$ = $1 + $3; } | expr '-' term { $$ = $1 - $3; } | term { $$ = $1; } ; continued declarations actions attributes

17. 241-437 Compilers: Attr. Grammars/8 17 term: term '*' factor { $$ = $1 * $3; } | term '/' factor{ $$ = $1 / $3; } /* integer division */ | factor ; factor: '(' expr ')' { $$ = $2; } | NUMBER ; continued more actions

18. 241-437 Compilers: Attr. Grammars/8 18 $$ #include "lex.yy.c" int yyerror(char *s) { fprintf(stderr, "%s\n", s); return 0; } int main(void) { yyparse(); // the syntax analyzer return 0; } c code

19. 241-437 Compilers: Attr. Grammars/8 19 Evaluation in yacc Stack $ $ 3 $ F $ T $ T * $ T * 5 $ T * F $ T $ E $ E + $ E + 4 $ E + F $ E + T $ E $ E \n $ Es Input 3*5+4\n$ *5+4\n$ *5+4\n$ *5+4\n$ 5+4\n$ +4\n$ +4\n$ +4\n$ +4\n$ 4\n$ \n$ \n$ \n$ \n$ $ $ Stack Action shift reduce F  num reduce T  F shift shift reduce F  num reduce T  T * F reduce E  T shift shift reduce F  num reduce T  F reduce E  E + T shift reduce Es  E \n accept val _ 3 3 3 3 3 5 3 5 15 15 15 15 4 15 4 15 4 19 19 19 Semantic Action $$ = $1 (implicit) $$ = $1 (implicit) $$ = $1 (implicit) $$ = $1 * $3 $$ = $1 (implicit) $$ = $1 (implicit) $$ = $1 (implicit) $$ = $1 + $3 printf $1 Input: 3 * 5 + 4\n

20. 241-437 Compilers: Attr. Grammars/8 20 5. A Grid Grammar A robot starts at (0,0) on a grid, and is given compass directions: – –n = north, s = south, e = east, w = west Evaluate the sequence of directions to work out the final position of the robot.

21. 241-437 Compilers: Attr. Grammars/8 21 Example The robot receives the directions: – –n e e n n w – –what is the 'meaning' (semantics) of the directions? – –the 'meaning' is the final robot position, (1,3) start final n e w s

22. 241-437 Compilers: Attr. Grammars/8 22 5.1. Grid Grammar Input: n w s s robot  path path  path step | e step  e | w | s | n robot path step s path step s path step w path step n e

23. 241-437 Compilers: Attr. Grammars/8 23 Grid Attribute Grammar robot  path path  path step path  e step  e step  w step  s step  n printf( path body.(x,y) ) path.x := path body.x + step body.dx path.y := path body.y + step body.dy path.(x,y) = (0,0) step.(dx,dy) := (1,0) step.(dx,dy) := (-1,0) step.(dx,dy) := (0,-1) step.(dx,dy) := (0,1) ProductionSemantic Actions

24. 241-437 Compilers: Attr. Grammars/8 24 Data Types The path rules use (x,y), the position of the robot. The step rules use (dx,dy), the step taken by the robot. Implementing these data types requires new features of yacc. (x,y) dx,dy

25. 241-437 Compilers: Attr. Grammars/8 25 Parse Tree with Actions Input: n w s s robot path step s path step s path step w path step n e (0,0) (0,1) (-1,1) (-1,0) (-1,-1) 0,1 -1,0 0,-1 printf (-1,-1) evaluate bottom-up

26. 241-437 Compilers: Attr. Grammars/8 26 5.2. Non-integer Yacc Attributes The default yacc attributes (e.g. $$, $1, etc) are integers. We want data structures for (x,y) and (dx,dy), coded as two struct types.

27. 241-437 Compilers: Attr. Grammars/8 27 Defining New Types The new types are collected together inside a %union in the yacc definitions section: %union{ type1 name1; type2 name2;... } For the grid: %union{ struct (int x, int y; } pos; struct (int dx, int dy; } offset; }

28. 241-437 Compilers: Attr. Grammars/8 28 The non-terminals that return the new types must be listed. Any tokens that use the types must be listed. For the grid: % type step % type path Using the Types these non-terminals return values of the specified type

29. 241-437 Compilers: Attr. Grammars/8 29 Using Typed Variables If an attribute (variable) is a record, then dotted-name notation is used to refer to its fields – –e.g. $$.dx, $1.y The default action ($$ = $1) will cause an error if $$ and $1 are not the same type.

30. 241-437 Compilers: Attr. Grammars/8 30 5.3. Grid Compiler $ flex grid.l $ bison grid.y $ gcc grid.tab.c -o gridEval grid.l, a flex file grid.y, a bison file bison flexlex.yy.c grid.tab.c gcc gridEval, c executable #include

31. 241-437 Compilers: Attr. Grammars/8 31 Usage $./gridEval nwss Robot is at (-1,-1) $./gridEval n n n w w w s e Robot is at (-2,2) $ I typed these lines. I typed ctrl-D

32. 241-437 Compilers: Attr. Grammars/8 32 grid.l % [nN]{return NORTH;} [sS]{return SOUTH;} [eE] {return EAST;} [wW]{return WEST;} [ \n\t]; % int yywrap(void) { return 1; }

33. 241-437 Compilers: Attr. Grammars/8 33 grid.y %union{ struct { int x; int y; } pos; struct { int dx; int dy; } offset; } %token EAST WEST NORTH SOUTH %type step %type path % continued type definitions types use by the non-terminals

34. 241-437 Compilers: Attr. Grammars/8 34 robot: path { printf("Robot is at (%d,%d)\n", $1.x, $1.y); } ; path: path step {$$.x = $1.x + $2.dx; $$.y = $1.y + $2.dy;} | {$$.x = 0; $$.y = 0;} ; step: EAST {$$.dx = 1; $$.dy = 0;} | WEST {$$.dx = -1; $$.dy = 0;} | SOUTH {$$.dx = 0; $$.dy = -1;} | NORTH {$$.dx = 0; $$.dy = 1;} ; % continued

35. 241-437 Compilers: Attr. Grammars/8 35 #include "lex.yy.c" int yyerror(char *s) { fprintf(stderr, "%s\n", s); return 0; } int main(void) { yyparse(); return 0; }

36. 241-437 Compilers: Attr. Grammars/8 36 6. Recursive Descent and Attributes It is easy to add semantic actions to a recursive descent parser – –in many cases, there's no need for the parser to build a parse tree in order to evaluate the attributes The basic translation strategy: – –each production becomes a function continued

37. 241-437 Compilers: Attr. Grammars/8 37 The function (e.g. f()) calls other functions representing its body non-terminals – –those functions return values (attributes) to f() – –f() combines the values, and returns a value (attribute)

38. 241-437 Compilers: Attr. Grammars/8 38 6.1. The Expressions Parser Again The basic LL(1) grammar: Stats => ( [ Stat ] \n )* Stat => let ID = Expr | Expr Expr => Term ( (+ | - ) Term )* Term => Fact ( (* | / ) Fact ) * Fact => '(' Expr ')' | Int | Id

39. 241-437 Compilers: Attr. Grammars/8 39 An Expressions Program (test3.txt) 5 + 6  give answer let x = 2  declare variable 3 + ( (x*y)/2) // comments // y let x = 5 let y = x /0  error // comments

40. 241-437 Compilers: Attr. Grammars/8 40 exprParse1.c is a recursive descent parser using the expressions language. It differs from exprParse0.c by having semantic actions attached to its productions – –these actions evaluate the expressions, and assign values to expression variables 6.2. Parsing with Actions

41. 241-437 Compilers: Attr. Grammars/8 41 Grammar with Actions ProductionsActions Stats => ( [ Stat ] \n )* --- Stat => let ID = Expr add id to symbol table; id.val = expr.val; print( id.val ); Stat => Exprprint( expr.val ); continued

42. 241-437 Compilers: Attr. Grammars/8 42 Expr => Term ( (+ | - ) Term )* return term 1.val (+| -) term 2.val (+| -)... term n.val; Term => Fact ( (* | / ) Fact ) * return fact 1.val (*| /) fact 2.val (*| /)... fact n.val; continued

43. 241-437 Compilers: Attr. Grammars/8 43 Fact => '(' Expr ')return expr.val; Fact => Int return int.val; Fact => Idlookup id; if not found then add (id, 0) to table; return id.val;

44. 241-437 Compilers: Attr. Grammars/8 44 The Symbol Table The symbol table is a data structure used to store expression variables and their values. In exprParse1.c, it's an array of structs, with each struct holding the name of the variable and its current integer value... id value syms[]

45. 241-437 Compilers: Attr. Grammars/8 45 6.3. Usage $ gcc -Wall -o exprParse1 exprParse1.c $./exprParse1 < test3.txt == 11 x being declared x = 2 y being declared == 3 x = 5 Error: Division by zero; using 1 instead y = 5 $

46. 241-437 Compilers: Attr. Grammars/8 46 6.4. exprParse1.c Callgraph same as in exprParse0.c symbol table (new) generated from grammar (now with actions)

47. 241-437 Compilers: Attr. Grammars/8 47 6.5. Symbol Table Data Structures #define MAX_SYMS 15 // max no of variables typedef struct SymInfo { char *id; // name of variable int value; // value (an integer) } SymbolInfo; int symNum = 0; // number of symbols stored SymbolInfo syms[MAX_SYMS];.. id value syms[] 01214

48. 241-437 Compilers: Attr. Grammars/8 48 Symbol Table Functions SymbolInfo *getIDEntry(void) /* find _OR_ create symbol table entry for current tokString; return a pointer to it */ { SymbolInfo *si = NULL; if ((si = lookupID(tokString)) != NULL) // already declared return si; // add id to table printf("%s being declared\n", tokString); return addID(tokString, 0); //0 is default value } // end of getIDEntry()

49. 241-437 Compilers: Attr. Grammars/8 49 SymbolInfo *lookupID(char *nm) /* is nm in the symbol table? return pointer to struct or NULL */ { int i; for(i=0; i<symNum; i++) if (!strcmp(syms[i].id, nm)) return &syms[i]; return NULL; } // end of lookupID()

50. 241-437 Compilers: Attr. Grammars/8 50 SymbolInfo *addID(char *nm, int value) /* add nm and value to the symbol table; return pointer to struct */ { if (symNum == MAX_SYMS) { printf("Symbol table full; cannot add %s\n", nm); exit(1); } syms[symNum].id = (char *) malloc(strlen(nm)+1); strcpy(syms[symNum].id, nm); syms[symNum].value = value; SymbolInfo *si = &syms[symNum]; symNum++; return si; } // end of addID()

51. 241-437 Compilers: Attr. Grammars/8 51 Using the Symbol Table The grammar functions use the symbol table via the matchID() function. SymbolInfo *matchId(void) // checks current ID with symbol table { SymbolInfo *si; dprint("Parsing ident\n"); if ((si = getIDEntry()) == NULL) { printf("Error: id is NULL on line %d\n",lineNum); exit(1); } match(ID); // ok, so consume ID token return si; } // end of matchId()

52. 241-437 Compilers: Attr. Grammars/8 52 6.6. Translating the Grammar Rules The same translation is carried out as before, but the code is augmented with actions. The semantic actions are translated into extra C code in the grammar functions.

53. 241-437 Compilers: Attr. Grammars/8 53 The Grammar Functions main() and statements() are unchanged from exprParse0.c since they don't have any semantic actions. Functions with extra actions: – –statement(), expression(), term(), factor()

54. 241-437 Compilers: Attr. Grammars/8 54 int main(void) { nextToken(); statements(); match(SCANEOF); return 0; } void statements(void) // statements ::= { // [ statement] '\n' } { dprint("Parsing statements\n"); dprint("Parsing statements\n"); while (currToken != SCANEOF) { while (currToken != SCANEOF) { if (currToken != NEWLINE) if (currToken != NEWLINE) statement(); statement(); match(NEWLINE); match(NEWLINE); } } // end of statements() Unchanged Functions

55. 241-437 Compilers: Attr. Grammars/8 55 statement() Before and After void statement(void) // statement ::= ( 'let' ID '=' EXPR ) | EXPR { if (currToken == LET) { match(LET); match(ID); match(ASSIGNOP); expression(); } else expression(); } // end of statement() with no semantic actions

56. 241-437 Compilers: Attr. Grammars/8 56 void statement(void) // statement ::= ( 'let' ID '=' EXPR ) | EXPR { SymbolInfo *si; int value; dprint("Parsing statement\n"); if (currToken == LET) { match(LET); si = matchId(); // was match(ID); match(ASSIGNOP); value = expression(); si->value = value; printf("%s = %d\n", si->id, value); } else { // expression value = expression(); printf("== %d\n", value); } Actions: add id to table; id.val = expr.val; print( id.val ); or print( expr.val );

57. 241-437 Compilers: Attr. Grammars/8 57 expression() Before and After void expression(void) // expression ::= term ( ('+'|'-') term )* { term(); while((currToken == PLUSOP) || (currToken == MINUSOP)) { match(currToken); term(); } } // end of expression() with no semantic actions

58. 241-437 Compilers: Attr. Grammars/8 58 int expression(void) // expression ::= term ( ('+'|'-') term )* { int result, v2; int isAddOp; dprint("Parsing expression\n"); result = term(); while((currToken == PLUSOP) || (currToken == MINUSOP)) { isAddOp = (currToken == PLUSOP) ? 1 : 0; match(currToken); v2 = term(); if (isAddOp == 1) // addition result += v2; else // subtraction result -= v2; } return result; } // end of expression() Action: return term 1.val (+| -) term 2.val (+| -)... term n.val;

59. 241-437 Compilers: Attr. Grammars/8 59 term() Before and After void term(void) // term ::= factor ( ('*'|'/') factor )* { factor(); while((currToken == MULTOP) || (currToken == DIVOP)) { match(currToken); factor(); } } // end of term() with no semantic actions

60. 241-437 Compilers: Attr. Grammars/8 60 int term(void) // term ::= factor ( ('*'|'/') factor )* { int result, v2; int isMultOp; dprint("Parsing term\n"); result = factor(); while((currToken == MULTOP) || (currToken == DIVOP)) { isMultOp = (currToken == MULTOP) ? 1 : 0; match(currToken); v2 = factor(); if (isMultOp == 1) // multiplication result *= v2; else { // division if (v2 == 0) printf("Error: Division by zero; using 1 instead\n"); else result = result / v2; } return result; } // end of term() Action: return fact 1.val (*| / ) fact 2.val (*| / )... fact n.val;

61. 241-437 Compilers: Attr. Grammars/8 61 factor() Before and After void factor(void) // factor ::= '(' expression ')' | INT | ID { if(currToken == LPAREN) { match(LPAREN); expression(); match(RPAREN); } else if(currToken == INT) match(INT); else if (currToken == ID) match(ID); else syntax_error(currToken); } // end of factor() with no semantic actions

62. 241-437 Compilers: Attr. Grammars/8 62 int factor(void) // factor ::= '(' expression ')' | INT | ID { int result = 0; dprint("Parsing factor\n"); if(currToken == LPAREN) { match(LPAREN); result = expression(); match(RPAREN); } else if(currToken == INT) { match(INT); result = currTokValue; } else if (currToken == ID) { SymbolInfo *si = matchId(); result = si->value; } else syntax_error(currToken); return result; } // end of factor() Actions: return expr.val; or return int.val; or add id to table (if new); return id.val;