1. CS 153: Concepts of Compiler Design November 2 Class Meeting Department of Computer Science San Jose State University Fall 2015 Instructor: Ron Mak www.cs.sjsu.edu/~mak

2. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 2 Midterm Results Median76.0 Mean72.2 Standard deviation20.0

3. 3 What JJTree, JJDoc, and JavaCC Do  You feed JJTree a.jjt grammar file Token specifications using regular expressions Production rules using EBNF  JJTree produces a.jj grammar file  JavaCC generates a scanner, parser, and tree-building routines Code for the visitor design pattern to walk the parse tree.  JJDoc produces a.html containing the ENBF

4. 4 What JJTree, JJDoc, and JavaCC Do  However, JJTree and JavaCC will not: Generate code for a symbol table Generate any backend code You have to provide this code!

5. 5 Pcl  Pcl is a teeny, tiny subset of Pascal.  Use JavaCC to generate a Pcl parser and integrate with our Pascal interpreter’s symbol table components parse tree components  We’ll be able to parse and print the symbol table and the parse tree in our favorite XML format  Sample program test.pcl : PROGRAM test; VAR i, j, k : integer; x, y, z : real; BEGIN i := 1; j := i + 3; x := i + j; y := 314.15926e-02 + i - j + k; z := x + i*j/k - x/y/z END.

6. 6 Pcl Challenges  Get the JJTree parse trees to build properly with respect to operator precedence. Use embedded definite node descriptors!  Decorate the parse tree with data type information. Can be done as the tree is built, or as a separate pass. You can use the visitor pattern to implement the pass.  Hook up to the symbol table and parse tree printing classes from the Pascal interpreter.

7. 7 Pcl, cont’d options{ JJTREE_OUTPUT_DIRECTORY="src/wci/frontend"; NODE_EXTENDS="wci.intermediate.icodeimpl.ICodeNodeImpl";... } PARSER_BEGIN(PclParser)... public class PclParser { // Create and initialize the symbol table stack. symTabStack = SymTabFactory.createSymTabStack(); Predefined.initialize(symTabStack);... // Parse a Pcl program. Reader reader = new FileReader(sourceFilePath); PclParser parser = new PclParser(reader); SimpleNode rootNode = parser.program();...

8. 8 Pcl, cont’d... // Print the cross-reference table. CrossReferencer crossReferencer = new CrossReferencer(); crossReferencer.print(symTabStack); // Visit the parse tree nodes to decorate them with type information. TypeSetterVisitor typeVisitor = new TypeSetterVisitor(); rootNode.jjtAccept(typeVisitor, null); // Create and initialize the ICode wrapper for the parse tree. ICode iCode = ICodeFactory.createICode(); iCode.setRoot(rootNode); programId.setAttribute(ROUTINE_ICODE, iCode); // Print the parse tree. ParseTreePrinter treePrinter = new ParseTreePrinter(System.out); treePrinter.print(symTabStack); } PARSER_END(PclParser) Demo

9. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 9 JavaCC Grammar Repository  Check these out to get ideas and models: http://mindprod.com/jgloss/javacc.html http://mindprod.com/jgloss/javacc.html

10. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 10 Syntax Error Handling and JavaCC 1. Detect the error.  JavaCC does that based on the grammar in the.jj file. 2. Flag the error.  JavaCC does that for you with its error messages. 3. Recover from the error so you can continue parsing.  You set this up using JavaCC.

11. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 11 Token Errors  By default, JavaCC throws an exception whenever it encounters a bad token.  Token errors are considered extremely serious and will stop the translation unless you take care to recover from them.  Example LOGO program that moves a cursor on a screen: FORWARD 20 RIGHT 120 FORWARD 20

12. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 12 Token Errors, cont’d  What happens if we feed the tokenizer bad input? SKIP : { " " | "\n" | "\r" | "\r\n" } TOKEN : { | logo_tokenizer.jj FORWARD 20 LEFT 120 FORWARD 20 Demo

13. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 13 Token Errors, cont’d  One way to recover from a token error is to skip over the erroneous token. public static void main(String[] args) throws Exception { java.io.Reader reader = new java.io.FileReader(args[0]); SimpleCharStream scs = new SimpleCharStream(reader); LogoTokenManager mgr = new LogoTokenManager(scs); while (true) { try { if (readAllTokens(mgr).kind == EOF) break; } catch (TokenMgrError tme) { System.out.println("TokenMgrError: " + tme.getMessage()); skipTo(' '); } } } logo_skip_chars.jj

14. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 14 Token Errors, cont’d private static void skipTo(char delimiter) throws java.io.IOException { String skipped = ""; char ch; System.out.print("*** SKIPPING... "); while ((ch = input_stream.readChar()) != delimiter) { skipped += ch; } System.out.println("skipped '" + skipped + "'"); } logo_skip_chars.jj Demo

15. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 15 Synchronize the Parser  Skipping over a bad token isn’t a complete solution.  The parser still needs to synchronize at the next good token and then attempt to continue parsing.  First, add an error token to represent any invalid input characters: SKIP : { " " } TOKEN : { | } Any character except \r or \n. logo_synchronize.jj

16. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 16 Synchronize the Parser, cont’d  A program consists of one or more move (FORWARD) and turn (RIGHT) commands. Must also allow for an erroneous command. void Program() : {} { ( try { MoveForward() {System.out.println("Processed Move FORWARD");} | TurnRight() {System.out.println("Processed Turn RIGHT");} | Error() {handleError(token);} } catch (ParseException ex) { handleError(ex.currentToken); } )+ } logo_synchronize.jj

17. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 17 Synchronize the Parser, cont’d  The Error() production rule is invoked for the token. The token consumes the bad character. void MoveForward() : {} { } void TurnRight() : {} { } void Error() : {} { } logo_synchronize.jj

18. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 18 Synchronize the Parser, cont’d  The JAVACODE header precedes pure Java code that’s inserted into the generated parser. JAVACODE String handleError(Token token) { System.out.println("*** ERROR: Line " + token.beginLine + " after \"" + token.image + "\""); Token t; do { t = getNextToken(); } while (t.kind != EOL); return t.image; } logo_synchronize.jj Synchronize the parser to the next “good” token (EOL). You can do this better with a complete synchronization set! Demo

19. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 19 Repair the Parse Tree  After the parser recovers from an error, you may want to remove a partially-built AST node. The erroneous production must call jjtree.popNode(). JAVACODE String handleError(Token token) #void { System.out.println("*** ERROR: Line " + token.beginLine + " after \"" + token.image + "\""); Token t; do { t = getNextToken(); } while (t.kind != EOL); jjtree.popNode(); return t.image; } logo_tree_recover.jjt Demo

20. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 20 Debugging the Parser  Add the option to debug the parser. options { DEBUG_PARSER=true; } Print production rule method calls and returns. Print which tokens are consumed.

21. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 21 Review: Interpreter vs. Compiler  Same front end parser, scanner, tokens  Same intermediate tier symbol tables, parse trees  Different back end operations

22. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 22 Review: Interpreter vs. Compiler, cont’d  Interpreter: Use the symbol tables and parse trees to execute the source program. executor  Compiler: Use the symbol tables and parse trees to generate an object program for the source program. code generator

23. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 23 Target Machines  A compiler’s back end code generator produces object code for a target machine.  Target machine: the Java Virtual Machine (JVM)  Object language: the Jasmin assembly language The Jasmin assembler translates the assembly language program into.class files. Java implements the JVM which loads and executes.class files.

24. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 24 Target Machines, cont’d  Instead of using javac to compile a source program written in Java into a.class file.  Use your compiler to compile a source program written in your chosen language into a Jasmin object program.  Then use the Jasmin assembler to create the.class file.

25. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 25 Target Machines, cont’d  No matter what language the source program was originally written in, once it’s been compiled into a.class file, Java will be able to load and execute it.  The JVM as implemented by Java runs on a wide variety of hardware platforms.

26. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 26 Java Virtual Machine (JVM) Architecture  Java stack runtime stack  Heap area dynamically allocated objects automatic garbage collection  Class area code for methods constants pool  Native method stacks support native methods, e.g., written in C (not shown)

27. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 27 Java Virtual Machine Architecture, cont’d  The runtime stack contains stack frames. Stack frame = activation record.  Each stack frame contains local variables array operand stack program counter (PC) What is missing in the JVM that we had in our Pascal interpreter?

28. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 28 The JVM’s Java Runtime Stack  Each method invocation pushes a stack frame.  Equivalent to the activation record of our Pascal interpreter.  The stack frame currently on top of the runtime stack is the active stack frame.  A stack frame is popped off when the method returns, possibly leaving behind a return value on top of the stack.

29. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 29 Stack Frame Contents  Operand stack For doing computations.  Local variables array Equivalent to the memory map in our Pascal interpreter’s activation record.  Program counter (PC) Keeps track of the currently executing instruction.

30. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 30 JVM Instructions  Load and store values  Arithmetic operations  Type conversions  Object creation and management  Runtime stack management (push/pop values)  Branching  Method call and return  Throwing exceptions  Concurrency

31. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 31 Jasmin Assembler  Download from: http://jasmin.sourceforge.net/  Site also includes: User Guide Instruction set Sample programs

32. SJSU Dept. of Computer Science Fall 2013: October 29 CS 153: Concepts of Compiler Design © R. Mak 32 Example Jasmin Program  Assemble: java –jar jasmin.jar hello.j  Execute: java HelloWorld.class public HelloWorld.super java/lang/Object.method public static main([Ljava/lang/String;)V.limit stack 2.limit locals 1 getstatic java/lang/System/out Ljava/io/PrintStream; ldc " Hello World. " invokevirtual java/io/PrintStream/println(Ljava/lang/String;)V return.end method hello.j Demo