1. CS 153: Concepts of Compiler Design November 3 Class Meeting Department of Computer Science San Jose State University Fall 2014 Instructor: Ron Mak www.cs.sjsu.edu/~mak 1

2. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 2 Unofficial Field Trip  Computer History Museum in Mt. View http://www.computerhistory.org/  Saturday, November 8, 11:30 – closing time Special free admission. Do a self-guided tour of the new Revolution exhibit. See a life-size working model of Charles Babbage’s Difference Engine in operation, a hand-cranked mechanical computer designed in the early 1800s. Experience a fully restored IBM 1401 mainframe computer from the early 1960s in operation.  General info: http://en.wikipedia.org/wiki/IBM_1401http://en.wikipedia.org/wiki/IBM_1401  My summer seminar: http://www.cs.sjsu.edu/~mak/1401/http://www.cs.sjsu.edu/~mak/1401/  Restoration: http://ed- thelen.org/1401Project/1401RestorationPage.htmlhttp://ed- thelen.org/1401Project/1401RestorationPage.html

3. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 3 Unofficial Field Trip  The new Revolution exhibit is now open! Walk through a timeline of the First 2000 Years of Computing History. Historic computer systems, data processing equipment, and other artifacts. Small theater presentations. Atanasoff-Berry Computer Hollerith Census Machine

4. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 4 Unofficial Field Trip  Babbage Difference Engine, fully operational. Hand-cranked mechanical computer. Computed polynomial functions. Designed by Charles Babbage in the early to mid 1800s.  Arguably the world’s first computer scientist, lived 1791-1871. He wasn’t able to build it because he lost his funding.  Live demo at 1:00  His plans survived and this working model was built. Includes a working printer! http://www.computerhistory.org/babbage/

5. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 5 Unofficial Field Trip  IBM 1401 computer, fully restored and operational A small transistor-based mainframe computer. Extremely popular with small businesses in the late 1950s through the mid 1960s  Maximum of 16K bytes of memory.  800 card/minute card reader (wire brushes).  600 line/minute line printer (impact).  6 magnetic tape drives, no disk drives.

6. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak Tesla Headquarters Visit 6 BourneJosephstudentComputer Engineering CairesDebraprofessorComputer Science DhillonGurleenstudentComputer Engineering DimovDimastudentComputer Science EstellKhalilstudentComputer Engineering FloresPedrostudentComputer Science GallegosKevinstudentComputer Engineering InzunzaOscarstudentComputer Science JoshiHardikstudentComputer Science KangJackstudentComputer Science KannanPrakasamstudentComputer Science KoumisAlexstudentComputer Engineering LongCamillestudentComputer Engineering MakRonprofessorComputer Science NarahariSrikanthstudentComputer Science NguyenDuystudentComputer Engineering PatelJaystudentComputer Science ShahShukanstudentComputer Science ThorpeDavidstudentComputer Science TorrefrancaDanielstudentComputer Engineering TranDucstudentComputer Engineering TranHungstudentComputer Engineering TsuiHelenstudentComputer Engineering WeiEileenstudentComputer Science WongNelsonstudentComputer Engineering Friday, November 14 at 2:00 3500 Deer Creek Road Palo Alto

7. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 7 Pcl  Pcl is a teeny, tiny subset of Pascal.  Use JavaCC to generate a Pcl parser and integrate with our Pascal interpreter’s symbol table components parse tree components  We’ll be able to parse and print the symbol table and the parse tree in our favorite XML format  Sample program test.pcl : PROGRAM test; VAR i, j, k : integer; x, y, z : real; BEGIN i := 1; j := i + 3; x := i + j; y := 314.15926e-02 + i - j + k; z := x + i*j/k - x/y/z END.

8. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 8 Pcl Challenges  Get the JJTree parse trees to build properly with respect to operator precedence. Use embedded definite node descriptors!  Decorate the parse tree with data type information. Can be done as the tree is built, or as a separate pass. You can use the visitor pattern to implement the pass.  Hook up to the symbol table and parse tree printing classes from the Pascal interpreter.

9. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 9 Pcl, cont’d options{ JJTREE_OUTPUT_DIRECTORY="src/wci/frontend"; NODE_EXTENDS="wci.intermediate.icodeimpl.ICodeNodeImpl";... } PARSER_BEGIN(PclParser)... public class PclParser { // Create and initialize the symbol table stack. symTabStack = SymTabFactory.createSymTabStack(); Predefined.initialize(symTabStack);... // Parse a Pcl program. Reader reader = new FileReader(sourceFilePath); PclParser parser = new PclParser(reader); SimpleNode rootNode = parser.program();...

10. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 10 Pcl, cont’d... // Print the cross-reference table. CrossReferencer crossReferencer = new CrossReferencer(); crossReferencer.print(symTabStack); // Visit the parse tree nodes to decorate them with type information. TypeSetterVisitor typeVisitor = new TypeSetterVisitor(); rootNode.jjtAccept(typeVisitor, null); // Create and initialize the ICode wrapper for the parse tree. ICode iCode = ICodeFactory.createICode(); iCode.setRoot(rootNode); programId.setAttribute(ROUTINE_ICODE, iCode); // Print the parse tree. ParseTreePrinter treePrinter = new ParseTreePrinter(System.out); treePrinter.print(symTabStack); } PARSER_END(PclParser) Demo

11. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 11 JavaCC Grammar Repository  Check these out to get ideas and models http://mindprod.com/jgloss/javacc.html http://mindprod.com/jgloss/javacc.html

12. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 12 Syntax Error Handling and JavaCC 1. Detect the error.  JavaCC does that based on the grammar in the.jj file. 2. Flag the error.  JavaCC does that for you with its error messages. 3. Recover from the error so you can continue parsing.  You set this up using JavaCC. _

13. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 13 Token Errors  By default, JavaCC throws an exception whenever it encounters a bad token.  Token errors are considered extremely serious and will stop the translation unless you take care to recover from them.  Example LOGO program that moves a cursor on a screen: FORWARD 20 RIGHT 120 FORWARD 20

14. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 14 Token Errors, cont’d  What happens if we feed the tokenizer bad input? SKIP : { " " | "\n" | "\r" | "\r\n" } TOKEN : { | logo_tokenizer.jj FORWARD 20 LEFT 120 FORWARD 20

15. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 15 Token Errors, cont’d  One way to recover from a token error is to skip over the erroneous token. public static void main(String[] args) throws Exception { java.io.Reader reader = new java.io.FileReader(args[0]); SimpleCharStream scs = new SimpleCharStream(reader); LogoTokenManager mgr = new LogoTokenManager(scs); while (true) { try { if (readAllTokens(mgr).kind == EOF) break; } catch (TokenMgrError tme) { System.out.println("TokenMgrError: " + tme.getMessage()); skipTo(' '); } } }

16. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 16 Token Errors, cont’d private static void skipTo(char delimiter) throws java.io.IOException { String skipped = ""; char ch; System.out.print("*** SKIPPING... "); while ((ch = input_stream.readChar()) != delimiter) { skipped += ch; } System.out.println("skipped '" + skipped + "'"); } logo_skip_chars.jj

17. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 17 Synchronize the Parser  Skipping over a bad token isn’t a complete solution.  The parser still needs to synchronize at the next good token and then attempt to continue parsing.

18. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 18 Synchronize the Parser, cont’d  First, add an error token to represent any invalid input characters: SKIP : { " " } TOKEN : { | } Any character except \r or \n.

19. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 19 Synchronize the Parser, cont’d  A program consists of one or more move (FORWARD) and turn (RIGHT) commands. Must also allow for an erroneous command. void Program() : {} { ( try { MoveForward() {System.out.println("Processed Move FORWARD");} | TurnRight() {System.out.println("Processed Turn RIGHT");} | Error() {handleError(token);} } catch (ParseException ex) { handleError(ex.currentToken); } )+ }

20. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 20 Synchronize the Parser, cont’d  The Error() production rule is invoked for the token. The token consumes the bad character. void MoveForward() : {} { } void TurnRight() : {} { } void Error() : {} { }

21. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 21 Synchronize the Parser, cont’d  The JAVACODE header precedes pure Java code that’s inserted into the generated parser. JAVACODE String handleError(Token token) { System.out.println("*** ERROR: Line " + token.beginLine + " after \"" + token.image + "\""); Token t; do { t = getNextToken(); } while (t.kind != EOL); return t.image; } logo_synchronize.jj Synchronize the parser to the next “good” token (EOL). You can do this better with a complete synchronization set!

22. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 22 Repair the Parse Tree  After the parser recovers from an error, you may want to remove a partially-built AST node. The erroneous production must call jjtree.popNode(). JAVACODE String handleError(Token token) #void { System.out.println("*** ERROR: Line " + token.beginLine + " after \"" + token.image + "\""); Token t; do { t = getNextToken(); } while (t.kind != EOL); jjtree.popNode(); return t.image; } logo_tree_recover.jjt

23. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 23 Debugging the Parser  Add the option to debug the parser.  Print production rule method calls and returns.  Print which tokens are consumed. options { DEBUG_PARSER=true; }

24. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 24 Review: Interpreter vs. Compiler  Same front end parser, scanner, tokens  Same intermediate tier symbol tables, parse trees

25. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 25 Review: Interpreter vs. Compiler, cont’d  Different back end operations.  Interpreter: Use the symbol tables and parse trees to execute the source program. executor  Compiler: Use the symbol tables and parse trees to generate an object program for the source program. code generator

26. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 26 Target Machines  A compiler’s back end code generator produces object code for a target machine.  Target machine: the Java Virtual Machine (JVM)  Object language: the Jasmin assembly language The Jasmin assembler translates the assembly language program into.class files. Java implements the JVM and loads and executes.class files. _

27. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 27 Target Machines, cont’d  Instead of using javac to compile a source program written in Java into a.class file, use your compiler to compile a source program written in your chosen language into a Jasmin object program, and then use the Jasmin assembler to create the.class file. No matter what language the source program was originally written in, once it’s been compiled into a.class file, the JVM will be able to load and execute it.  The JVM runs on a wide variety of hardware platforms.

28. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 28 Java Virtual Machine (JVM) Architecture  Java stack runtime stack  Heap area dynamically allocated objects automatic garbage collection  Class area code for methods constants pool  Native method stacks support native methods, e.g., written in C (not shown)

29. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 29 Java Virtual Machine Architecture, cont’d  The runtime stack contains stack frames. Stack frame = activation record.  Each stack frame contains local variables array operand stack program counter (PC) What is missing in the JVM that we had in our Pascal interpreter?

30. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 30 The JVM’s Java Runtime Stack  Each method invocation pushes a stack frame. Equivalent to the activation record of our Pascal interpreter.  The stack frame currently on top of the runtime stack is the active stack frame.  A stack frame is popped off when the method returns, possibly leaving behind a return value on top of the stack.

31. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 31 Contents of a JVM Stack Frame  Operand stack for doing computations  Local variables array equivalent to the memory map in our Pascal interpreter’s activation record  Program counter (PC) keeps track of the currently executing instruction

32. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 32 JVM Instructions  Load and store values  Arithmetic operations  Type conversions  Object creation and management  Runtime stack management (push/pop values)  Branching  Method call and return  Throwing exceptions  Concurrency

33. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 33 Jasmin Assembler  Download from: http://jasmin.sourceforge.net/  The site also includes: User Guide Instruction set Sample programs _

34. Computer Science Dept. Fall 2014: November 3 CS 153: Concepts of Compiler Design © R. Mak 34 Example Jasmin Program  Assemble: java –jar jasmin.jar hello.j  Execute: java HelloWorld.class public HelloWorld.super java/lang/Object.method public static main([Ljava/lang/String;)V.limit stack 2.limit locals 1 getstatic java/lang/System/out Ljava/io/PrintStream; ldc " Hello World. " invokevirtual java/io/PrintStream/println(Ljava/lang/String;)V return.end method hello.j Demo