Elements of Computing Systems, Nisan & Schocken, MIT Press, www.nand2tetris.org, Chapter 10: Compiler I: Syntax Analysis slide 1www.nand2tetris.org Building.

Slides:



Advertisements
Similar presentations
Chapter 4: Machine Language
Advertisements

Elements of Computing Systems, Nisan & Schocken, MIT Press, 2005, Chapter 1: Boolean Logic slide 1www.idc.ac.il/tecs Chapter 1: Boolean.
CPSC Compiler Tutorial 9 Review of Compiler.
CS 330 Programming Languages 09 / 13 / 2007 Instructor: Michael Eckmann.
BİL744 Derleyici Gerçekleştirimi (Compiler Design)1.
1.3 Executing Programs. How is Computer Code Transformed into an Executable? Interpreters Compilers Hybrid systems.
Elements of Computing Systems, Nisan & Schocken, MIT Press, 2005, Chapter 8: VM II: Program Control slide 1www.idc.ac.il/tecs Chapter.
Elements of Computing Systems, Nisan & Schocken, MIT Press, 2005, Chapter 7: VM I: Stack Arithmetic slide 1www.idc.ac.il/tecs Chapter.
Elements of Computing Systems, Nisan & Schocken, MIT Press, 2005, Introduction: Hello, World Below slide 1www.idc.ac.il/tecs Introduction:
Elements of Computing Systems, Nisan & Schocken, MIT Press, 2005, Chapter 10: Compiler I: Syntax Analysis slide 1www.idc.ac.il/tecs.
Elements of Computing Systems, Nisan & Schocken, MIT Press, 2005, Chapter 2: Boolean Logic slide 1www.idc.ac.il/tecs Chapter 2: Boolean.
Invitation to Computer Science 5th Edition
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 1www.idc.ac.il/tecs Compiler.
2.2 A Simple Syntax-Directed Translator Syntax-Directed Translation 2.4 Parsing 2.5 A Translator for Simple Expressions 2.6 Lexical Analysis.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 7: Virtual Machine I: Stack Arithmetic slide 1www.nand2tetris.org.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 7: Virutal Machine, Part I slide 1www.idc.ac.il/tecs Virtual Machine.
ICS611 Introduction to Compilers Set 1. What is a Compiler? A compiler is software (a program) that translates a high-level programming language to machine.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 4: Machine Language slide 1www.idc.ac.il/tecs Machine Language.
INTRODUCTION TO COMPUTING CHAPTER NO. 06. Compilers and Language Translation Introduction The Compilation Process Phase 1 – Lexical Analysis Phase 2 –
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 6: Assembler slide 1www.idc.ac.il/tecs Assembler Elements of Computing.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 5: Computer Architecture slide 1www.idc.ac.il/tecs Computer Architecture.
Chapter 1 Introduction Dr. Frank Lee. 1.1 Why Study Compiler? To write more efficient code in a high-level language To provide solid foundation in parsing.
Chapter 10: Compilers and Language Translation Invitation to Computer Science, Java Version, Third Edition.
1 Programming Languages Tevfik Koşar Lecture - II January 19 th, 2006.
Compiler course 1. Introduction. Outline Scope of the course Disciplines involved in it Abstract view for a compiler Front-end and back-end tasks Modules.
Concordia University Department of Computer Science and Software Engineering Click to edit Master title style COMPILER DESIGN Review Joey Paquet,
CST320 - Lec 11 Why study compilers? n n Ties lots of things you know together: –Theory (finite automata, grammars) –Data structures –Modularization –Utilization.
Building a Modern Computer From First Principles
Elements of Computing Systems, Nisan & Schocken, MIT Press, 2005, Chapter 11: Compiler II: Code Generation slide 1www.idc.ac.il/tecs.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 1: Boolean Logic slide 1www.nand2tetris.org Building a Modern.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 1: Boolean Logic slide 1www.idc.ac.il/tecs Boolean Logic Elements.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 2: Boolean Arithmetic slide 1www.nand2tetris.org Building a Modern.
Joey Paquet, Lecture 12 Review. Joey Paquet, Course Review Compiler architecture –Lexical analysis, syntactic analysis, semantic.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 8: Virtual Machine, Part II slide 1www.idc.ac.il/tecs Virtual Machine.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 8: Virtual Machine, Part II slide 1www.nand2tetris.org Building.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 1: Compiler II: Code Generation slide 1www.nand2tetris.org Building.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 11: Compiler II: Code Generation slide 1www.nand2tetris.org Building.
Review 1.Lexical Analysis 2.Syntax Analysis 3.Semantic Analysis 4.Code Generation 5.Code Optimization.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 7: Virutal Machine, Part I slide 1www.nand2tetris.org Building.
Unit-1 Introduction Prepared by: Prof. Harish I Rathod
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 3: Sequential Logic slide 1www.idc.ac.il/tecs Sequential Logic.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 3: Sequential Logic slide 1www.nand2tetris.org Building a Modern.
1.  10% Assignments/ class participation  10% Pop Quizzes  05% Attendance  25% Mid Term  50% Final Term 2.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 1www.nand2tetris.org Building.
CPS 506 Comparative Programming Languages Syntax Specification.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 9: High-Level Language slide 1www.nand2tetris.org Building a Modern.
Chapter 1 Introduction. Chapter 1 - Introduction 2 The Goal of Chapter 1 Introduce different forms of language translators Give a high level overview.
1. 2 Preface In the time since the 1986 edition of this book, the world of compiler design has changed significantly 3.
CS 460/660 Compiler Construction. Class 01 2 Why Study Compilers? Compilers are important – –Responsible for many aspects of system performance Compilers.
Comp 311 Principles of Programming Languages Lecture 3 Parsing Corky Cartwright August 28, 2009.
1 Compiler Design (40-414)  Main Text Book: Compilers: Principles, Techniques & Tools, 2 nd ed., Aho, Lam, Sethi, and Ullman, 2007  Evaluation:  Midterm.
Building a Modern Computer From First Principles
Compiler Introduction 1 Kavita Patel. Outlines 2  1.1 What Do Compilers Do?  1.2 The Structure of a Compiler  1.3 Compilation Process  1.4 Phases.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 6: Assembler slide 1www.nand2tetris.org Building a Modern Computer.
Overview of Compilation Prepared by Manuel E. Bermúdez, Ph.D. Associate Professor University of Florida Programming Language Principles Lecture 2.
Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 1www.nand2tetris.org Building.
Presented by : A best website designer company. Chapter 1 Introduction Prof Chung. 1.
CS416 Compiler Design1. 2 Course Information Instructor : Dr. Ilyas Cicekli –Office: EA504, –Phone: , – Course Web.
CS510 Compiler Lecture 1. Sources Lecture Notes Book 1 : “Compiler construction principles and practice”, Kenneth C. Louden. Book 2 : “Compilers Principles,
Compiler Design (40-414) Main Text Book:
Chapter 1 Introduction.
Introduction to Compiler Construction
Introduction: From Nand to Tetris
SOFTWARE DESIGN AND ARCHITECTURE
Chapter 1 Introduction.
课程名 编译原理 Compiling Techniques
Compiler Lecture 1 CS510.
Compiler I: Sytnax Analysis
Introduction CI612 Compiler Design CI612 Compiler Design.
Chapter 10: Compilers and Language Translation
Presentation transcript:

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 1www.nand2tetris.org Building a Modern Computer From First Principles Compiler I: Syntax Analysis

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 2www.nand2tetris.org Usage and Copyright Notice: Copyright © Noam Nisan and Shimon Schocken This presentation contains lecture materials that accompany the textbook “The Elements of Computing Systems” by Noam Nisan & Shimon Schocken, MIT Press, We provide both PPT and PDF versions. Our web site, a set of presentations, one for each book chapter. Each presentation is designed to support about 3 hours of classroom or self-study instruction. You are welcome to use or edit this presentation as you see fit for instructional and non- commercial purposes. If you use our materials, please include a reference to If you have any questions or comments, please write us at

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 3www.nand2tetris.org Course map Assembler Chapter 6 H.L. Language & Operating Sys. abstract interface Compiler Chapters VM Translator Chapters Computer Architecture Chapters Gate Logic Chapters Electrical Engineering Physics Virtual Machine abstract interface Software hierarchy Assembly Language abstract interface Hardware hierarchy Machine Language abstract interface Hardware Platform abstract interface Chips & Logic Gates abstract interface Human Thought Abstract design Chapters 9, 12

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 4www.nand2tetris.org Motivation: Why study about compilers? Because Compilers … Are an essential part of applied computer science Are very relevant to computational linguistics Are implemented using classical programming techniques Employ important software engineering principles Train you in developing software for transforming one structure to another (programs, files, transactions, …) Train you to think in terms of ”description languages”.

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 5www.nand2tetris.org The big picture Intermediate code VM implementation over CISC platforms VM imp. over RISC platforms VM imp. over the Hack platform VM emulator VM lectures (Projects 7-8) Some Other language Jack language Some compiler Some Other compiler Jack compiler... Some language... Compiler lectures (Projects 10,11) Modern compilers are two-tiered: Front-end: from high-level language to some intermediate language Back-end: from the intermediate language to binary code.

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 6www.nand2tetris.org Compiler architecture (front end) Syntax analysis: understanding the semantics implied by the source code Code generation: reconstructing the semantics using the syntax of the target code.  Tokenizing: creating a stream of “atoms”  Parsing: matching the atom stream with the language grammar XML output = one way to demonstrate that the syntax analyzer works (source)(target)

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 7www.nand2tetris.org Tokenizing / Lexical analysis Remove white space Construct a token list (language atoms) Things to worry about: Language specific rules: e.g. how to treat “++” Language-specific classifications: keyword, symbol, identifier, integerCconstant, stringConstant,... While we are at it, we can have the tokenizer record not only the token, but also its lexical classification (as defined by the source language grammar).

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 8www.nand2tetris.org Jack Tokenizer if (x < 153) {let city = ”Paris”;} Source code if ( x < 153 ) { let city = Paris ; } if ( x < 153 ) { let city = Paris ; } Tokenizer’s output Tokenizer

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 9www.nand2tetris.org Parsing The tokenizer discussed thus far is part of a larger program called parser Each language is characterized by a grammar. The parser is implemented to recognize this grammar in given texts The parsing process: A text is given and tokenized The parser determines weather or not the text can be generated from the grammar In the process, the parser performs a complete structural analysis of the text The text can be in an expression in a : Natural language (English, …) Programming language (Jack, …).

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 10www.nand2tetris.org Parsing examples (5+3)*2 – sqrt(9*4)she discussed sex with her doctor JackEnglish discussed shesex with her doctor parse 1 discussed she with her doctor parse 2 sex

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 11www.nand2tetris.org More examples of challenging parsing We gave the monkeys the bananas because they were hungry We gave the monkeys the bananas because they were over-ripe I never said she stole my money Time flies like an arrow

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 12www.nand2tetris.org Simple (terminal) forms / complex (non-terminal) forms Grammar = set of rules on how to construct complex forms from simpler forms Highly recursive. A typical grammar of a typical C-like language while (expression) { if (expression) statement; while (expression) { statement; if (expression) statement; } while (expression) { statement; } if (expression) { statement; while (expression) statement; } if (expression) statement; } while (expression) { if (expression) statement; while (expression) { statement; if (expression) statement; } while (expression) { statement; } if (expression) { statement; while (expression) statement; } if (expression) statement; } Code sample program: statement; statement: whileStatement | ifStatement | // other statement possibilities... | '{' statementSequence '}' whileStatement: 'while' '(' expression ')' statement ifStatement: simpleIf | ifElse simpleIf: 'if' '(' expression ')' statement ifElse: 'if' '(' expression ')' statement 'else' statement statementSequence: '' // null, i.e. the empty sequence | statement ';' statementSequence expression: // definition of an expression comes here // more definitions follow program: statement; statement: whileStatement | ifStatement | // other statement possibilities... | '{' statementSequence '}' whileStatement: 'while' '(' expression ')' statement ifStatement: simpleIf | ifElse simpleIf: 'if' '(' expression ')' statement ifElse: 'if' '(' expression ')' statement 'else' statement statementSequence: '' // null, i.e. the empty sequence | statement ';' statementSequence expression: // definition of an expression comes here // more definitions follow Grammar

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 13www.nand2tetris.org Parse tree program: statement; statement: whileStatement | ifStatement | // other statement possibilities... | '{' statementSequence '}' whileStatement: 'while' '(' expression ')' statement... program: statement; statement: whileStatement | ifStatement | // other statement possibilities... | '{' statementSequence '}' whileStatement: 'while' '(' expression ')' statement...

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 14www.nand2tetris.org Recursive descent parsing Parser implementation: a set of parsing methods, one for each rule: parseStatement() parseWhileStatement() parseIfStatement() parseStatementSequence() parseExpression(). Highly recursive LL(0) grammars: the first token determines in which rule we are In other grammars you have to look ahead 1 or more tokens Jack is almost LL(0). while (expression) { statement; while (expression) { while (expression) statement; } while (expression) { statement; while (expression) { while (expression) statement; } code sample

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 15www.nand2tetris.org A linguist view on parsing Parsing: One of the mental processes involved in sentence comprehension, in which the listener determines the syntactic categories of the words, joins them up in a tree, and identifies the subject, object, and predicate, a prerequisite to determining who did what to whom from the information in the sentence. (Steven Pinker, The Language Instinct)

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 16www.nand2tetris.org The Jack grammar ’x’ : x appears verbatim x : x is a language construct x? : x appears 0 or 1 times x* : x appears 0 or more times x|y : either x or y appears (x,y) : x appears, then y. ’x’ : x appears verbatim x : x is a language construct x? : x appears 0 or 1 times x* : x appears 0 or more times x|y : either x or y appears (x,y) : x appears, then y.

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 17www.nand2tetris.org The Jack grammar (cont.) ’x’ : x appears verbatim x : x is a language construct x? : x appears 0 or 1 times x* : x appears 0 or more times x|y : either x or y appears (x,y) : x appears, then y. ’x’ : x appears verbatim x : x is a language construct x? : x appears 0 or 1 times x* : x appears 0 or more times x|y : either x or y appears (x,y) : x appears, then y.

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 18www.nand2tetris.org Jack syntax analyzer in action Class Bar { method Fraction foo(int y) { var int temp; // a variable let temp = (xxx+12)*-63; Class Bar { method Fraction foo(int y) { var int temp; // a variable let temp = (xxx+12)*-63; Syntax analyzer Using the language grammar, a programmer can write a syntax analyzer program (parser) The syntax analyzer takes a source text file and attempts to match it on the language grammar If successful, it can generate a parse tree in some structured format, e.g. XML. var int temp ; let temp = ( xxx var int temp ; let temp = ( xxx Syntax analyzer The syntax analyzer’s algorithm shown in this slide: If xxx is non-terminal, output: Recursive code for the body of xxx If xxx is terminal (keyword, symbol, constant, or identifier), output: xxx value

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 19www.nand2tetris.org JackTokenizer: a tokenizer for the Jack language (proposed implementation)

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 20www.nand2tetris.org JackTokenizer (cont.)

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 21www.nand2tetris.org CompilationEngine: a recursive top-down parser for Jack The CompilationEngine effects the actual compilation output. It gets its input from a JackTokenizer and emits its parsed structure into an output file/stream. The output is generated by a series of compilexxx() routines, one for every syntactic element xxx of the Jack grammar. The contract between these routines is that each compilexxx() routine should read the syntactic construct xxx from the input, advance() the tokenizer exactly beyond xxx, and output the parsing of xxx. Thus, compilexxx() may only be called if indeed xxx is the next syntactic element of the input. In the first version of the compiler, which we now build, this module emits a structured printout of the code, wrapped in XML tags (defined in the specs of project 10). In the final version of the compiler, this module generates executable VM code (defined in the specs of project 11). In both cases, the parsing logic and module API are exactly the same.

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 22www.nand2tetris.org CompilationEngine (cont.)

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 23www.nand2tetris.org CompilationEngine (cont.)

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 24www.nand2tetris.org CompilationEngine (cont.)

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 25www.nand2tetris.org Summary and next step Syntax analysis: understanding syntax Code generation: constructing semantics The code generation challenge: Extend the syntax analyzer into a full-blown compiler that, instead of generating passive XML code, generates executable VM code Two challenges: (a) handling data, and (b) handling commands.

Elements of Computing Systems, Nisan & Schocken, MIT Press, Chapter 10: Compiler I: Syntax Analysis slide 26www.nand2tetris.org Perspective The parse tree can be constructed on the fly Syntax analyzers can be built using: Lex tool for tokenizing Yacc tool for parsing Do everything from scratch (our approach...) The Jack language is intentionally simple: Statement prefixes: let, do,... No operator priority No error checking Basic data types, etc. Richer languages require more powerful compilers The Jack compiler: designed to illustrate the key ideas that underlie modern compilers, leaving advanced features to more advanced courses Industrial-strength compilers: Have good error diagnostics Generate tight and efficient code Support parallel (multi-core) processors.