 Welcome Back!  Introduction  Webpage: ◦

 Announcements ◦ No lab tomorrow  Take a look at HW#1  Take a look at Lab#1 ◦ #include, #define, #ifndef/#else/#endif ◦ Add the capability to your language even if it’s not # preprocessor directives ◦ Remove comments (any style)  //  /* */ - multiple lines ◦ Separate or integrated with lexical analyzer? ◦ More details later today and Friday  Introduction to Compilers

Here is a simple grammar: S -> cABd A -> aA | B -> bbB | Here is an input string: caaabbbbbbd Parse it!!

PROGRAM -> program id ( IDLIST ) ; DECLS SUBPROG_DECLS COMPOUND_STAT. IDLIST -> id | IDLIST, id DECLS -> DECLS var IDLIST : TYPE ; | λ TYPE -> integer | real First part of input string: program example(input, output); var x, y : integer;

 Questions? Comments?  No Class on Monday. HW#1 due on Wednesday  In-Class Exercise #1  Preprocessor in general  CFront was a preprocessor for early C++  Remove comments  Change cases if the language is not case sensitive  Lexical Analyzer ◦ How much to read it at a time? ◦ Implement it as a DFA (don’t have to worry about whitespaces)  Lab1 details ◦ Go over #ifndef/#else/#endif  See a real #include example  But don’t have to handle real #include files in this lab ◦ Go over #include ◦ Go over #define

SUBPROG_DECLS -> SUBPROG_DECLS SUB_DECL ; | λ SUB_DECL -> SUB_HEAD DECLS COMPOUND_STAT SUB_HEAD -> function id ARGS : TYPE ; ARGS -> ( PARAMLIST ) | λ PARAMLIST -> IDLIST : TYPE | PARAMLIST ; IDLIST : TYPE DECLS -> DECLS var IDLIST : TYPE ; | λ IDLIST -> id | IDLIST, id TYPE -> integer COMPOUND_STAT -> begin OPTIONAL_STAT end OPTIONAL_STAT -> STATLIST | λ function gcd(a, b: integer): integer; begin end;

 HW#1 due today. Questions?  Lab#1 due next Wednesday. Check off next Thursday.  Nothing to check off tomorrow in lab, but Sherry will be around all day.  Preprocessor in general  CFront was a preprocessor for early C++  Remove comments  Change cases if the language is not case sensitive  Lab1 details ◦ Python Example  Finish introduction slides

 Lab#1 due on Wednesday. Check-off on Thursday.  HW#2 due on Friday, Jan. 27 th.  In-Class Exercise #2  Top-down vs. Bottom-up Parsers  Top-down parsers ◦ Recursive Descent ◦ LL(1) table driven  Given a grammar, make sure it works for the parser that you are building.  If not, change it.

 Grammar Modifications ◦ Remove Lambda productions. ◦ Remove useless productions ◦ Remove unit productions. ◦ Remove left recursion in the grammar. ◦ Left factor the grammar.

 Lab#1 due today. Check-off tomorrow in lab (PV114) from 10am to 3:45pm. Please contact Sherry for other times.  HW#2 due on Friday. Just a draft for now.  Grammar Transformations ◦ Remove unit productions ◦ Remove left recursion (for top-down parsers) ◦ Left factor the grammar (to minimize backtracking)  Lab#2 – Top-down parser  In-Class Exercise #3

Program source file Lab#1 Lab#2 List of tokens Type Value token

 HW#2 due today. Just a draft for now.  Grammar Transformations ◦ Remove unit productions ◦ Remove left recursion (for top-down parsers) ◦ Left factor the grammar (to minimize backtracking)  Lab#2 – Top-down parser  In-Class Exercise #3

 HW#2 due today  HW#3 due on Wednesday  Lab#1 check-off continues this week.  In-Class Exercise#4  Finish grammar transformations ◦ Remove Unit productions ◦ Remove Left recursion ◦ Left Factor  Start looking at lab#2 ◦ Recursive Descent Parser

 Remove left recursion from the following grammar EXPR -> EXPR + TERM | EXPR – TERM | TERM TERM -> TERM * FACTOR | TERM / FACTOR | FACTOR FACTOR -> ( EXPR ) | ID | NUM  Verify that your new grammar works for: xy + ab * 35

 HW#3 Due today. Questions?  Comment on HW#2  In-Class Exercise #5: EXPR grammar  Code Example ◦ Recursive Descent Parser  More about Recursive Descent Parsers

 Lab#2 Due on Wednesday 2/8  Lab#2 check-off on Thursday 2/9  Test#1 on Friday 2/10  Go Over HW#`3  Recursive Descent Parser example posted (EXPR grammar)  Recursive Descent Parser more details  Using First set to improve top-down parsers  Using Follow set to handle lambdas.  Compute First & Follow sets

 Top-Down Parser  Easiest Parser to write.  Ideal for small languages.  Grammar Pre-req: ◦ NO LEFT RECURSION ◦ Left Factor (preferred) ◦ Removal of Unit Productions (helpful)

 Simple Algorithm ◦ bool function for each non-terminal in the grammar ◦ HUGE nested if-else statements trying to match each rule for a non-terminal. ◦ Always return true for lambda rules. ◦ Only advance token AFTER having matched a terminal/token.

 Lab#2 Due on Wednesday 2/8  Lab#2 check-off on Thursday 2/9 (1-6pm)  Test#1 on Friday 2/10  Using First set to improve top-down parsers  Using Follow set to handle lambdas.  Compute First & Follow sets  Use First & Follow sets to construct LL(1) table.  LL(1) table-driven parser

 Lab#2 Due Tomorrow in lab for check-off.  Lab#2 check-off on Thursday 2/9 (1-6pm)  Will hand out graded homework in lab.  Test#1 on Friday 2/10. What’s on it?  No lab & class next Thursday & Friday (Feb )  Review First & Follow sets computation  Use First & Follow sets to construct LL(1) table. ◦ In-Class Exercise #6  Finish EXPR grammar

 Use grammar to construct a parse tree.  How to modify grammar: ◦ Remove unit productions ◦ Remove left recursion ◦ Left factor a grammar  How to construct a small grammar for a given small subset of a language.  General knowledge of preprocessors and Recursive Descent Parser

 Circle the first column to determine which token that starts that rule S-> A B c First(A)- A-> a A-> B-> b B->

 Watch out for lambdas S-> A B c First(A)- + First(B)- A-> a A-> B-> b B->

 Watch out for lambdas S-> A B c First(A) - + First(B) - + c A-> a A-> B-> b B->

 What comes after? S-> A B c c follows B A-> a A-> B-> b B->

 What comes after? S-> A B c First(B) - follows A A-> a A-> B-> b B->

 Watch for lambdas S-> A B c c follows A A-> a A-> B-> b B->

S -> ABCd A -> e | f | B -> g | h | C -> p | q

 Hand back Test#1  No lab & class this Thursday & Friday (Feb )  Lab#2 continues this week (during office hours)  Finish EXPR grammar  Take a closer look at Follow set  How to construct the LL(1) parser (Lab#3)  Homework #4

EXPR -> TERM X X -> + TERM X | - TERM X | TERM -> FACTOR Z Z -> * FACTOR Z | / FACTOR Z | FACTOR -> id id $ +, -, $ id +, -, $ *, /, +, -, $ id *, /, +, -, $

 Engineering Month celebration, Tuesday, Feb. 22 nd 5:30pm, Shasta Complex. Free Pizza & Free T- Shirts.  Interview Tips & Tricks TODAY 5pm PV147  No lab & class this Thursday & Friday (Feb )  Lab#2 continues this week (during office hours)  Take a closer look at Follow set  Take a look at LL(1) parser implementation  Why isn’t it LL(1)?  In-Class Exercise #7  Homework #4 due on Monday, Feb. 20 st  Lab#3 due on Monday, Feb. 27 th.

 Compute First & Follow sets for this grammar.  Construct a LL(1) table using First & Follow sets S -> Ac | Bd A -> cA | B -> caB |

 Homework #4 due today. Questions?  Lab#3 due on Monday, Feb. 27 th.  Engineering Month celebration, Tuesday, Feb. 22 nd 5:30pm, Shasta Complex. Free Pizza & Free T- Shirts.  Lab#2 continues this week  Top-Down Parsers vs. Bottom-Up Parsers  Bottom-Up Parsers  Looking Ahead: ◦ Using Tools (Demo) ◦ Semantic Analysis ◦ Back-End Compiler (Project Options)

Grammar Program / Engine Test Program Test.c

Grammar Program / Engine Test Program Test.c Yes/No Tree structure

LL(1) table Engine Test Program Test.c Yes/No Tree structure Stack

0) S' –> S 1) S –> XX 2) X –> aX 3) X –> b

StateActiongoto Id+*()$ETF0S5S4123 1S6accept 2R2S7R2 3R4R4R4 4S5S4823 5R6R6R6 6S5S493 7S5S410 8S6S11 9R1S7R1R1 10R3R3R3R3 11R5R5R5R5 (0) E’ -> E (4) T -> F (1) E-> E + T (5) F -> ( E ) (2) E -> T (6) F -> id (3) T -> T * F

Lab#3 due on Monday, Feb. 27 th.Lab#3 due on Monday, Feb. 27 th. Engineering Month celebration, Wednesday, Feb. 22 nd 5:30pm, Shasta Complex. Free Pizza & Free T- Shirts.Engineering Month celebration, Wednesday, Feb. 22 nd 5:30pm, Shasta Complex. Free Pizza & Free T- Shirts. Lab#2 check-off continues this weekLab#2 check-off continues this week LR(1) Parser Table ExampleLR(1) Parser Table Example In-Class Exercise #8 Final Project OptionsFinal Project Options Looking Ahead:Looking Ahead: Using Tools (Demo) Semantic Analysis Back-End Compiler (Project Options) CST320 Week 7 Wednesday

Final Project Options Parser GeneratorsParser Generators Given a grammar, generate the code for a Recursive Descent Parser Given a grammar, generate the LL(1) table to be used in a LL(1) parser Program TranslatorsProgram Translators Convert a program in one language (i.e. C) to another language (i.e. Python) Convert a program in one language (i.e. C) to a low-level language (i.e. Assembly, Microsoft IL, etc.) CST116 Cheat DetectorCST116 Cheat Detector Your parser MUST work for typical CST116 labs (get them from Todd or Randy) Your parser must read in at least 3 files to compare. Compute software metrics (see final project description) and report on how similar or different the programs are.

Final Project Options Small InterpreterSmall Interpreter Grammar Parser Runtime Environment int x; write “Enter a num”; read x; if (x > 3) write “Hi”; else write “Hello”; Enter a num: 45 Hello

StateActiongoto Id+*()$ETF0S5S4123 1S6accept 2R2S7R2 3R4R4R4 4S5S4823 5R6R6R6 6S5S493 7S5S410 8S6S11 9R1S7R1R1 10R3R3R3R3 11R5R5R5R5 (0) E’ -> E (4) T -> F (1) E-> E + T (5) F -> ( E ) (2) E -> T (6) F -> id (3) T -> T * F

Lab#3 due tonight.Lab#3 due tonight. Lab#3 check-off on ThursdayLab#3 check-off on Thursday Final ProjectFinal Project Check-off date on Thursday of dead week. Last day to turn in late labs is Friday of dead week. Incompletes will be given if not turned in by Friday of dead week. (One letter grade lower for completion later). Test #2 on Friday, Feb. 9 th.Test #2 on Friday, Feb. 9 th. Dead week – time for project, no new topics.Dead week – time for project, no new topics. Make-up test during Dead Week.Make-up test during Dead Week. CST320 Week 8 Monday

44 Conventional Translator skeletal source program preprocessor source program library, relocatable object files compiler assembler target assembly program loader / linker relocatable machine code absolute machine code

45 Structure of Compilers Lexical Analyzer (scanner) Source Program Syntax Analysis (Parser) Tokens Semantic Analysis Syntactic Structure Optimizer Code Generator Intermediate Representation Target machine code Symbol Table

Intermediate Representation Almost no compiler produces code without first converting a program into some intermediate representation that is used just inside the compiler. This intermediate representation is called by various names: –Internal Representation –Intermediate representation –Intermediate language

Intermediate Representation Different forms of intermediate representation: –tuples –Abstract Syntax Trees –Simplified language

48 Abstract Syntax Tree x = y + 3; = x+ y 3

49 Quadruples y= a*(x+b)/(x-c); T1= x+b;(+, 3, 4, 5) T2=a*T1; (*, 2, 5, 6) T3=x-c;(-, 3, 7, 8) T4=T2/T3;(/, 6, 8, 9) y=T4;(=, 0, 9, 1) y a x b T1 T2 c T3 T4