1. 1 Lecture 5: Syntax Analysis (Section 2.2) CSCI 431 Programming Languages Fall 2002 A modification of slides developed by Felix Hernandez-Campos at UNC Chapel Hill

2. 2 Review: Compilation/Interpretation Compiler or Interpreter Translation Execution Source Code Target Code Interpre-tation

3. 3 Review: Syntax Analysis Compiler or Interpreter Translation Execution Source Code Specifying the formSpecifying the form of a programming language –Tokens »Regular Expressions (also F.A.s & Reg. Grammars) –Syntax »Context-Free Grammars (also P.D.A.s) Target Code

4. 4 Phases of Compilation

5. 5 Syntax Analysis Syntax:Syntax: –Webster’s definition: 1 a : the way in which linguistic elements (as words) are put together to form constituents (as phrases or clauses) The syntax of a programming languageThe syntax of a programming language –Describes its form »Organization of tokens »Context Free Grammars (CFGs) –Must be recognizable by compilers and interpreters »Parsing »LL and LR parsers

6. 6 Context Free Grammars CFGsCFGs –Add recursion to regular expressions »Nested constructions –Notation expression  identifier | number | - expression | ( expression ) | ( expression ) | expression operator expression | expression operator expression operator  + | - | * | / »Terminal symbols »Non-terminal symbols »Production rule (i.e. substitution rule) terminal symbol  terminal and non-terminal symbols

7. 7 Parsing Parsing an arbitrary Context Free GrammarParsing an arbitrary Context Free Grammar –O(n 3 ) –Too slow for large programs Linear-time parsingLinear-time parsing –LL parsers (a ‘Left-to-right, Left-most’ derivation) »Recognize LL grammar »Use a top-down strategy –LR parsers (a ‘Left-to-right, Right-most’ derivation) »Recognize LR grammar »Use a bottom-up strategy

8. 8 Parsing example Example: comma-separated list of identifierExample: comma-separated list of identifier –CFG id_list  id id_list_tail id_list_tail , id_list_tail id_list_tail  ; –Parsing A, B, C;

9. 9 Top-down derivation of A, B, C; CFG Left-to-right, Left-most derivation LL(1) parsing

10. 10 Top-down derivation of A, B, C; CFG

11. 11 Bottom-up parsing of A, B, C; CFG Left-to-right, Right-most derivation LR parsing (a shift-reduce parser)

12. 12 Bottom-up parsing of A, B, C; CFG

13. 13 CFG

14. 14 LR Parsing vs. LL Parsing LLLL –A ‘top-down’ or ‘predictive’ parser –Predict needed productions based on the current left-most non-terminal in the tree and the current input token –The top-of-stack contains the left-most non-terminal –The stack contains a record of what the parser expects to see LRLR –A ‘bottom-up’ or shift-reduce parser –Shifts tokens onto the stack until it recognizes a right-hand side then reduces those tokens to their left-hand side –The stack contains a record of what the parser has already seen

15. 15 An appropriate LR Grammar id_list  id_list_prefix ; id_list_prefix  id_list_prefix, id  id  id This grammar can’t be parsed top-down! Problems for LL grammars: - left recursion, example above - common prefixes, example: stmt  id := expr | id (arg_list)

16. 16 LL(1) Grammar for the Calculator Language

17. 17 LR(1) Grammar for the Calculator Language

18. 18 Hierarchy of Linear Parsers Basic containment relationshipBasic containment relationship –All CFGs can be recognized by LR parser –Only a subset of all the CFGs can be recognized by LL parsers LL parsing CFGs LR parsing

19. 19 Bigger Picture Chomsky Hierarchy of GrammarsChomsky Hierarchy of Grammars RegularGrammar Context Free Grammar Context Sensitive Grammar Unrestricted Grammar

20. 20 Implementation of an LL Parser Two options:Two options: –A recursive descent parser (section 2.2.3) »For LL grammars only –Parse table and a driver (section 2.2.5) »LR parsers covered in section 2.2.6

21. 21 Recursive Descent Parser Example LL(1) grammarLL(1) grammar

22. 22 Recursive Descent Parser Example Outline ofOutline of recursive parser –This parser only verifies syntax –match is the scanner

23. 23 Recursive Descent Parser Example

24. 24

25. 25 A program that develops recursive decent parsers: JavaCC JavaCC

26. 26 Semantic Analysis Compiler or Interpreter Translation Execution Source Code Specifying the meaningSpecifying the meaning of a programming language –Attribute Grammars Target Code