1. CS 326 Programming Languages, Concepts and Implementation
Instructor: Mircea Nicolescu
Lecture 4
Good afternoon and thank you everyone for coming.
My talk today will describe the research I performed at the IRIS at USC, the object of this work being to build a computational framework that addresses the problem of motion analysis and interpretation.

2. Derivation and Parse Trees
A new grammar for arithmetic expressions:
expression → term | expression add_op term
term → factor | term mult_op factor
factor → identifier | number | - factor | ( expression )
add_op → + | -
mult_op → * | /
Unambiguous, also captures associativity and precedence

3. Derivation and Parse Trees
Precedence: * 5

4. Derivation and Parse Trees
Associativity:

5. Recognizing Syntax
Verify if a given program conforms to the syntax of the language
Discover the syntactic structure of the program
Corresponds to language implementation (writing compilers)
Will discuss:
scanners
parsers

6. Architecture Scanner Parser
ignores white space (blanks, tabs, new-lines)
ignores comments
recognizes tokens
implemented as a function that returns next token every time it is called
Parser
calls the scanner to obtain tokens
builds parse tree
passes it to the later phases (semantic analysis, code generation and improvement)
Parser controls the compilation process - "syntax-directed translation"

7. Scanning The scanner is usually implemented as either:
an ad-hoc program
an explicit finite automaton
a table-driven finite automaton
General rule - always accept longest possible token:
foobar is foobar, not f or foo or foob
3.14 is 3.14, not 3 then . then 14

8. Ad-hoc Scanner
Hand-written scanner for Pascal (incomplete):

9. Finite Automaton (FA)

10. Scanner - Explicit FA

11. Look-Ahead
In Pascal:
3.14 real number
3..5 the range of integers between 3 and 5
If it has seen the 3 and the next character coming is a dot, cannot decide yet - needs to peek one more character ahead

12. Look-Ahead In Fortran IV:
DO 5 I = 1, loop (for I = 1 to 25)
DO 5 I = assignment (DO5I = 1.25)
After seeing DO, cannot decide until reaching the comma or dot
DO 5,I = 1, alternate syntax for loop, FORTRAN 77

13. Scanner – Table-Driven FA

14. Automatic Scanner Generators
Unix: lex / flex
Take regular expresions as input
Produce a C program as output, that implements the scanner (table-driven)

15. Parsing
Given any context-free grammar, we can create a parser that runs in O(n3) time – too long
Particular classes of grammars that run in O(n) (linear) time:
LL (left-to-right, leftmost derivation)
LR (left-to-right, rightmost derivation)
LL parsers are also called “top-down” or “predictive”
LR parsers are also called “bottom-up” or “shift-reduce”

16. Parsing Example – consider the grammar:
id_list → id id_list_tail
id_list_tail → , id id_list_tail
id_list_tail → ;
Describes a comma-separated list of identifiers, such as:
A, B, C;
Let’s parse the input above

17. Parsing
Top-down
Bottom-up
Input:
A, B, C;
Grammar:

18. Parsing
The example grammar is not very well suited for bottom-up parsing. Why?
Because it needs to shift all input tokens until it finds the ;
Here is a more suitable one:
id_list → id_list_prefix ;
id_list_prefix → id_list_prefix , id
→ id
Other classes of grammars:
Subclasses of LR: SLR, LALR, …
Subclasses of LL: SLL, …
Notation for number of tokens to look ahead:
LL(k), LR(k)
In general, only LL(1) and LR(1) are used

19. Parsing Implementing a parser:
Recursive descent parser (top-down) – section from textbook
Automatic parser generators – in Unix: yacc / bison
Take a grammar as input
Produce a C program as output, that implements the parser

20. Announcements
Readings
Chapter 2, up to (and including) 2.2.3