1. Syntax
COMP 640 Lecture 2

2. Topics Formal Grammars Lexical Grammars Syntactic Analysis Jay BNF
EBNF
Syntax Diagrams
Lexical Grammars
Regular Expressions
Syntactic Analysis
Concrete Grammars
Ambiguity
Jay
COMP 640 – CWB

3. Definitions Language Derivation (rightmost, leftmost) Sentential form
Lexical analysis
“Lexer”
Parse (n. and v.)
Top down, bottom up
COMP 640 – CWB

4. Major Stages in the Compiling Process
Figure 2.4
Jflex
CUP
From T & N
Alt: source code for another language
COMP 640 – CWB

5. Formal Syntax BNF Backus and Naur Noam Chomsky’s linguistic theory
A  ω (Or A ::= ω)
COMP 640 – CWB

6. Extended BNF FPLiteral  Digits DOT Digitsopt Exponentopt Suffixopt |
Sometimes: DigitsOpt
FPLiteral  Digits DOT Digitsopt Exponentopt Suffixopt |
DOT Digits Exponentopt Suffixopt |
Digits Exponent Suffixopt |
Digits Exponentopt Suffix
Digits  Digit { Digit }* …
COMP 640 – CWB

7. Alternative BNF FPLiteral: Digits DOT Digitsopt Exponentopt Suffixopt
DOT Digits Exponentopt Suffixopt
Digits Exponent Suffixopt
Digits Exponentopt Suffix
Digits: Digit { Digit }* …
COMP 640 – CWB

8. Examples from Java Spec
TOC:
Lexical Syntax:
Literals:
COMP 640 – CWB

9. Syntax Diagram for Expressions with Addition
Figure 2.14
AKA Railroad Diagrams
From T & N
COMP 640 – CWB

10. Java Syntax Diagrams
COMP 640 – CWB

11. Lexical Analysis
Typical grammatical categories: identifier, literal, operator, separator, keyword
What defines what is a “lexical element”?
COMP 640 – CWB

12. A Simple Lexical Syntax for a Small Language, Jay
Figure 2.3
Resort to informal description
From T & N
COMP 640 – CWB

13. Skeleton Lexical Analysis Method That Returns Tokens
Figure 2.5
From T & N
COMP 640 – CWB

14. Hand Coded Lexer for Java from the CUP Site
Expanded ZIP file from the CUP website is located here:
The Lex/ directory contains a Java lexer: see Lexer.java
COMP 640 – CWB

15. Question
Why use a separate lexer? Why not just combine the lexical grammar in with the overall syntactic grammar? After all, a regular grammar is context free.
COMP 640 – CWB

16. Regular Expressions ( M ) apparently omitted From T&N, Figure 2.6
COMP 640 – CWB

17. Common REs Integer: [0-9]+ Identifier: [a-zA-Z][a-zA-Z0-9]*
End of line: (“\r” | “\n” | “\r\n”)
COMP 640 – CWB

18. Regular Expressions in Jflex
Features that are not in the text’s RE:
! Logical not
~ Up to
{n} n occurrences
{n,m} n to M occurrences
Grammar of Jflex’s Language, Including Its RE Language:
Franklin~(ity)
(Boing){2}
COMP 640 – CWB

19. Syntactic Analysis – Concrete Syntax
X + 2 * y
Higher Precedence
Postorder traversal
Grammatical elements parsed at the lexical level
From T & N, Figures 2.7 & 2.8
COMP 640 – CWB

20. Omitted from Syntactical Analysis
Type validation
Reachability analysis
Assignment before use
Sometimes: syntax checking that is “hard”
COMP 640 – CWB

21. Sketch of a Parse Tree for a Complete Program
Figure 2.9
void main ( ) {
int x;
x = 1; }
From T & N
COMP 640 – CWB

22. Ambiguity
2 – 3 - 4
From T & N, Figure 2.10
COMP 640 – CWB

23. Shift-Reduce Ambiguity
Expr  Term | Expr + Term | Expr - Term
Term  Factor | Term * Factor
Factor  Identifier | Literal | (Expr)
Tokens read so far:
Identifier - Identifier
a – b - c
Next token: -
Reduce to:
Expr (a-b)
Left Associativity
Choice
Or, “shift” (move right in the input stream)
Identifier - Identifier - Identifier
Right Associativity
COMP 640 – CWB

24. If-Else Ambiguity Statement  IfStatement | Assignment
IfStatement  if ( Expression ) Statement |
if ( Expression ) Statement else Statement
Assignment  Identifier = Expression ;
Tokens read so far:
if ( Identifier < Identifier ) if ( Identifier < Identifier ) Identifier = Number ;
if(a < b)
if(d < g)
x = 5;
else
x = 6000;
Reduced to:
if ( Expression ) if ( Expression) Statement
Next token:
else
COMP 640 – CWB

25. Java’s Solution to the If-else Ambiguity
IfThenStatement  if ( Expression ) Statement
IfThenElseStatement  if ( Expression ) StatementNoShortIf else Statement
StatementNoShortIf  list all statements except IfThenStatement
COMP 640 – CWB

26. Examples of Grammar from the Java Language Spec
TOC:
If statement:
“No Short If Statement”:
While statement:
COMP 640 – CWB

27. Java’s if-else Disambiguation
See next slide
Statement  StatementWithoutTrailingSubstatement \ LabeledStatement \ IfThenStatement \ IfThenElseStatement \ WhileStatement \ ForStatement
IfThenStatement  if ( Expression ) Statement
IfThenElseStatement  if ( Expression ) StatementNoShortIf else Statement
StatementNoShortIf  StatementWithoutTrailingSubstatement |
LabeledStatementNoShortIf | IfThenElseStatementNoShortIf |
WhileStatementNoShortIf | ForStatementNoShortIf
WhileStatement  while ( Expression ) Statement
WhileStatementNoShortIf  while ( Expression ) StatementNoShortIf
COMP 640 – CWB

28. Statement Without Trailing Substatement
StatementWithoutTrailingSubstatement  Block | EmptyStatement | ExpressionStatement | SwitchStatement | DoStatement | BreakStatement | ContinueStatement | ReturnStatement | SynchronizedStatement | ThrowStatement | TryStatement
COMP 640 – CWB

29. Java Generic Types Generic methods introduce issue:
Vector<String> a = new Vector<String>();
Vector<Vector<String>> b =
new Vector<Vector<String>>();
Generic methods introduce issue:
method_invocation ::= … |
type_arguments name LPAREN argument_list_opt
RPAREN
A((B)<C,D>E());
COMP 640 – CWB

30. Jay Data types: int and boolean
Statements: assignment, if-else, while, and blocks
NO: functions, reals, strings, arrays, I/O
Turing complete
Concrete syntax (Noonan’s site):
COMP 640 – CWB

31. Next Week Basic semantics Parsing
Read Chapter 3 Intro (not assigned in Syllabus)
Read Section 2.3
Read Wikipedia articles (not assigned in Syllabus)
COMP 640 – CWB