Winter 2003/4Pls – syntax – Catriel Beeri1 SYNTAX Syntax: form, structure The syntax of a pl: The set of its well-formed programs The rules that define these programs Two views: Concrete syntax: program as text Abstract syntax: program as composite structure, a tree

Winter 2003/4Pls – syntax – Catriel Beeri2 Concrete syntax The common view – program as text (a string) Common practice in compilers – divide into two levels Lexical structure - the words מבנה מילוני –Lexical specification / analysis ניתוח מילוני מפרט, Phrase structure – the sentences מבנה תחבירי –Phrase structure specification / parsing מפרט, ניתוח תחבירי

Winter 2003/4Pls – syntax – Catriel Beeri3 Lexical A word: lexeme מילה A class of words: token אסימון for example: int, ident, real, leftpar, if …  (real, 2.3) (4+5)  leftpar (int, 4) plus (int, 5) rightpar Lexical analysis – convert text to (token, lexeme) - stream

Winter 2003/4Pls – syntax – Catriel Beeri4 Lexical analysis: implementaion Token specified by regular expression Regular expression  (ndet) finite automaton,  (det) finite automaton  a program – a lexical analyzer Issues: Many tokens Where to stop …..

Winter 2003/4Pls – syntax – Catriel Beeri5 Phrase structure/analysis Specified by context free grammar (CFG) (BNF --- Backus-Naur form) T – terminals (here, tokens – sets of lexemes ) N – non-terminals = names of syntactical categories P –production rules Rule: A  w (w is a string on N T) S – start non-terminal A CFG as a generative device: Start from S Replace non-terminals by strings, using rules

Winter 2003/4Pls – syntax – Catriel Beeri6 Example: CFG for simple arithmetic expressions T = {int, op}, N = {E}, S = E Rules: E ::= int | E op E (2 rules, | means `or’) Generation by a derivation: E op E => E => int op E => int op E op E => int op int op E => int op int op int Could represent the expression

Winter 2003/4Pls – syntax – Catriel Beeri7 Here are two derivations: Cont’d E op E => E => int op E => E op E op E => int op E op E => int op int op E =>int op int op int Are they really different? And from both:

Winter 2003/4Pls – syntax – Catriel Beeri8 A derivation corresponds to a derivation tree: * E => E op E => E op E op E => int op E op E => E EEop E E int E => E op E =>int op E => int op int op E => int op into op int

Winter 2003/4Pls – syntax – Catriel Beeri9 Derivations vs. derivation trees A derivation tree represents many derivations If there is a word with several derivation trees, the CFG is ambiguous. Example: E E E op E E int E op E E E E int

Winter 2003/4Pls – syntax – Catriel Beeri10 The problem is addressed by: Adopting left associativity Allowing parentheses in expressions Changing the CFG: –New non-terminal T (for term) –New rules: E ::= E op T | T T ::= int | (E)

Winter 2003/4Pls – syntax – Catriel Beeri11 This CFG is unambiguous, and reflects left associativity E => E op T int op int op int

Winter 2003/4Pls – syntax – Catriel Beeri12 A derivation tree More complex than expression tree E Eop T ()E E T int T

Winter 2003/4Pls – syntax – Catriel Beeri13 Phrase structure -summary A language is specifiable by many CFG’s A CFG needs to address: –Ambiguity (avoid) –Associativity (express) –Precedence (express) –Efficient parsing (ensure) Methodologies for transforming CFG’s to account for the above are known The resulting CFG’s are complex; so are the derivation trees.

Winter 2003/4Pls – syntax – Catriel Beeri14 Abstract Syntax Consider: (if (< x 3) 4 7) (scheme) X < 3? 4 : 7 (C) (let ((x 5) (+ x 3) (scheme) let x = 5 in x + 3 (OCAML) Each pair is the “same” expression, same components The meaning is explained in same way: E.g., for the conditional: Evaluate the test if its value is true evaluate the 1 st branch, else evaluate the 2 nd

Winter 2003/4Pls – syntax – Catriel Beeri15 In abstract syntax: a program/expression is viewed as a labeled tree/ a compound structure A labeled leaf, represents an atomic phrase. label represents the category A larger tree represents a compound phrase –The root label is its category –The children are its components int (3)

Winter 2003/4Pls – syntax – Catriel Beeri16 Typical building blocks: Record: IfExpr test branch2 branch1 E3 E2 E1 IfExpr : {test = E1, branch1 = E2 branch3 = E3} Type can be expressed as an OCAML datatype type ifexpr = IfExpr of {test : expr; branch1 : expr; branch2 : expr}

Winter 2003/4Pls – syntax – Catriel Beeri17 Tuple: IfExpr E3 E2 E1 IfExpr : (E1, E2,E3) type ifexpr = IfExpr of expr * expr * expr Tuple vs. Record: field name vs. ordering

Winter 2003/4Pls – syntax – Catriel Beeri18 Sequence: CmpdStmt : (S1, S2, …, Sn) Tuple vs. sequence: In a tuple type, number of fields is known & fixed type cmpd_stmt = CmpdStmt of stmt list

Winter 2003/4Pls – syntax – Catriel Beeri19 Summary of abstract syntax Abstract syntax is the structure of the program keywords, separators, conventions - not included associativity, precedence, unambiguity - non-issues Parsing: convert from concrete to abstract syntax Type-checking, semantics, compiler translation use abstract syntax In rest of course: abstract syntax

Winter 2003/4Pls – syntax – Catriel Beeri20 Q: Can a cfg derivation tree serve as abstract syntax tree?

Winter 2003/4Pls – syntax – Catriel Beeri21 Syntax (concrete/abstract) is an inductive definition Example : E ::= int | id | E op E As rules: How will the rules look like for type expr = Int of int | Id of string | Expr of expr * exp ?

Winter 2003/4Pls – syntax – Catriel Beeri22 Common informal approach to abstract syntax specification Use a string CFG, interpret as a tree grammar Ignore keywords Labels and structures - left to reader to decide This shows the category, the components Sufficient for semantics Example : If-Expr ::= if Expr then Expr else Expr This is the approach in the course

Winter 2003/4Pls – syntax – Catriel Beeri23 A convention for abstract syntax Use variables, declare them before rules, omit indices Example : A similar convention often used for inductive definitions