1. FACS Week 14 Grammars and their properties; Introduction to Parsing Lee McCluskey, room 2/07 Email lee@hud.ac.uklee@hud.ac.uk

2. GRAMMARS Theoretical properties and results about the nature of GRAMMARS have been used for 40 years to influence the design of PARSERS. A CONTEXT FREE (BNF) GRAMMAR contains a vocabulary = {terminals, non-terminals} and productions of the form: non-terminal ::= v1 v2.........vn where the vi are members of the vocabulary. One non-terminal is called the SPECIAL SYMBOL

3. PROPERTIES OF GRAMMARS -1 A syntax tree of a BNF Grammar G is a tree where - the root is the special symbol of G, - the leaves are terminals of G, - the nodes are non-terminals of G, - each node is the LHS of some production P of G in which the node’s children form the RHS of P

4. PROPERTIES OF GRAMMARS -2 A string is in the language generated by a BNF grammar G IFF it forms the leaves (in the correct order) of some syntax tree generated by G G is AMBIGUOUS IFF at least one of the strings in its language has more than one distinct syntax tree

5. PROPERTIES OF GRAMMARS -3 A BNF grammar G is LEFT-RECURSIVE if has at least one production of the form: X ::= X w where X is a non-terminal and w is a string of symbols in G’s vocabulary

6. PROPERTIES OF GRAMMARS -4 If w is a list of symbols in the vocabulary of BNF grammar G then First(w) = set of TERMINAL symbols that may be at the front of ANY string derived from w using G’s productions

7. PROPERTIES OF GRAMMARS -5 If X is a non-terminal of BNF grammar G then Follow(X) = set of TERMINAL symbols that can follow X in a derivation from G’s special symbol using G’s productions. Nullable(X) is true IFF the empty word can be derived from X

8. PARSING A PARSER (or ‘PARSING ALGORITHM’) derives SYNTAX/PARSE TREES from a sequence of TOKENS Some well-known tools are little more than PARSERS e.g. Syntax Directed Editors - those that colour different syntax classes or point out syntax errors as you make them PARSING ALGORITHMS invariably are based on the CONTEXT FREE GRAMMAR that defines the language being parsed.

9. Example Input String: b = 5; a = 5 * b; PRINT(b * a) Ouput PARSE TREE (as a Java data structure) new CompoundStm( new AssignStm("b",new NumExp(5)), new CompoundStm( new AssignStm("a", new OpExp(new NumExp(5), OpExp.Times, new IdExp("b"))), new PrintStm(new LastExpList(new OpExp(new IdExp("b"), OpExp.Times, new IdExp("a")))) ) );

10. Example Input String: b = 5; a = 5 * b; PRINT(b * a) Ouput PARSE TREE (as a Prolog data structure) compoundStm(assignStm("b",numExp(5)), compoundStm( assignStm("a", opExp(numExp(5),opExp(times),idExp("b"))), printStm(lastExpList(opExp(idExp("b"),opExp(times),idExp("a")))) )). Eg | ?- compoundStm(X,compoundStm(Y,Z)). X = assignStm([98],numExp(5)), Y = assignStm([97],opExp(numExp(5),opExp(times),idExp([98]))), Z = printStm(lastExpList(opExp(idExp([98]),opExp(times),idExp([97])))) ? yes | ?-

11. PARSING ALGORITHMS Parsers that follow the language’s grammar can be TOP DOWN: - starting with the grammar’s special symbol, try to find a derivation of the string of tokens being parsed, consuming one token at a time BOTTOM UP: - start with the string viewed as a stack. Match the top ‘n’ tokens of the stack with the RHS of a grammar rule P and replace those tokens with the LHS of P

12. PRACTICAL EXAMPLE OF THEORY A “LL(1) PARSING TABLE” (-like a program) can be automatically derived from a grammar G if and only if (i) G is NOT ambiguous (ii) G is NOT left recursive (iii) for EVERY two of G’s productions of the form X ::= W1, X ::= W2, it is the case that First(W1) and First(W2) have no common element

13. SUMMARY Parsers check strings are legal according to grammatical definitions, and build up a structure representing legal strings (parse trees) Parsers can sometimes be auto-generated from the defining grammar Theory of grammars helps us in the auto- generation of parsers