Fangfang Cui Mar. 29,

Overview 1. LL(k) 1. LL(k) Definition 2. LL(1) 3. Using a Parsing Table 4. First Sets 5. Follow Sets 6. Building a Parsing Table 2. ANTLR 1. Definition 2. History 3. Main Customers 4. ANTLR Introduction 5. ANTLR Example 3. Works Cited 2

LL(k) “A top-down parser for a subset of context-free languages. It parses the input from Left to right, performing Leftmost derivation of the sentence” (“LL Parser”). LL(k) parser looks ahead k tokens when parsing a sentence. 3

LL(1) “Recursive descent parsers needing no backtracking, can be constructed for a class of grammars called LL(1)”(Aho). 1 means one input symbol of look ahead at each step to make parsing action decisions. 4

Problems with LL(1) Grammar (Aiken) E  T+E|T T  int|int*T|(E) Problems: 1. For E: two predictions begins with T 2. For T: two predictions begins with int 5

Solution Left factor the grammar, eliminate common prefixes. E  TX X  +E|ε T  int Y|(E) Y  *T|ε 6

LL(1) Parsing Table int*+()$ ETX X+Eεε Tint Y(E) Y*Tεεε Column Header: Next Input Token Row Header: Leftmost Non-terminal Note: [Y, )] entry: Y can only be followed by ) if and only if Y  ε, which means throw away Y, and go to next non-terminal [E, (] entry: use TX production [Y, int] entry: error entry, need to throw error message Production to use 7

How to Use a Parsing Table Method 1. For the left most non-terminal S 2. Look at the next input token a 3. Choose production shown at [S, a] A stack records frontier of parse tree Non-terminals that have to be expanded Terminals that have to be matched against the input Top of stack is the left most pending terminal or non- terminal Reject on reaching error state Accept on end of input and empty state 8

Parsing Process Initialize stack and next Repeat Case stack of : If T[X, *next] = Y1….Yn Then stack  ; Else error(); : If t== *next++ Then stack  Else error() Until stack == <> 9

Example: Use A Parsing Table StackInputAction E$Int * int $TX 10

Example Answer StackInputAction E $Int * int $T X T X $Int * int $Int Y Int Y X $int * int $Terminal Y X $* int $* T* T * T X $* int $Terminal T X $int $Int Y Int Y X $int $Terminal Y X $$ε X $$ε $$Accept 11

How to Build a Parsing Table 12

First Sets 13

First Sets Example E  T X X  +E | ε T  (E) | int YY  *T | ε First (+) = First(E) = First (*) =First(T) = First (() = First(X) = First ()) = First(Y) = First (int) = 14

First Sets Answer 15

Follow Sets 16

Follow Sets 17

Follow Sets Example E  T X X  +E | ε T  (E) | int YY  *T | ε Follow (+) = Follow(E) = Follow (*) =Follow(T) = Follow (() = Follow(X) = Follow ()) = Follow(Y) = Follow (int) = 18

Follow Sets Example Answer 19

Construct a Parsing Table 20

Example: build a parsing table E  T X X  +E | ε T  (E) | int YY  *T | ε int*+()$ E X T Y 21

Example - Answer int*+()$ ETX X+Eεε Tint Y(E) Y*Tεεε E  T X X  +E | ε T  (E) | int YY  *T | ε 22

23

ANTLR Introduction ANTLR: Another Tool for Language Recognition Definition: “A tool that accepts grammatical language descriptions and generates programs that recognize sentences in target languages” (ANTLR). 24

History ANTLR was first developed in 1989 by Terence Parr, professor of the University of San Francisco (ANTLR). It is included on all Linux and OS X distributions. Fig. 1: Fig. 1 Terence Parr 25

Main Customers Tweeter Hive PigHadoop Oracle NetBeans IDE HQL 26

ANTLR Information Flow PARSE TREE WALKER Target Language Recognizers 27

ANTLR Introduction ANTLR  Input: Language description using Extended Backus-Naur Form (EBNF) grammar.  Output: recognizer for the language ANTLR build recognizers for three kinds of input:  Character streams (Scanner)  Token streams (Parser)  Node streams (Tree walker) ANTLR 4  LL(*) compiler generator  Generates recognizers in C#, Java, JavaScript, Python2, Python3 28

ANTLR Introduction ANTLR generates LL(k) or LL(*) recognizers  Computes first and follow sets  Verifies syntax conditions  Generates scanner/lexer using predictive recursive- descent method Note: LL(*) changes “from conventional fixed k ≥ 1 look ahead to arbitrary look ahead and, finally, fail over to backtracking depending on the complexity of the parsing decision and the input symbols” (Parr, “LL(*)”). 29

Example Source Program (Hello.g4) HelloLexer.java HelloParser.java HelloListener.java Token Streams Tree Node Streams Character Streams Target Code Lexer Parser Tree Walker 30

Extra Example Calculator Example 31

Works Cited 1. Aho, Alfred. “Compilers: Principals, Techniques, and Tools”. Boston: Pearson, Aiken, Alex. “Compilers”. coursera. 18 Mar “LL Parser”. The Free Encyclopedia. 17 Mar Wikimedia Foundation Inc. 22 Mar Parr, Terence. “ANTLR”. ANTLR. 15 Mar Parr, Terence. “LL(*): The Foundation of the ANTLR Parser Generator”. ANTLR. 26 Mar Parr, Terence. “Getting Started with ANTLR v4”. GitHub. 22 Mar Shmatov, Roman. “ANTLR4 Calculator”. GitHub. 22 Mar

Thank you. 33