1. Fangfang Cui Mar. 29, 2016 1

2. Overview 1. LL(k) 1. LL(k) Definition 2. LL(1) 3. Using a Parsing Table 4. First Sets 5. Follow Sets 6. Building a Parsing Table 2. ANTLR 1. Definition 2. History 3. Main Customers 4. ANTLR Introduction 5. ANTLR Example 3. Works Cited 2

3. LL(k) “A top-down parser for a subset of context-free languages. It parses the input from Left to right, performing Leftmost derivation of the sentence” (“LL Parser”). LL(k) parser looks ahead k tokens when parsing a sentence. 3

4. LL(1) “Recursive descent parsers needing no backtracking, can be constructed for a class of grammars called LL(1)”(Aho). 1 means one input symbol of look ahead at each step to make parsing action decisions. 4

5. Problems with LL(1) Grammar (Aiken) E  T+E|T T  int|int*T|(E) Problems: 1. For E: two predictions begins with T 2. For T: two predictions begins with int 5

6. Solution Left factor the grammar, eliminate common prefixes. E  TX X  +E|ε T  int Y|(E) Y  *T|ε 6

7. LL(1) Parsing Table int*+()$ ETX X+Eεε Tint Y(E) Y*Tεεε Column Header: Next Input Token Row Header: Leftmost Non-terminal Note: [Y, )] entry: Y can only be followed by ) if and only if Y  ε, which means throw away Y, and go to next non-terminal [E, (] entry: use TX production [Y, int] entry: error entry, need to throw error message Production to use 7

8. How to Use a Parsing Table Method 1. For the left most non-terminal S 2. Look at the next input token a 3. Choose production shown at [S, a] A stack records frontier of parse tree Non-terminals that have to be expanded Terminals that have to be matched against the input Top of stack is the left most pending terminal or non- terminal Reject on reaching error state Accept on end of input and empty state 8

9. Parsing Process Initialize stack and next Repeat Case stack of : If T[X, *next] = Y1….Yn Then stack  ; Else error(); : If t== *next++ Then stack  Else error() Until stack == <> 9

10. Example: Use A Parsing Table StackInputAction E$Int * int $TX 10

11. Example Answer StackInputAction E $Int * int $T X T X $Int * int $Int Y Int Y X $int * int $Terminal Y X $* int $* T* T * T X $* int $Terminal T X $int $Int Y Int Y X $int $Terminal Y X $$ε X $$ε $$Accept 11

12. How to Build a Parsing Table 12

13. First Sets 13

14. First Sets Example E  T X X  +E | ε T  (E) | int YY  *T | ε First (+) = First(E) = First (*) =First(T) = First (() = First(X) = First ()) = First(Y) = First (int) = 14

15. First Sets Answer 15

16. Follow Sets 16

17. Follow Sets 17

18. Follow Sets Example E  T X X  +E | ε T  (E) | int YY  *T | ε Follow (+) = Follow(E) = Follow (*) =Follow(T) = Follow (() = Follow(X) = Follow ()) = Follow(Y) = Follow (int) = 18

19. Follow Sets Example Answer 19

20. Construct a Parsing Table 20

21. Example: build a parsing table E  T X X  +E | ε T  (E) | int YY  *T | ε int*+()$ E X T Y 21

22. Example - Answer int*+()$ ETX X+Eεε Tint Y(E) Y*Tεεε E  T X X  +E | ε T  (E) | int YY  *T | ε 22

23. 23

24. ANTLR Introduction ANTLR: Another Tool for Language Recognition Definition: “A tool that accepts grammatical language descriptions and generates programs that recognize sentences in target languages” (ANTLR). 24

25. History ANTLR was first developed in 1989 by Terence Parr, professor of the University of San Francisco (ANTLR). It is included on all Linux and OS X distributions. Fig. 1: http://www.amazon.com/Terence-Parr/e/B001JS3O0U Fig. 1 Terence Parr 25

26. Main Customers Tweeter Hive PigHadoop Oracle NetBeans IDE HQL 26

27. ANTLR Information Flow PARSE TREE WALKER Target Language Recognizers 27

28. ANTLR Introduction ANTLR  Input: Language description using Extended Backus-Naur Form (EBNF) grammar.  Output: recognizer for the language ANTLR build recognizers for three kinds of input:  Character streams (Scanner)  Token streams (Parser)  Node streams (Tree walker) ANTLR 4  LL(*) compiler generator  Generates recognizers in C#, Java, JavaScript, Python2, Python3 28

29. ANTLR Introduction ANTLR generates LL(k) or LL(*) recognizers  Computes first and follow sets  Verifies syntax conditions  Generates scanner/lexer using predictive recursive- descent method Note: LL(*) changes “from conventional fixed k ≥ 1 look ahead to arbitrary look ahead and, finally, fail over to backtracking depending on the complexity of the parsing decision and the input symbols” (Parr, “LL(*)”). 29

30. Example Source Program (Hello.g4) HelloLexer.java HelloParser.java HelloListener.java Token Streams Tree Node Streams Character Streams Target Code Lexer Parser Tree Walker 30

31. Extra Example Calculator Example 31

32. Works Cited 1. Aho, Alfred. “Compilers: Principals, Techniques, and Tools”. Boston: Pearson, 2007. 2. Aiken, Alex. “Compilers”. coursera. 18 Mar. 2016.. 3. “LL Parser”. The Free Encyclopedia. 17 Mar. 2016. Wikimedia Foundation Inc. 22 Mar. 2016. 4. Parr, Terence. “ANTLR”. ANTLR. 15 Mar. 2016.. 5. Parr, Terence. “LL(*): The Foundation of the ANTLR Parser Generator”. ANTLR. 26 Mar. 2016.. 6. Parr, Terence. “Getting Started with ANTLR v4”. GitHub. 22 Mar. 2016.. 7. Shmatov, Roman. “ANTLR4 Calculator”. GitHub. 22 Mar. 2016.. 32

33. Thank you. 33