Top-Down Parsing - recursive descent - predictive parsing

Slides:



Advertisements
Similar presentations
Chap. 5, Top-Down Parsing J. H. Wang Mar. 29, 2011.
Advertisements

YANGYANG 1 Chap 5 LL(1) Parsing LL(1) left-to-right scanning leftmost derivation 1-token lookahead parser generator: Parsing becomes the easiest! Modifying.
Mooly Sagiv and Roman Manevich School of Computer Science
Top-Down Parsing.
CS 310 – Fall 2006 Pacific University CS310 Parsing with Context Free Grammars Today’s reference: Compilers: Principles, Techniques, and Tools by: Aho,
By Neng-Fa Zhou Syntax Analysis lexical analyzer syntax analyzer semantic analyzer source program tokens parse tree parser tree.
CS Summer 2005 Top-down and Bottom-up Parsing - a whirlwind tour June 20, 2005 Slide acknowledgment: Radu Rugina, CS 412.
COS 320 Compilers David Walker. last time context free grammars (Appel 3.1) –terminals, non-terminals, rules –derivations & parse trees –ambiguous grammars.
Prof. Bodik CS 164 Lecture 61 Building a Parser II CS164 3:30-5:00 TT 10 Evans.
Prof. Fateman CS 164 Lecture 8 1 Top-Down Parsing CS164 Lecture 8.
1 Predictive parsing Recall the main idea of top-down parsing: Start at the root, grow towards leaves Pick a production and try to match input May need.
CS 310 – Fall 2006 Pacific University CS310 Parsing with Context Free Grammars Today’s reference: Compilers: Principles, Techniques, and Tools by: Aho,
Prof. Fateman CS 164 Lecture 91 Bottom-Up Parsing Lecture 9.
1 The Parser Its job: –Check and verify syntax based on specified syntax rules –Report errors –Build IR Good news –the process can be automated.
1 Chapter 4: Top-Down Parsing. 2 Objectives of Top-Down Parsing an attempt to find a leftmost derivation for an input string. an attempt to construct.
Professor Yihjia Tsai Tamkang University
COS 320 Compilers David Walker. last time context free grammars (Appel 3.1) –terminals, non-terminals, rules –derivations & parse trees –ambiguous grammars.
Top-Down Parsing.
– 1 – CSCE 531 Spring 2006 Lecture 7 Predictive Parsing Topics Review Top Down Parsing First Follow LL (1) Table construction Readings: 4.4 Homework: Program.
CPSC 388 – Compiler Design and Construction
COP4020 Programming Languages Computing LL(1) parsing table Prof. Xin Yuan.
Parsing Chapter 4 Parsing2 Outline Top-down v.s. Bottom-up Top-down parsing Recursive-descent parsing LL(1) parsing LL(1) parsing algorithm First.
Review: –How do we define a grammar (what are the components in a grammar)? –What is a context free grammar? –What is the language defined by a grammar?
4 4 (c) parsing. Parsing A grammar describes the strings of tokens that are syntactically legal in a PL A recogniser simply accepts or rejects strings.
Chapter 5 Top-Down Parsing.
4 4 (c) parsing. Parsing A grammar describes syntactically legal strings in a language A recogniser simply accepts or rejects strings A generator produces.
# 1 CMPS 450 Parsing CMPS 450 J. Moloney. # 2 CMPS 450 Check that input is well-formed Build a parse tree or similar representation of input Recursive.
Parsing Jaruloj Chongstitvatana Department of Mathematics and Computer Science Chulalongkorn University.
-Mandakinee Singh (11CS10026).  What is parsing? ◦ Discovering the derivation of a string: If one exists. ◦ Harder than generating strings.  Two major.
Profs. Necula CS 164 Lecture Top-Down Parsing ICOM 4036 Lecture 5.
Compiler Parsing 邱锡鹏 复旦大学计算机科学技术学院
Lesson 5 CDT301 – Compiler Theory, Spring 2011 Teacher: Linus Källberg.
CSI 3120, Syntactic analysis, page 1 Syntactic Analysis and Parsing Based on A. V. Aho, R. Sethi and J. D. Ullman Compilers: Principles, Techniques and.
4 4 (c) parsing. Parsing A grammar describes syntactically legal strings in a language A recogniser simply accepts or rejects strings A generator produces.
COP4020 Programming Languages Parsing Prof. Xin Yuan.
Parsing Top-Down.
Transparency No. 1 Lecture 3 Introduction to Parsing and Top-Down Parsing Cheng-Chia Chen.
Top-down Parsing Recursive Descent & LL(1) Comp 412 Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled in Comp 412.
Top-Down Parsing CS 671 January 29, CS 671 – Spring Where Are We? Source code: if (b==0) a = “Hi”; Token Stream: if (b == 0) a = “Hi”; Abstract.
TOP-DOWN PARSING Recursive-Descent, Predictive Parsing.
1 Context free grammars  Terminals  Nonterminals  Start symbol  productions E --> E + T E --> E – T E --> T T --> T * F T --> T / F T --> F F --> (F)
1 Nonrecursive Predictive Parsing  It is possible to build a nonrecursive predictive parser  This is done by maintaining an explicit stack.
1 Compiler Construction (CS-636) Muhammad Bilal Bashir UIIT, Rawalpindi.
Top-Down Parsing.
CSE 5317/4305 L3: Parsing #11 Parsing #1 Leonidas Fegaras.
Top-Down Predictive Parsing We will look at two different ways to implement a non- backtracking top-down parser called a predictive parser. A predictive.
Parsing methods: –Top-down parsing –Bottom-up parsing –Universal.
1 Introduction to Parsing. 2 Outline l Regular languages revisited l Parser overview Context-free grammars (CFG ’ s) l Derivations.
COMP 3438 – Part II-Lecture 5 Syntax Analysis II Dr. Zili Shao Department of Computing The Hong Kong Polytechnic Univ.
1 Topic #4: Syntactic Analysis (Parsing) CSC 338 – Compiler Design and implementation Dr. Mohamed Ben Othman ( )
Chapter 2 (part) + Chapter 4: Syntax Analysis S. M. Farhad 1.
UMBC  CSEE   1 Chapter 4 Chapter 4 (b) parsing.
COMP 3438 – Part II-Lecture 6 Syntax Analysis III Dr. Zili Shao Department of Computing The Hong Kong Polytechnic Univ.
Fangfang Cui Mar. 29, Overview 1. LL(k) 1. LL(k) Definition 2. LL(1) 3. Using a Parsing Table 4. First Sets 5. Follow Sets 6. Building a Parsing.
Spring 16 CSCI 4430, A Milanova 1 Announcements HW1 due on Monday February 8 th Name and date your submission Submit electronically in Homework Server.
CMSC 330: Organization of Programming Languages Pushdown Automata Parsing.
Transparency No. 1 Lecture 3 Introduction to Parsing and Top-Down Parsing Cheng-Chia Chen.
Grammars, Derivations and Parsing. Sample Grammar Simple arithmetic expressions (E) Basis Rules: –A Variable is an E –An Integer is an E Inductive Rules:
Programming Languages Translator
Context free grammars Terminals Nonterminals Start symbol productions
Top-down parsing cannot be performed on left recursive grammars.
Parsing with Context Free Grammars
CS 404 Introduction to Compiler Design
Top-Down Parsing.
4 (c) parsing.
Compiler Design 7. Top-Down Table-Driven Parsing
Ambiguity, Precedence, Associativity & Top-Down Parsing
Syntax Analysis - Parsing
Nonrecursive Predictive Parsing
Compilers Principles, Techniques, & Tools Taught by Jing Zhang
Presentation transcript:

Top-Down Parsing - recursive descent - predictive parsing Lecture 9 CS 536 Spring 2001

Implementation of parsers Two approaches Lecture Outline Implementation of parsers Two approaches Top-down Bottom-up Today: Top-Down Easier to understand and program manually Then: Bottom-Up More powerful and used by most parser generators CS 536 Spring 2001

Intro to Top-Down Parsing The parse tree is constructed From the top From left to right Terminals are seen in order of appearance in the token stream: t2 t5 t6 t8 t9 1 t2 3 4 t5 7 t6 t9 t8 CS 536 Spring 2001

Recursive Descent Parsing Consider the grammar E  T + E | T T  int | int * T | ( E ) Token stream is: int5 * int2 Start with top-level non-terminal E Try the rules for E in order CS 536 Spring 2001

Recursive Descent Parsing. Example (Cont.) Try E0  T1 + E2 Then try a rule for T1  ( E3 ) But ( does not match input token int5 Try T1  int . Token matches. But + after T1 does not match input token * Try T1  int * T2 This will match but + after T1 will be unmatched Has exhausted the choices for T1 Backtrack to choice for E0 CS 536 Spring 2001

Recursive Descent Parsing. Example (Cont.) Try E0  T1 Follow same steps as before for T1 And succeed with T1  int * T2 and T2  int With the following parse tree E0 T1 int5 * T2 int2 CS 536 Spring 2001

A Recursive Descent Parser. Preliminaries Let TOKEN be the type of tokens Special tokens INT, OPEN, CLOSE, PLUS, TIMES Let the global next point to the next token CS 536 Spring 2001

A Recursive Descent Parser (2) Define boolean functions that check the token string for a match of A given token terminal bool term(TOKEN tok) { return *next++ == tok; } A given production of S (the nth) bool Sn() { … } Any production of S: bool S() { … } These functions advance next CS 536 Spring 2001

A Recursive Descent Parser (3) For production E  T bool E1() { return T(); } For production E  T + E bool E2() { return T() && term(PLUS) && E(); } For all productions of E (with backtracking) bool E() { TOKEN *save = next; return (next = save, E1()) || (next = save, E2()); } CS 536 Spring 2001

A Recursive Descent Parser (4) Functions for non-terminal T bool T1() { return term(OPEN) && E() && term(CLOSE); } bool T2() { return term(INT) && term(TIMES) && T(); } bool T3() { return term(INT); } bool T() { TOKEN *save = next; return (next = save, T1()) || (next = save, T2()) || (next = save, T3()); } CS 536 Spring 2001

Recursive Descent Parsing. Notes. To start the parser Initialize next to point to first token Invoke E() Notice how this simulates our previous example Easy to implement by hand But does not always work … CS 536 Spring 2001

When Recursive Descent Does Not Work Consider a production S  S a bool S1() { return S() && term(a); } bool S() { return S1(); } S() will get into an infinite loop A left-recursive grammar has a non-terminal S S + S for some  Recursive descent does not work in such cases CS 536 Spring 2001

Elimination of Left Recursion Consider the left-recursive grammar S  S  |  S generates all strings starting with a  and followed by a number of  Can rewrite using right-recursion S   S’ S’   S’ |  CS 536 Spring 2001

More Elimination of Left-Recursion In general S  S 1 | … | S n | 1 | … | m All strings derived from S start with one of 1,…,m and continue with several instances of 1,…,n Rewrite as S  1 S’ | … | m S’ S’  1 S’ | … | n S’ |  CS 536 Spring 2001

General Left Recursion The grammar S  A  |  A  S  is also left-recursive because S + S   This left-recursion can also be eliminated See book, Section 4.3 for general algorithm CS 536 Spring 2001

Summary of Recursive Descent Simple and general parsing strategy Left-recursion must be eliminated first … but that can be done automatically Unpopular because of backtracking Thought to be too inefficient In practice, backtracking is eliminated by restricting the grammar CS 536 Spring 2001

Predictive parsers accept LL(k) grammars Like recursive-descent but parser can “predict” which production to use By looking at the next few tokens No backtracking Predictive parsers accept LL(k) grammars L means “left-to-right” scan of input L means “leftmost derivation” k means “predict based on k tokens of lookahead” In practice, LL(1) is used CS 536 Spring 2001

Can be specified via 2D tables LL(1) Languages In recursive-descent, for each non-terminal and input token there may be a choice of production LL(1) means that for each non-terminal and token there is only one production Can be specified via 2D tables One dimension for current non-terminal to expand One dimension for next token A table entry contains one production CS 536 Spring 2001

Predictive Parsing and Left Factoring Recall the grammar E  T + E | T T  int | int * T | ( E ) Hard to predict because For T two productions start with int For E it is not clear how to predict A grammar must be left-factored before use for predictive parsing CS 536 Spring 2001

Left-Factoring Example Recall the grammar E  T + E | T T  int | int * T | ( E ) Factor out common prefixes of productions E  T X X  + E |  T  ( E ) | int Y Y  * T |  CS 536 Spring 2001

LL(1) Parsing Table Example Left-factored grammar E  T X X  + E |  T  ( E ) | int Y Y  * T |  The LL(1) parsing table: int * + ( ) $ E T X X + E  T int Y ( E ) Y * T CS 536 Spring 2001

LL(1) Parsing Table Example (Cont.) Consider the [E, int] entry “When current non-terminal is E and next input is int, use production E  T X This production can generate an int in the first place Consider the [Y,+] entry “When current non-terminal is Y and current token is +, get rid of Y” Y can be followed by + only in a derivation in which Y   CS 536 Spring 2001

LL(1) Parsing Tables. Errors Blank entries indicate error situations Consider the [E,*] entry “There is no way to derive a string starting with * from non-terminal E” CS 536 Spring 2001

Method similar to recursive descent, except Using Parsing Tables Method similar to recursive descent, except For each non-terminal S We look at the next token a And chose the production shown at [S,a] We use a stack to keep track of pending non-terminals We reject when we encounter an error state We accept when we encounter end-of-input CS 536 Spring 2001

LL(1) Parsing Algorithm initialize stack = <S $> and next repeat case stack of <X, rest> : if T[X,*next] = Y1…Yn then stack  <Y1… Yn rest>; else error (); <t, rest> : if t == *next ++ then stack  <rest>; else error (); until stack == < > CS 536 Spring 2001

LL(1) Parsing Example Stack Input Action E $ int * int $ T X T X $ int * int $ int Y int Y X $ int * int $ terminal Y X $ * int $ * T * T X $ * int $ terminal T X $ int $ int Y int Y X $ int $ terminal Y X $ $  X $ $  $ $ ACCEPT CS 536 Spring 2001

Constructing Parsing Tables LL(1) languages are those defined by a parsing table for the LL(1) algorithm No table entry can be multiply defined We want to generate parsing tables from CFG CS 536 Spring 2001

Constructing Parsing Tables (Cont.) If A  , where in the line of A we place  ? In the column of t where t can start a string derived from   * t  We say that t  First() In the column of t if  is  and t can follow an A S *  A t  We say t  Follow(A) CS 536 Spring 2001

Computing First Sets Definition: First(X) = { t | X * t}  { | X * } Algorithm sketch (see book for details): for all terminals t do First(t)  { t } for each production X   do First(X)  {  } if X  A1 … An  and   First(Ai), 1  i  n do add First() to First(X) for each X  A1 … An s.t.   First(Ai), 1  i  n do add  to First(X) repeat steps 4 & 5 until no First set can be grown CS 536 Spring 2001

First( ( ) = { ( } First( T ) = {int, ( } First Sets. Example Recall the grammar E  T X X  + E |  T  ( E ) | int Y Y  * T |  First sets First( ( ) = { ( } First( T ) = {int, ( } First( ) ) = { ) } First( E ) = {int, ( } First( int) = { int } First( X ) = {+,  } First( + ) = { + } First( Y ) = {*,  } First( * ) = { * } CS 536 Spring 2001

Follow(X) = { t | S *  X t  } Intuition Computing Follow Sets Definition: Follow(X) = { t | S *  X t  } Intuition If S is the start symbol then $  Follow(S) If X  A B then First(B)  Follow(A) and Follow(X)  Follow(B) Also if B *  then Follow(X)  Follow(A) CS 536 Spring 2001

Computing Follow Sets (Cont.) Algorithm sketch: Follow(S)  { $ } For each production A   X  add First() - {} to Follow(X) For each A   X  where   First() add Follow(A) to Follow(X) repeat step(s) ___ until no Follow set grows CS 536 Spring 2001

Follow( + ) = { int, ( } Follow( * ) = { int, ( } Follow Sets. Example Recall the grammar E  T X X  + E |  T  ( E ) | int Y Y  * T |  Follow sets Follow( + ) = { int, ( } Follow( * ) = { int, ( } Follow( ( ) = { int, ( } Follow( E ) = {), $} Follow( X ) = {$, ) } Follow( T ) = {+, ) , $} Follow( ) ) = {+, ) , $} Follow( Y ) = {+, ) , $} Follow( int) = {*, +, ) , $} CS 536 Spring 2001

Constructing LL(1) Parsing Tables Construct a parsing table T for CFG G For each production A   in G do: For each terminal t  First() do T[A, t] =  If   First(), for each t  Follow(A) do If   First() and $  Follow(A) do T[A, $] =  CS 536 Spring 2001

Notes on LL(1) Parsing Tables If any entry is multiply defined then G is not LL(1) If G is ambiguous If G is left recursive If G is not left-factored And in other cases as well Most programming language grammars are not LL(1) There are tools that build LL(1) tables CS 536 Spring 2001

For some grammars there is a simple parsing strategy Review For some grammars there is a simple parsing strategy Predictive parsing Next time: a more powerful parsing strategy CS 536 Spring 2001