Tomita‘s Parser Tihomira Panayotova Paolina Teneva Seminar für Sprachwissenschaft 31.01.2007.

Slides:



Advertisements
Similar presentations
Parsing V: Bottom-up Parsing
Advertisements

Bottom-up Parser Table Construction David Walker COS 320.
A question from last class: construct the predictive parsing table for this grammar: S->i E t S e S | i E t S | a E -> B.
Lesson 8 CDT301 – Compiler Theory, Spring 2011 Teacher: Linus Källberg.
Compiler construction in4020 – lecture 4 Koen Langendoen Delft University of Technology The Netherlands.
Compiler Principles Fall Compiler Principles Lecture 4: Parsing part 3 Roman Manevich Ben-Gurion University.
 CS /11/12 Matthew Rodgers.  What are LL and LR parsers?  What grammars do they parse?  What is the difference between LL and LR?  Why do.
Bottom up Parsing Bottom up parsing trys to transform the input string into the start symbol. Moves through a sequence of sentential forms (sequence of.
Pushdown Automata Consists of –Pushdown stack (can have terminals and nonterminals) –Finite state automaton control Can do one of three actions (based.
6/12/2015Prof. Hilfinger CS164 Lecture 111 Bottom-Up Parsing Lecture (From slides by G. Necula & R. Bodik)
Formal Aspects Term 2, Week4 LECTURE: LR “Shift-Reduce” Parsers: The JavaCup Parser-Generator CREATES LR “Shift-Reduce” Parsers, they are very commonly.
Lecture #8, Feb. 7, 2007 Shift-reduce parsing,
Prof. Fateman CS 164 Lecture 91 Bottom-Up Parsing Lecture 9.
1 LR parsing techniques SLR (not in the book) –Simple LR parsing –Easy to implement, not strong enough –Uses LR(0) items Canonical LR –Larger parser but.
1 Bottom-up parsing Goal of parser : build a derivation –top-down parser : build a derivation by working from the start symbol towards the input. builds.
Bottom-up parsing Goal of parser : build a derivation
Parsing IV Bottom-up Parsing Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved. Students enrolled in Comp 412 at Rice University.
Syntax and Semantics Structure of programming languages.
Chap. 6, Bottom-Up Parsing J. H. Wang May 17, 2011.
CS 321 Programming Languages and Compilers Bottom Up Parsing.
LANGUAGE TRANSLATORS: WEEK 17 scom.hud.ac.uk/scomtlm/cis2380/ See Appel’s book chapter 3 for support reading Last Week: Top-down, Table driven parsers.
Review 1.Lexical Analysis 2.Syntax Analysis 3.Semantic Analysis 4.Code Generation 5.Code Optimization.
Syntactic Analysis Natawut Nupairoj, Ph.D. Department of Computer Engineering Chulalongkorn University.
11 Outline  6.0 Introduction  6.1 Shift-Reduce Parsers  6.2 LR Parsers  6.3 LR(1) Parsing  6.4 SLR(1)Parsing  6.5 LALR(1)  6.6 Calling Semantic.
1 Syntax Analysis Part II Chapter 4 COP5621 Compiler Construction Copyright Robert van Engelen, Florida State University, 2005.
CS3230R. What is a parser? What is an LR parser? A bottom-up parser that efficiently handles deterministic context-free languages in guaranteed linear.
4. Bottom-up Parsing Chih-Hung Wang
Bernd Fischer RW713: Compiler and Software Language Engineering.
Bottom Up Parsing CS 671 January 31, CS 671 – Spring Where Are We? Finished Top-Down Parsing Starting Bottom-Up Parsing Lexical Analysis.
Bottom-Up Parsing Algorithms LR(k) parsing L: scan input Left to right R: produce Rightmost derivation k tokens of lookahead LR(0) zero tokens of look-ahead.
CS412/413 Introduction to Compilers and Translators Spring ’99 Lecture 6: LR grammars and automatic parser generators.
1 Syntax Analysis Part II Chapter 4 COP5621 Compiler Construction Copyright Robert van Engelen, Florida State University, 2007.
Conflicts in Simple LR parsers A SLR Parser does not use any lookahead The SLR parsing method fails if knowing the stack’s top state and next input token.
Parsing Bottom Up CMPS 450 J. Moloney CMPS 450.
Programming Languages Translator
Bottom-up parsing Goal of parser : build a derivation
Bottom-Up Parsing.
Lexical and Syntax Analysis
Unit-3 Bottom-Up-Parsing.
COP 4620 / 5625 Programming Language Translation / Compiler Writing Fall 2003 Lecture 5, 09/25/2003 Prof. Roy Levow.
Parsing IV Bottom-up Parsing
Table-driven parsing Parsing performed by a finite state machine.
Chapter 4 Syntax Analysis.
CS 404 Introduction to Compiler Design
Compiler Construction
Compiler design Bottom-up parsing: Canonical LR and LALR
Fall Compiler Principles Lecture 4: Parsing part 3
Bottom-Up Syntax Analysis
Syntax Analysis Part II
4 (c) parsing.
Top-Down Parsing.
Subject Name:COMPILER DESIGN Subject Code:10CS63
Regular Grammar - Finite Automaton
Lexical and Syntax Analysis
Parsing #2 Leonidas Fegaras.
Bottom Up Parsing.
Ambiguity in Grammar, Error Recovery
Parsing IV Bottom-up Parsing
LR Parsing. Parser Generators.
Parsing #2 Leonidas Fegaras.
5. Bottom-Up Parsing Chih-Hung Wang
Kanat Bolazar February 16, 2010
Parsing Bottom-Up.
Parsing Bottom-Up LR Table Construction.
Lecture 18 Bottom-Up Parsing or Shift-Reduce Parsing
Parsing Bottom-Up Introduction.
Compiler Construction
Parsing Bottom-Up LR Table Construction.
Chap. 3 BOTTOM-UP PARSING
Parsing CSCI 432 Computer Science Theory
Presentation transcript:

Tomita‘s Parser Tihomira Panayotova Paolina Teneva Seminar für Sprachwissenschaft

A simple overview LR(0) conflicts Tomita‘s method summarized Complications Two optimiziations A moderately ambiguous grammar Stack duplication Combinig equal states Combinig equal stack prefixes Discussion Summary

LR(0) Conflicts LR Parser: 1.handle recognizing FS automaton 2. no inadequate states There exist grammars for which the automaton has some inadequate states

LR(0) Conflicts

Tomita‘s method summarized Simple definition: – A breadth-first search over those parsing decisions that are not solved by the LR automaton – It gives an efficient and very effective approach to grammars for which the automaton has some inadequate states

Tomita‘s method summarized How does the parser act when it encounters an inadequate state on the top of the stack? Step1. It duplicates the stack and splits the parse into a different process for each copy: One copy is reserved for the REDUCE step The other copy is reserved for the SHIFT step Step2. Stacks that have a right-most state that does not allow a shift on the next input token are DISCARDED

Tomita‘s method summarized SHIFT step: push a new symbol and the new state onto the stack REDUCE step: removes part of the right end of the stack and replaces it with a non-terminal; using this non-terminal as a move in the automaton, we find a new state to put on the top of the stack Conclusion: Every time we encounter an inadequate state on the top of the stack, the duplication process is repeated untill all reduces have been treated.

Complications The repetition of the duplication process can cause a proliferation of stacks. A great number will be copied and subsequently discarded If all stacks are discarded in Step2 => the input was in error Grammars with loops : A->B B->A - the process may not terminate

Complications Some ideas to cope with the complications: 1. Use of look-ahead to decide which reduces can be made in Step1 2.Grammar with loops: 2.1. upon creating a stack, check if it is already there (and then ignore it) 2.2 check the grammar in advance for loops (and then reject it).

Two optimizations Combining equal states Combibing equal stack prefixes

A moderately ambiguous grammar S S -> E # E -> E + E E -> d Figure 9.38 A moderately ambiguous grammar

LR(0) automaton for the grammar

Stack Duplication a. 1 d+d+d# shift b. 1 d 2 +d+d# reduce c. 1 E 3 +d+d# shift d. 1 E d+d# shift e. 1 E d 2 +d# reduce f. 1 E E 5 +d# duplicate to g1 and g2 g1. 1 E E 5 +d# REDUCE; reduce to g1.1 g2. 1 E E 5 +d# SHIFT; shift to g1.2 g1.1 1 E 3 +d# shift to h1 g1.2 1 E E d# shift to h2 h1 1 E d# shift to h1.1 h2 1 E E d 2 # reduce to h1.2 h1.1 1 E d 2 # reduce to i h1.2 1 E E E 5 # duplicate to i1 and i2 i 1 E E 5 # duplicate to j1 and j2

Stack Duplication i1 1 E E E 5 #REDUCE, reduce to k1 I2 1 E E E 5 #SHIFT - DISCARDED j1 1 E E 5 #REDUCE, reduce to k2 j2 1 E E 5 #SHIFT - DISCARDED k1 1 E E 5 # reduce to l1 k2 1 E 3 # shift to l2 l1 1 E 3 # shift to m1 l2 1 E 3 # 6 reduce to m2 m1 1 E 3 # 6 reduce to n m2 1 S ACCEPT n 1 S ACCEPT

Parse trees

Combining equal states Examine the following: Both stacks have the same state on top=>further actions on both stacks will be identical Combine the two stacks to avoid duplicate work

Combining equal states f) 1. 1 E d 2 # both 2. 1 E E d 2 # REDUCE to g

Combining equal states f) 1. 1 E d 2 # both 2. 1 E E d 2 # REDUCE to g g) 1. 1 E E 5 # duplicate to 2. 1 E E E 5 # g’ and g ’’

Combining equal states f) 1. 1 E d 2 # both 2. 1 E E d 2 # REDUCE to g g) 1. 1 E E 5 # duplicate to 2. 1 E E E 5 # g’ and g ’’ g’) 1. 1 E E 5 # for REDUCE 2. 1 E E E 5 #

Combining equal states f) 1. 1 E d 2 # both 2. 1 E E d 2 # REDUCE to g g) 1. 1 E E 5 # duplicate to 2. 1 E E E 5 # g’ and g ’’ g’) 1. 1 E E 5 # for REDUCE 2. 1 E E E 5 # g’’) 1. 1 E E 5 # copy to h3) 2. 1 E E E 5 # for SHIFT

Combining equal states g’.1) 1 E E 5 # REDUCE to h.1)

Combining equal states g’.1) 1 E E 5 # REDUCE to h.1) g’.2) 1 E E E 5 # REDUCE to h.2)

Combining equal states g’.1) 1 E E 5 # REDUCE to h.1) g’.2) 1 E E E 5 # REDUCE to h.2) h.1 ) 1 E 5 # SHIFT

Combining equal states g’.1) 1 E E 5 # REDUCE to h.1) g’.2) 1 E E E 5 # REDUCE to h.2) h.1 ) 1 E 5 # SHIFT h2 ) 1 E E 5 # REDUCE to h2.1) and h2.2)

Combining equal states g’.1) 1 E E 5 # REDUCE to h.1) g’.2) 1 E E E 5 # REDUCE to h.2) h.1 ) 1 E 5 # SHIFT h2 ) 1 E E 5 # REDUCE to h2.1) and h2.2) h2.1) 1 E 5 # SHIFT

Combining equal states g’.1) 1 E E 5 # REDUCE to h.1) g’.2) 1 E E E 5 # REDUCE to h.2) h.1 ) 1 E 5 # SHIFT h2 ) 1 E E 5 # REDUCE to h2.1) and h2.2) h2.1) 1 E 5 # SHIFT h2.2) 1 E E 5 # SHIFT

Combining equal states g’.1) 1 E E 5 # REDUCE to h.1) g’.2) 1 E E E 5 # REDUCE to h.2) h.1 ) 1 E 5 # SHIFT h2 ) 1 E E 5 # REDUCE to h2.1) and h2.2) h2.1) 1 E 5 # SHIFT h2.2) 1 E E 5 # SHIFT h3) 1. 1 E E 5 # SHIFT 2. 1 E E E 5

Combining equal states Now we have five stacks (h1, h2.1, h2.2, h3). h1) and h2.1) carry state (3) on top h2.2) and h3) carry state (5) on top h1 ) 1 E 3 # SHIFT h2 ) 1 E E 5 # REDUCE to h2.1), copy to h2.2) h2.1) 1 E 3 # SHIFT h2.2) 1 E E 5 # SHIFT h3) 1. 1 E E 5 # SHIFT 2. 1 E E E 5

Combining equal states We combine the stacks with identical states on top into two bundles h’ and h’’. h’) h1) 1 E 3 #copy to i) h2.1) 1 E 3

Combining equal states We combine the stacks with identical states on top into two bundles h’ and h’’. h’) h1) 1 E 3 #copy to i) h2.1) 1 E 3 h’’) h3) 1. 1 E E E E E 5 #discard h2) 1 E E 5

Combining equal states i) 1) 1 E 3 3 # 6 2) 1 E 3

Combining equal states i) 1) 1 E 3 3 # 6 2) 1 E 3 i’) 1 E 3 # 6 REDUCE to j1)

Combining equal states i) 1) 1 E 3 3 # 6 2) 1 E 3 i’) 1 E 3 # 6 REDUCE to j1) i”) 1 E 3 # 6 REDUCE to j2)

Combining equal states i) 1) 1 E 3 3 # 6 2) 1 E 3 i’) 1 E 3 # 6 REDUCE to j1) i”) 1 E 3 # 6 REDUCE to j2) j1) 1 S ACCEPT

Combining equal states i) 1) 1 E 3 3 # 6 2) 1 E 3 i’) 1 E 3 # 6 REDUCE to j1) i”) 1 E 3 # 6 REDUCE to j2) j1) 1 S ACCEPT j2) 1 S ACCEPT

Combining equal stack prefixes When the parser makes the call for the stack to be copied,there is no actual need to copy the entire stack! It is enough to copy the top state suffixes

Combining Equal Stack-Prefixes If we observe the example : e) 1 E E 5 +d#

Combining Equal Stack-Prefixes If we observe the example : e) 1 E E 5 +d# When we duplicate the stack we have two copies of It and REDUCE is applied only to one of the copies and only “so much” of the stack is copied:

Combining Equal Stack-Prefixes If we observe the example : e) 1 E E 5 +d# When we duplicate the stack we have two copies of it and REDUCE is applied only to one of the copies and only “so much” of the stack is copied: e’) 1 E 3 +d# SHIFT e’’) 1 E E 5 +d# SHIFT

Discussion Table characteristics: -the method can work with every bottom-up table -the weaker the table, the more non-determinism will have to be resolved by breadth-first search Time requirements: - in theory – exponential - in practice - linear or slightly more than linear

SUMMARY Breadth-first search over those parsing decisions that are not solved by the LR automaton Important notions that should be memorized: – stack duplication (inadequate states, reduce, shift, discarded) – combining equal states – combining equal stack prefixes

References Dick Grune & Ceriel Jacobs (1990). Parsing Techniques