Parser Generation Using SLK and Flex++ Copyright © 2015 Curt Hill

Introduction Previously we used SLK and Flex++ to generate a Scanner –SLK found the terminals –We used regular expressions and C code to generate Flex++ input Now we have to finish the parser that uses it Copyright © 2015 Curt Hill

Scanner Review 1 Here are the steps we went through Start with a BNF file –Fed this into SLK Used the XXXConstants.h to identify terminal symbols Coded the definition section for Flex+ –Directive, includes and useful regular expressions Copyright © 2015 Curt Hill

Scanner Review 2 For each terminal in the language we made a Flex++ rule consisting of: –A regular expression –Some C++ code to process –Returned a token value Created a main program that executed the scanner –This could have been the third section Used make to execute Flex++ and then compile everything Copyright © 2015 Curt Hill

Original Scanner Modified The original scanner took a file as a command line parameter Each token was displayed with the token itself and the token number Next we modify this to feed a parser –Our first task is to take all that display stuff out –The scanner should just return values Copyright © 2015 Curt Hill

SLK Again When we fed the BNF into SLK it generated a number of parser routines –4.cpp These need to be augmented with three classes that we need to write: –XXXToken –XXXError –XXXAction –Where XXX was supplied by the –n= option of SLK Copyright © 2015 Curt Hill

XXXToken Class Next create the XXXToken class This is the interface between the parser and scanner It has two public methods: –int get() –int peek() or int peek()const; Both return a token but get consumes it and peek does not The peek function is used for the one token look ahead of LL(1) or LR(1) languages Copyright © 2015 Curt Hill

Look Ahead Copyright © 2015 Curt Hill factor constant ident ()expression When parsing a factor, we have three possible choices as to what comes next: A constant (which will be a numeric token) An identified (which is also a token) A left parenthesis (also a token) We use 1 token look ahed to determine what to do next An LR(3) or LL(3) language would require three token look ahead

XXXToken again This is the look ahead buffer Parser calls get when it knows it needs the next token It calls peek when it is deciding how to proceed –Peek returns the next token, but also leaves it there for the next get Copyright © 2015 Curt Hill

XXXError Class Error recovery is sometimes a sticky issue Do you: –Just quit there? –Attempt a recovery? –Generate an error message? The generalized parser cannot make these decisions, so leave it up to this class What methods are needed? Copyright © 2015 Curt Hill

XXXError Methods int mismatch (int terminal, int token); –Called if token does not match what is expected int no_entry (int nonterminal, int token, int level); –Called if the token is not among the known tokens void input_left (); –Called if there is input after parse thought it was complete void message(char *); –Called to display a message Copyright © 2015 Curt Hill

Beyond Parsing One of the possibilities of generating a parser is a simple recognizer The recognizer scans the file and gives us a simple answer: –File was correctly formed or not In general we want more: –Semantic analysis –Semantic content Copyright © 2015 Curt Hill

Semantic Analysis We want to know some things: –Was this variable defined before it was used? –Was this variable of the correct type for the expression it was in? These are things difficult or impossible to describe in BNF This is somewhere between syntax and semantics Copyright © 2015 Curt Hill

Semantic Content What does the program mean? What is it telling us? For a programming language we must now either: –Generate equivalent code as a compiler –Execute it as an interpreter In most SE applications we will do the latter This is why we need the action class Copyright © 2015 Curt Hill

XXXAction Class This class is used to communicate that the parser has figured out something That something is one of the following: –It has entered a new state –It is about to do a reduction –It wants to execute an action Each of these corresponds to a method that you need to define for the class Copyright © 2015 Curt Hill

State The method is: void state(int state); The state tells us what the parser is currently working on It is the least useful of the methods for us Copyright © 2015 Curt Hill

Reduction A reduction is when the parser has found everything on the RHS of a production and it is about to reduce it to the LHS of the production The signature for a reduce is: void reduce(int entry); The entry is a numbered production Copyright © 2015 Curt Hill

Reduction Processing The first thing, from a debugging perspective, that needs to be done is display the production This is done with the following function: char * XXXGetProductionName(int entry); This provides a copy of the production Copyright © 2015 Curt Hill

Augmented productions In the BNF that SLK accepts you may put actions to execute at the end of productions The format of the action is _action_X –The X is a number When the production is about to be reduced the execute function will get a call with this number Copyright © 2015 Curt Hill

Example Augmented Production Here are two productions or one with alternation: return_type ::= void _action_7 type _action_8 When the reduction void to type occurs you should be able to see the 7 It is almost that easy –As we will see Copyright © 2015 Curt Hill

Execute The signature is: void execute(int action_number); Every time one of the reductions is about to occur execute is called However, you always get a negative number and not the one you expect –If you add the value of the largest non- terminal constant to the received value it equal the action number Copyright © 2015 Curt Hill

Finally We generate a makefile to do everything but run it Next we do a demo Then an assignment Next lets consider the generated parser next Copyright © 2015 Curt Hill