Presentation is loading. Please wait.

Presentation is loading. Please wait.

Using SLK and Flex++ Followed by a Demo

Similar presentations


Presentation on theme: "Using SLK and Flex++ Followed by a Demo"— Presentation transcript:

1 Using SLK and Flex++ Followed by a Demo
Scanner Generation Using SLK and Flex++ Followed by a Demo Copyright © Curt Hill

2 Introduction There are a variety of parser generators and scanner generators The standards for UNIX seem to be lex and yacc Yacc seems to have replaced something earlier The GNU versions are flex and bison We will use flex++ which is somewhat more parameterizable We will also use SLK which seems to accept a better set of languages Copyright © Curt Hill

3 Flex++ This is a scanner generator It takes an input of three sections
Definitions for later use Rules that accept a string and produce a token Code which is passed through as-is These three are separated by a %% The percent is used to indicate options and a variety of other things as well Copyright © Curt Hill

4 Theory of Operation Almost all the scanner generators produce a program that is a character driven finite state automaton Uses regular expressions Each character moves the automaton from one state to another Self loops are possible Some of these states recognize the completion of a keyword or multi-character symbol Copyright © Curt Hill

5 Regular Expressions The typical way to describe what you want is using regular expressions Recall that a type 3 language is regular Every regular language may be recognized by a finite state automaton Thus we will define what we mean by an identifier with a regular expression Or an integral constant, floating constant, character string etc Copyright © Curt Hill

6 Sections As mentioned there are three sections
Definition Rules Code Separated by the %% directive We will now look at these Copyright © Curt Hill

7 Definition Section As the name suggests we set up directives and definitions for later use Directives are options used in Flex++ Usually start with a % We also define regular expressions we will use later in one or more rules Copyright © Curt Hill

8 Directives %name scanner name %header{ … %} %define name content
This is the class name %header{ … %} Code that will be inserted at the beginning of the file – includes etc. May also use %{ … %} %define name content Defines a macro Next screen shows some predefined macros Copyright © Curt Hill

9 Predefined Directives
%define TEXT yytext The text of something that matched a regular expression Need this for identifiers and the like %define LENG yyleng The length of the text %define LEX yylex Scanner function Copyright © Curt Hill

10 Return Type %define LEX_RETURN int The default is int
The return type of LEX The default is int If you want a class there it should have an int constructor Could also be void * This should signify the end of file token Copyright © Curt Hill

11 More Predefined %define CLASS name %define INHERIT name
Defaults to same as %name %define INHERIT name Only needed if your class is a derivation %define MEMBERS mem Extra member data for the class %define CONSTRUCTOR_INIT lg The initialization list including : Any of these predefined names may be changed to something else Copyright © Curt Hill

12 Named Expressions The definition section may also contain any regular expressions that will be handy in the next section The format is: NAME expression Where is the name you will use later and expression is the regular expression Example: DIGIT [0-9] This is a set that matches any digit Copyright © Curt Hill

13 Example Definitions %name Scanner
%define MEMBERS public: int line, column; %define CONSTRUCTOR_INIT : line(1), column(1) %header{ #include<sstream> #include <iostream> #include "CDHConstants.h" using namespace std; %} LETTER [A-Za-z] DIGIT [0-9] DIGIT1 [1-9] Copyright © Curt Hill

14 Rules Section You make rules Each rule matches a construct
A match provokes an action Each rule matches a terminal and returns the token Or it disposes of things that may ignored The format is: RE action Where RE is a regular expression and action is C++ code to execute upon match Copyright © Curt Hill

15 Rule Examples Recall the definitions before
Here is a blank killer: " " { ++column; } The column variable is defined earlier The C++ code could be multiple lines For example removing comments will take some work Here is a terminal: "=" { ++column; cout << "equal\n"; return EQUAL_; } Copyright © Curt Hill

16 More Examples Reserved words are easy: "int“ { column += 3; cout << "int\n"; return INT_; } Other non-terminals are harder: {DIGIT1}{DIGIT}* { column += strlen(yytext); cout << "number: “ << yytext << “\n”; return NUMBER_; } Copyright © Curt Hill

17 Code Section This is just C++ code that is copied as-is onto the end of the file Often it is nothing It may be a main function that tests the scanner Copyright © Curt Hill

18 Command line Options -ofilename -hfilename -L There are many others
Gives the output name with extension Default is lex.yy.c -hfilename Generates a .h header for using this in another file -L Do not put a # comment in output There are many others The input file is last in the line Copyright © Curt Hill

19 Process 1 We saw this in a previous presentation Make the BNF
Feed this into your parser generator Make sure reserved words are all that you intended Design your token Start the definition section Determine what the scanner will return: Copyright © Curt Hill

20 Process 2 Start the rules section
For each terminal create a rule Tokens should return an initialized object Scanner should now be ready Copyright © Curt Hill

21 Finally Next we do a demo Then an assignment
This one with or without original BNF Then an assignment Then consider the generated parser Copyright © Curt Hill


Download ppt "Using SLK and Flex++ Followed by a Demo"

Similar presentations


Ads by Google