1. CMPE 152: Compiler Design September 11 Class Meeting
Department of Computer Engineering San Jose State University Fall 2018 Instructor: Ron Mak

2. Quick Review of the Framework
Chapter 7
Quick Review of the Framework
FROM:
TO:
Next topic:
Parsing control statements, and generating parse trees.

3. Pascal Control Statements
Looping statements
REPEAT UNTIL
WHILE DO
FOR TO
FOR DOWNTO
Conditional statements
IF THEN
IF THEN ELSE
CASE

4. Statement Syntax Diagram

5. Pascal Statement Parsers
New statement parser subclasses.
RepeatStatementParser
WhileStatementParser
ForStatementParser
IfStatementParser
CaseStatementParser
Each parse() method builds a parse subtree and returns the root node.

6. REPEAT Statement Example:
Keep looping until the boolean expression becomes true.
Execute the loop at least once.
REPEAT j := i; k := i UNTIL i <= j
Use LOOP and TEST nodes for
source language independence.
Exit the loop
when the test
expression
evaluates to
true.

7. Syntax Error Handling
Recall that syntax error handling in the front end is a three-step process.
Detect the error.
Flag the error.
Recover from the error.
Good syntax error handling is important!

8. Options for Error Recovery
Stop after the first error.
No error recovery at all.
Easiest for the compiler writer, annoying for the programmer.
Worse case: The compiler crashes or hangs.
Become hopelessly lost.
Attempt to continue parsing the rest of the source program.
Spew out lots of irrelevant and meaningless error messages.
No error recovery here, either …
… but the compiler writer doesn’t admit it!

9. Options for Error Recovery, cont’d
Skip tokens after the erroneous token until …
The parser finds a token it recognizes, and
It can safely resume syntax checking the rest of the source program.

10. Parser Synchronization
Skipping tokens to reach a safe, recognizable place to resume parsing is known as synchronizing.
“Resynchronize the parser” after an error.
Good error recovery with top-down parsers can be more art than science.
How many tokens should the parser skip?
Skipping too many (the rest of the program?) can be considered “panic mode” recovery.
For this class, we’ll take a rather simplistic approach to synchronization.

11. Function synchronize()
PascalToken *PascalParserTD::synchronize(
                              const set<PascalTokenType>& sync_set)
    throw (string) {
    Token *token = current_token();
    if (sync_set.find((PascalTokenType) token->get_type())
            == sync_set.end())
    {
        error_handler.flag(token, UNEXPECTED_TOKEN, this);
        do
        {
            token = next_token(token);
        } while ((token != nullptr) &&
                 (sync_set.find((PascalTokenType) token->get_type())
                      == sync_set.end()));
    }
    return (PascalToken *) token; }
Pass in an enumeration set of “good” token types.
The method skips tokens until it finds one
that is in the set.
Flag the first
bad token.
Recover by skipping tokens not in the
synchronization set.
Resume parsing at this token!
(It’s the first token after the error that is in the synchronization set.
wci/frontend/pascal/PascalParserTD.cpp

12. Parse a REPEAT Statement
ICodeNode *StatementParser::parse_statement(Token *token)
throw (string) {
    ICodeNode *statement_node = nullptr;
    int line_number = token->get_line_number();
    switch ((PascalTokenType) token->get_type())
    {
...
        case PT_REPEAT:
        {
            RepeatStatementParser repeat_parser(this);
            statement_node = repeat_parser.parse_statement(token);
            break;
        }
    }
    ...
    return statement_node; }
wci/frontend/pascal/parsers/StatementParser.cpp

13. Parse a REPEAT Statement, cont’d
ICodeNode *RepeatStatementParser::parse_statement(Token *token)
    throw (string) {
    // Create the LOOP and TEST nodes.
    ICodeNode *loop_node =
            ICodeFactory::create_icode_node((ICodeNodeType) NT_LOOP);
    ICodeNode *test_node =
            ICodeFactory::create_icode_node((ICodeNodeType) NT_TEST);
    token = next_token(token);  // consume the REPEAT
    // Parse the statement list terminated by the UNTIL token.
    // The LOOP node is the parent of the statement subtrees.
    StatementParser statement_parser(this);
    statement_parser.parse_list(token, loop_node, PT_UNTIL, MISSING_UNTIL);
    token = current_token();
    // Parse the expression.
    // The TEST node adopts the expression subtree as its only child.
    // The LOOP node adopts the TEST node.
    ExpressionParser expression_parser(this);
    test_node->add_child(expression_parser.parse_statement(token));
    loop_node->add_child(test_node);
    return loop_node; }
wci/frontend/pascal/parsers/RepeatStatementParser.cpp

14. Parse a REPEAT Statement, cont’d
set<PascalTokenType> StatementParser::STMT_START_SET = {
    PT_BEGIN, PT_CASE, PT_FOR, PT_IF, PT_REPEAT, PT_WHILE,
    PT_IDENTIFIER, PT_SEMICOLON, };
wci/frontend/pascal/parsers/StatementParser.cpp
void StatementParser::parse_list(Token *token, ICodeNode *parent_node,
                                 const PascalTokenType terminator,
                                 const PascalErrorCode error_code)
    throw (string) {
    // Synchronization set for the terminator.
    set<PascalTokenType> terminator_set(STMT_START_SET);
    terminator_set.insert(terminator);
    // Loop to parse each statement until the END token
    // or the end of the source file.
    while (   (token != nullptr)
           && (token->get_type() != (TokenType) terminator))
    {
        // Parse a statement.  The parent node adopts the statement node.
        ICodeNode *statement_node = parse_statement(token);
        parent_node->add_child(statement_node);
        token = current_token();
        TokenType token_type = token->get_type();

15. Parse a REPEAT Statement, cont’d
        // Look for the semicolon between statements.
        if (token_type == (TokenType) PT_SEMICOLON)
        {
            token = next_token(token);  // consume the ;
        }
        // If at the start of the next statement, then missing a semicolon.
        else if (STMT_START_SET.find((PascalTokenType) token_type)
                    != STMT_START_SET.end())
            error_handler.flag(token, MISSING_SEMICOLON, this);
        // Synchronize at the start of the next statement
        // or at the terminator.
        token = synchronize(terminator_set);
    }
    // Look for the terminator token.
    if (token->get_type() == (TokenType) terminator)
    {
        token = next_token(token);  // consume the terminator token
    else {
        error_handler.flag(token, error_code, this); }
wci/frontend/pascal/parsers/StatementParser.cpp

16. Pascal Syntax Checker II: REPEAT
Demo (Chapter 7)
./Chapter7cpp compile -i repeat.txt
./Chapter7cpp compile –i repeaterrors.txt

17. WHILE Statement Example: WHILE i > j DO k := I Exit the loop
when the test
expression
evaluates to
false.
How can we eliminate
the NOT node?

18. Class WhileStatementParser
set<PascalTokenType> StatementParser::STMT_START_SET = {
    PT_BEGIN, PT_CASE, PT_FOR, PT_IF, PT_REPEAT, PT_WHILE,
    PT_IDENTIFIER, PT_SEMICOLON, };
set<PascalTokenType> StatementParser::STMT_FOLLOW_SET =
    PT_SEMICOLON, PT_END, PT_ELSE, PT_UNTIL, PT_DOT,
wci/frontend/pascal/parsers/StatementParser.cpp
set<PascalTokenType> WhileStatementParser::DO_SET;
DO_SET = StatementParser::STMT_START_SET;
DO_SET.insert(PascalTokenType::DO);
set<PascalTokenType>::iterator it;
for (it  = StatementParser::STMT_FOLLOW_SET.begin();
     it != StatementParser::STMT_FOLLOW_SET.end();
     it++) {
    DO_SET.insert(*it); }
DO_SET contains
all the tokens
that can start a statement or follow a statement, plus
the DO token.
wci/frontend/pascal/parsers/WHILEStatementParser.cpp

19. Class WhileStatementParser, cont’d
ICodeNode *WhileStatementParser::parse_statement(Token *token) throw (string) {
    token = next_token(token);  // consume the WHILE
    ICodeNode *loop_node = ICodeFactory::create_icode_node((ICodeNodeType) NT_LOOP);
    ICodeNode *test_node = ICodeFactory::create_icode_node((ICodeNodeType) NT_TEST);
    ICodeNode *not_node = ICodeFactory::create_icode_node((ICodeNodeType) NT_NOT);
    loop_node->add_child(test_node);
    test_node->add_child(not_node);
    ExpressionParser expression_parser(this);
    not_node->add_child(expression_parser.parse_statement(token));
    token = synchronize(DO_SET);
    if (token->get_type() == (TokenType) PT_DO)
    {
        token = next_token(token);  // consume the DO
    }
    else {
        error_handler.flag(token, MISSING_DO, this);
    StatementParser statement_parser(this);
    loop_node->add_child(statement_parser.parse_statement(token));
    return loop_node; }
We’re in this method because the
parser has already seen WHILE.
Synchronize the parser here!
If the current token is not DO,
then skip tokens until we find
a token that is in DO_SET.
wci/frontend/pascal/parsers/WHILEStatementParser.cpp

20. Pascal Syntax Checker II: WHILE
We can recover (better) from syntax errors.
Demo.
./Chapter7cpp compile -i while.txt
./Chapter7cpp compile -i whileerrors.txt

21. FOR Statement Example: FOR k := j TO 5 DO n := k Increment/decrement:
Node type ADD for TO
and SUBTRACT for
DOWNTO.
Initial assignment.
Node type GT
for TO and LT
for DOWNTO.
DO statement.

22. Pascal Syntax Checker II: FOR
Demo.
./Chapter7cpp compile -i for.txt
./Chapter7cpp compile -i forerrors.txt

23. IF Statement Example: IF (i = j) THEN t := 200 ELSE f := -200;
Third child only if
there is an ELSE.

24. The “Dangling” ELSE Consider: Which THEN does the ELSE pair with?
Is it: IF i = 3 THEN IF j = 2 THEN t := 500 ELSE f := -500
Or is it: IF i = 3 THEN IF j = 2 THEN t := 500 ELSE f := -500
IF i = 3 THEN IF j = 2 THEN t := 500 ELSE f := -500

25. The “Dangling” ELSE, cont’d
According to Pascal syntax, the nested IF statement is the THEN statement of the outer IF statement IF i = 3 THEN IF j = 2 THEN t := 500 ELSE f := -500
Therefore, the ELSE pairs with the closest (i.e., the second) THEN.

26. Scanner and Parser Rules of Thumb
At any point in the source file, extract the longest possible token.
Example:
<<= is one shift-left-assign token
Not a shift-left token followed by an assign token
Parser
At any point in the source file, parse the longest possible statement.
Example: IF i = 3 THEN IF j = 2 THEN t := 500 ELSE f := -500

27. Pascal Syntax Checker II: IF
Demo.
./Chapter7cpp compile -i if.txt
./Chapter7cpp compile -i iftest.txt

28. Assignment #3: WHEN Statement
Add a new WHEN statement to Pascal!
Example:
Semantically similar to:
WHEN
i = 1 => f := 10;
i = 2 => f := 20;
i = 3 => f := 30;
i = 4 => f := 40;
OTHERWISE => f := -1
END
IF i = THEN f := 10
ELSE IF i = 2 THEN f := 20
ELSE IF i = 3 THEN t := 30
ELSE IF i = 4 THEN f := 40
ELSE f := -1;

29. Assignment #3, cont’d
Draw a syntax diagram to describe the grammar of the WHEN statement.
Start with Chapter 8’s source code.
Do the work in two stages:
Modify the frontend parser to parse the WHEN statement and build an appropriate parse tree.
Modify the backend executor to execute the parse tree.

30. Assignment #3, cont’d Modify and add files, roughly in this order:
Stage 1
PascalToken.h
PascalToken.cpp
PascalSpecialSymbolToken.cpp
ICodeNodeImpl.h
ICodeNodeImpl.cpp
WhenStatementParser.h
WhenStatementParser.cpp
StatementParser.cpp
Stage 2
WhenExecutor.h
WhenExecutor.cpp
StatementExecutor.cpp

31. Assignment #3, cont’d Parse and execute file when.txt: BEGIN i := 3;
    
    WHEN
        i = 1 => f := 10;
        i = 2 => f := 20;
        i = 3 => f := 30;
        i = 4 => f := 40;
        OTHERWISE => f := -1
    END;
    range := 5.7;
        (1.0 <= range) AND (range < 3.0) => level := 1;
        (4.5 <= range) AND (range < 7.5) => BEGIN
                                               level := 2;
                                               alpha := range;
                                           END;
        (8.0 <= range) AND (range < 9.9) => level := 3;
        OTHERWISE => level := -1
END.

32. Assignment #3, cont’d
Flag and recover from syntax errors in file whenerrors.txt:
Due Friday, September 28.
BEGIN
WHEN
i := 3 => k > 5;
m = n : m := 2*n;
END;
END.