1. Compiler Designs and Constructions (Page 83 – 92)
Chapter 4: Scanner (Lexical Analyzer)
Compiler Designs and Constructions (Page 83 – 92)
Chapter 4: Scanner (Lexical Analyzer)
Objectives:
Introduction to Scanner
Token Definition
Types of Scanners
Dr. Mohsen Chitsaz
Chapter 4: Scanner (Lexical Analyzer)
COSC 470

2. Scanner (Lexical Analyzer)
Chapter 4: Scanner (Lexical Analyzer)
Scanner (Lexical Analyzer)
Task
GetNextToken
Scanner
Parser
Token
Symbol Table
Chapter 4: Scanner (Lexical Analyzer)
COSC 470

3. Chapter 4: Scanner (Lexical Analyzer)
Token:
Input File:
Keywords
User id
Symbols
Numbers
Preparation:
Remove Comments
Case Conversion
White Spaces
Compiler Directives
(Token Continued)
Chapter 4: Scanner (Lexical Analyzer)

4. Chapter 4: Scanner (Lexical Analyzer)
Preparation of the output listing:
Create a listing file
Keep line numbers
Insert errors, warnings
Table Summary
Chapter 4: Scanner (Lexical Analyzer)

5. Chapter 4: Scanner (Lexical Analyzer)
Definition:
Total, Sum, LongNum, Max If
Input
Disk -> OS Buffer -> File Buffer -> Program Buffer
Reading large blocks
Min Disk access
Unite allocations
Lookahead/ Pushback
Chapter 4: Scanner (Lexical Analyzer)

6. Chapter 4: Scanner (Lexical Analyzer)
Buffering
Buffer Size
Max lexeme length
Chapter 4: Scanner (Lexical Analyzer)

7. Chapter 4: Scanner (Lexical Analyzer)
Lexeme: A, NumOfBooks, Sum
Token: ID
Chapter 4: Scanner (Lexical Analyzer)

8. Chapter 4: Scanner (Lexical Analyzer)
Buffer Markers
Start
End
Logical End of Buffer
Next Input
Previous Lexeme
Beginning of Current Lexeme
End of Current Lexeme
Danger Zone
Lookahead Characters
Chapter 4: Scanner (Lexical Analyzer)