1. Erik (Happi) Johansson happi@csd.uu.se Room: 1338 4711033
ML-Modules & ML-Lex
Erik (Happi) Johansson
Room:
Lesson1 KT1-VT01
02/01/2019

2. Introduction The lessons will be concerned with the assignment.
Today we will talk about some ML concepts you will need, and the first part of the assignment.
Find out any relevant background and interest of the audience
Lesson1 KT1-VT01
02/01/2019

3. Agenda The Assignment Advanced ML (SML-NJ) ML-Lex Records
Exception handling
Module system
Standalone execution
Makefiles
ML-Lex
Lesson1 KT1-VT01
02/01/2019

4. Overview The assignment has four parts
Lexical analysis
Syntactic analysis
Semantic analysis
Native code generation
The final compiler should be written in ML
Bar
Syntactic analysis
Lexical analysis ML
Compiler
(In ML)
Semantic analysis
Unix
Native
Code
SPIM MIPS
Lesson1 KT1-VT01
02/01/2019

5. The Assignment Goal Motivation Rules
A complete compiler from source code to native machine code.
Motivation
Give understanding of all parts of a compiler.
Give understanding of program execution.
Give experience in programming.
Rules
Groups of 2 persons.
Absolute deadlines.
Lesson1 KT1-VT01
02/01/2019

6. The Assignment Compile the language Bar Compile to MIPS
Bar is a call by value, imperative language with static lexical scoping.
Bar is strongly statically typed with explicit typing of declarations.
Bar has integers, floats, truthvalues (booleans), and arrays of these types.
Bar has basic control structures, functions and procedures.
Bar has destructiv updates of variabels.
Compile to MIPS
Lesson1 KT1-VT01
02/01/2019

7. The language Bar (example)
[int:1] a; || Declare a global array (a) of one integer |+
The function foo(b) takes an array of integer and
returns a boolean. +|
function foo([int] b):bool
begin
b[0] = 1; || Assign the value 1 to the first element in b.
|| Note that arrays are passed by reference,
|| so the array used in the call to foo will be updated.
return true; || Return the boolean true.
end
The main body of the program.
Calls the function foo and depending on the return value
either prints the first element in the global array a,
or the boolean false.
if foo(a)
then print a[0];
else print false;
Lesson1 KT1-VT01
02/01/2019

8. Bar Grammar 1 (of 4)
Lesson1 KT1-VT01
02/01/2019

9. Bar Grammar 2 (of 4)
Lesson1 KT1-VT01
02/01/2019

10. Bar Grammar 3 (of 4)
Lesson1 KT1-VT01
02/01/2019

11. Bar Grammar 4 (of 4)
Lesson1 KT1-VT01
02/01/2019

12. Advanced ML Records Exception handling Module system
Standalone execution
Makefiles
Lesson1 KT1-VT01
02/01/2019

13. Records Just like tuples with named elements.
- val car = {make = "SAAB", built = 2000};
val car = {built=2000,make="SAAB"} :
{built:int, make:string}
- #built car;
val it = 2000 : int
- fun uppgrade{make,built = year} = {make=make, built=year+1};
val uppgrade = fn : {built:int, make:'a} -> {built:int, make:'a}
- uppgrade car;
val it = {built=2001,make="SAAB"} :
Lesson1 KT1-VT01
02/01/2019

14. Exception handling exception Error of int * string; fun f v =
(case v of
1 => 42
| _ => raise Error (1,"not 1")
) handle
Error (no, s) =>
(print s; print “\n”; no);
- f 2;
not 1
val it = 1 : int
Lesson1 KT1-VT01
02/01/2019

15. Module system The basis for the ML module system is the structure.
You have probably seen and used structures several times. Ex. TextIO.print
TextIO is a structure that defines the function print.
Lesson1 KT1-VT01
02/01/2019

16. Structures A structure is defined as Ex structure foo = struct
structure NAME = struct DEFINITIONS
end Ex
structure foo = struct
type t = string
say = fn x => TextIO.print x
Lesson1 KT1-VT01
02/01/2019

17. Signatures Signatures gives the type of a structure.
They are declared as:
signature NAME = sig
DECLARATIONS
end Ex
signature bar = sig type t
val say : t -> unit
Lesson1 KT1-VT01
02/01/2019

18. Ascription To say that a structure implements a signature.
Can be transparent ( “:” ) or opaque ( “:>”).
structure foo:bar = struct
type t = string
say = fn x => TextIO.print x
end
Lesson1 KT1-VT01
02/01/2019

19. Standalone Execution
Created by saving the state of the run-time system to a file.
A start function of type
string * string list -> OS.Process.status
is needed.
The state is saved by calling
SMLofNJ.exportFn
with a tuple
(FileName, StartFunction)
The program is then executed by:
Args
Lesson1 KT1-VT01
02/01/2019

20. Example of standalone execution
- fun start (path, args) =
(print (path ^ "\n");
map (fn x => print x) args; OS.Process.success);
- SMLofNJ.exportFn ("testit", start);
> 42
/it/sw/ml/bin/sml 42
Lesson1 KT1-VT01
02/01/2019

21. Makefiles Use the ML - Compilation Manager
Create a file called sources.cm
List the source files of your project in this file preceded by the keywords
Group is
When you have a sources.cm file you can just type: CM.make(); in the ML-shell to compile all files as necessary.
Lesson1 KT1-VT01
02/01/2019

22. Example of sources.cm
Group is
testlexer.sml
errormsg.sml
tokens.sig
tokens.sml
lexer.lex
smlnj-lib.cm
The last file is needed to get the SML standard libraries. The compilation manager should find this file by it self.
The compilation manager recognizes .lex files and executes ml-lex.
Lesson1 KT1-VT01
02/01/2019

23. ML – Lex
ML lex takes a file with rules describing tokens of a language and “compiles” them into a program that can turn a string in the language to the tokens of the string.
We will look at
The lexer file .lex
How to run ML-Lex
How to call the lexer
Lesson1 KT1-VT01
02/01/2019

24. The lexer file (.lex) User declarations (ML-code) %%
ML-LEX definitions (abreviations)
Rules
Lesson1 KT1-VT01
02/01/2019

25. Declarations (.lex) Two required Any other help function you need.
The function ”eof” fun eof() = Tokens.EOF
The type of lexresult type lexresult = Tokens.tokens
Any other help function you need.
Lesson1 KT1-VT01
02/01/2019

26. ML-Lex definitions (.lex)
Give start states
%s SkipComment;
Give names to regular expressions digit=[0-9]; int={digit}+; exp=((e|E)"-"?{int}); floats={int}"."{int}{exp}?;
Lesson1 KT1-VT01
02/01/2019

27. ML-Lex Rules (.lex) Regexp ) action <STATE>Regexp ) action
(action should be of type lexresult.)
{floats} =>
(case Real.fromString yytext of
NONE =>
ErrorMsg.impossible ("Cannot convert" ^ yytext)
| SOME f => Tokens.FLOAT(f, yypos, yypos+ String.size yytext));
Lesson1 KT1-VT01
02/01/2019

28. To run ML-Lex Typing ml-lex lexer.lex
at the unix prompt, should give a file
lexer.lex.sml
(If your path contains /it/sw/ml/bin/ml-lex)
Lesson1 KT1-VT01
02/01/2019

29. To use the lexer in a program
The file *.lex.sml defines a ML structure Mlex, which contains the function makeLexer.
This function should be called with a function of the type int -> string
This second function should read up to N characters from the in-stream.
The function makeLexer returns a function that returns the next token each time it is called.
Lesson1 KT1-VT01
02/01/2019

30. Example of how to use the lexer in a program
fun makeget stream =
let fun get (_:int) = TextIO.input stream
in get end
fun tokenize (lexer) =
let val token = lexer()
in (case token of Tokens.EOF(_) => ()
| =>
(print (Tokens.toString token);
tokenize(lexer)))
end
fun do_it (s:string, slist:string list) =
let val lexer = Mlex.makeLexer (make_get TextIO.stdIn)
in (tokenize(lexer); OS.Process.success)
Lesson1 KT1-VT01
02/01/2019

31. Where to Get More Information
The assignment page:
A lexical analyzer generator for Standard ML:
Standard ML of New Jersey User's Guide:
CM -- A Compilation Manager for SML/NJ -- User Manual:
Lesson1 KT1-VT01
02/01/2019