Evolving the ML Module System Derek Dreyer Toyota Technological Institute at Chicago April 15, 2004

2 Data Abstraction Should be able to restrict how much clients of a program module know about its implementation Enforcement of program invariants Protect clients from implementation changes

3 Object-Oriented Approach (Java) Classes/objects encapsulate code with data Private fields/methods are inaccessible to clients Semantics of data abstraction tied up with other features of OOP –Inheritance, subtyping, dynamic dispatch

4 Module-Oriented Approach (ML) Modules are units of “core-language” code Interface of a module describes what other modules get to know about it Implementor-side data abstraction via sealing Client-side data abstraction via functors

5 Evolving the ML Module System Want to make the ML module system even better: –e.g. Add support for recursive modules

6 Evolving the ML Module System Want to make the ML module system even better: –e.g. Add support for recursive modules But where do we start? –Several variants of the ML module system: Standard ML, Objective Caml, Moscow ML, etc. –Relationships/tradeoffs between them are unclear

7 The Goal of This Work Develop a unifying account of existing variations on the ML module system Build upon this foundation with support for new features (e.g. recursive modules)

8 Overview Concrete examples of data abstraction in ML High-level analysis of data abstraction in ML Extending ML with recursive modules Future work

9 IntSet Module Module implementing sets of integers: module IntSet = mod type set = int list val emptyset : set = [] fun insert (x:int,S:set) : set = x::S fun member (x:int,S:set) : bool = end

10 Using the IntSet Module Clients use “dot notation” to refer to components of the IntSet module: module IntSet = mod... end val S : IntSet.set = IntSet.insert(3, IntSet.emptyset)

11 Using the IntSet Module Clients use “dot notation” to refer to components of the IntSet module: module IntSet = mod... end val S : int list = IntSet.insert(3, IntSet.emptyset)

12 Abstract IntSet Interface Hide definition of set type in interface of IntSet : interface INT_SET = iface type set val emptyset : set fun insert : int * set -> set fun member : int * set -> bool... end

13 Data Abstraction via Sealing Seal the implementation with the interface: module IntSet = (mod... end) :> INT_SET Clients of IntSet can’t see definition of IntSet.set IntSet.set is an abstract type

14 Modules Can Have Effects IntSet module was purely functional –Body just defined values and functions But modules can have side effects –E.g. creation of mutable state

15 Symbol Table Module When evaluated, generates a new symbol table: module SymbolTable = mod val table = HashTable.create(...) type symbol = int fun string_to_symbol (x:string) =... fun symbol_to_string (n:symbol) =... end

16 Sealing the Symbol Table module SymbolTable = (mod... end) :> iface type symbol fun string_to_symbol : string -> symbol fun symbol_to_string : symbol -> string... end

17 Making Sets More Generic IntSet module only supports integer sets Implementation of sets basically the same regardless of what the type of items in the set is Functors allow you to implement generic sets –Can be instantiated with different item types –Similar to templates in C++, but more powerful

18 Functors A functor is a function from modules to modules: Some Comparable Item Type Set Functor Sets of That Item inputoutput

19 Interface of Comparable Items interface COMPARABLE = iface type item fun compare : item * item -> bool end

20 The Set Functor module Set = functor (Item : COMPARABLE) -> mod type set = Item.item list fun member (x,S) =...Item.compare(x,y) end

21 Applying the Set Functor module IntItem = mod type item = int fun compare(x,y) = Int.compare(x,y) end module IntSet = Set(IntItem)

22 Applying the Set Functor module StringItem = mod type item = string fun compare(x,y) = String.compare(x,y) end module StringSet = Set(StringItem)

23 Two Forms of Data Abstraction Can seal a module with an abstract interface –Implementor-side abstraction Can use functors to make a module more generic –Client-side abstraction

24 Overview Concrete examples of data abstraction in ML High-level analysis of data abstraction in ML Extending ML with recursive modules Future work

25 Type Components of Modules Modules in ML have type components Can “project out” type components of modules: –e.g. IntSet.set, SymbolTable.symbol. In all the examples so far, we’ve only projected types out of module variables (or names)

26 Module Expressions Examples of module expressions: mod... end (mod... end) :> (iface... end) Set(IntItem)

27 Question Why not be able to project types from arbitrary module expressions? mod... end (mod... end) :> (iface... end) Set(IntItem)

28 Example interface I = iface type t... end module A = (mod type t = int... end) :> I module B = (mod type t = float... end) :> I

29 Non-Projectible Module Suppose M = if button_is_selected() then A else B

30 Non-Projectible Module Suppose M = if button_is_selected() then A else B module C = M module D = M

31 Non-Projectible Module Suppose M = if button_is_selected() then A else B module C = M module D = M If M.t is a valid type, then C.t = M.t = D.t But C.t might be int and D.t might be float !! Unsound for M to be projectible

32 Projectible Module Suppose M = mod type t = int; val x = 3 end module C = M module D = M Fine if M is projectible, since C.t = M.t = D.t = int

33 Purity “Impure” module expression: if button_is_selected() then A else B “Pure” module expression: mod type t = int; val x = 3 end

34 Purity “Impure” module expression: if button_is_selected() then A else B “Pure” module expression: mod type t = int; val x = ref 3 end

35 Purity “Impure” module expression: if button_is_selected() then A else B “Pure” module expression: mod type t = int; val x = ref 3 end Sound for M to be projectible, M is pure (w.r.t. type components)

36 Plan Consider how purity and projectibility relate to: –Sealing –Functors –The way that sealing and functors interact

37 Sealing Suppose M = (mod... end) :> (iface type t... end) module C = M module D = M Sealing has no run-time effect, so M is pure But if M is projectible, then C.t = M.t = D.t This violates abstraction!

38 Big Picture if button_is_selected() then A else B Impure Modules (all non-projectible) Pure Modules ProjectibleNon-projectible A mod type t = int val x = ref 3 end B M :> I

39 Functors To track purity in the presence of functors: –Need to know whether applying a functor will unleash an effect or not Distinguish types of total and partial functors: –F : I 1 ! I 2, body of F is pure –F : I 1 ! I 2, body of F is impure tot par

40 Total, Applicative F : I 1 ! I 2, M :  1, F and M are pure module C = F(M) module D = F(M) F(M) is pure ) projectible C.t = F(M). t = D.t tot

41 Partial, Generative F : I 1 ! I 2, M :  1, F and M are pure module C = F(M) module D = F(M) F(M) is impure ) non-projectible C.t   D.t par

42 Set Functor Example module Set = functor (Item : COMPARABLE) -> (mod... end :> iface type set fun member : Item.item * set -> set... end) Body is pure, so Set functor is total

43 Set Functor Example module Set = functor (Item : COMPARABLE) -> (mod... end :> iface type set fun member : Item.item * set -> set... end) module IntSet1 = Set(IntItem) module IntSet2 = Set(IntItem) IntSet1.set and IntSet2.set are compatible

44 SymbolTable Module module SymbolTable = mod... end :> iface type symbol fun string_to_symbol : string -> symbol fun symbol_to_string : symbol -> string end

45 SymbolTable Functor module SymbolTable = functor () -> (mod... end :> iface type symbol fun string_to_symbol : string -> symbol fun symbol_to_string : symbol -> string end) Body is pure, so SymbolTable functor is total

46 SymbolTable Functor module SymbolTable = functor () -> (mod... end :> iface type symbol fun string_to_symbol : string -> symbol fun symbol_to_string : symbol -> string end) module ST1 = SymbolTable() module ST2 = SymbolTable() But ST1.symbol and ST2.symbol are not compatible!

47 Pure vs. Impure Sealing if button_is_selected() then A else B Impure Modules (all non-projectible) Pure Modules ProjectibleNon-projectible A mod type t = int val x = ref 3 end B M :> I M :>> I Two forms of sealing

48 SymbolTable Functor Revisited module SymbolTable = functor () -> (mod... end :>> iface type symbol fun string_to_symbol : string -> symbol fun symbol_to_string : symbol -> string end) Body is impure, so SymbolTable functor is partial

49 Summary Analysis in terms of purity and projectibility Previous systems make “applicative” or “generative” a design choice, but we support both Previous systems just employ one or the other form of sealing and call it “sealing”

50 Unifying Previous Systems Standard ML ‘97 –Only has impure sealing, all functors are partial/generative Objective Caml / Leroy (1995) –Only has pure sealing, all functors are total/applicative Shao (1999) –Supports both total and partial functors –Only has impure sealing, can’t write applicative Set functor Russo (2000) –Two languages, one like SML and one like O’Caml –Moscow ML combines them, but language is unsound

51 Overview Concrete examples of data abstraction in ML High-level analysis of data abstraction in ML Extending ML with recursive modules Future work

52 Recursive Modules Existing proposals fall into two categories: –“Units” or “Mixin modules” meant to replace ML modules (Flatt-Felleisen 98, Ancona-Zucca 96, Duggan-Sourelis 98) –Extend ML module system with recursive module construct (Crary et al. 99, Russo 01) Issues provoked by recursive module construct: –Recursion over general expressions with effects (see [Dreyer 04] for details) –Interaction of recursion and data abstraction

53 Recursive Module Example rec (X : iface module A : I A module B : I B end. mod module A = M A :> I A module B = M B :> I B end )

54 Abstract Interface iface module A : iface type t... end module B : iface type u fun g : A.t -> u end X :

55 Recursive Module Body mod module A = mod type t = int fun f(x) =... X.B.g(3)... end :> I A module B =... end

56 The Two A.t ’s mod module A = mod type t = int fun f(x) =... X.B.g(3)... end :> I A module B =... end But X.B.g ’s argument type is X.A.t, which is an abstract type!

57 Moscow ML Solution iface module A : iface type t = int... end module B : iface type u fun g : A.t -> u end X :

58 Our Solution mod module A = mod type t = int fun f(x) =... X.B.g(3)... end :> I A module B =... end At this point, we know that A.t = int, so we know that X.A.t = int as well.

59 Our Solution mod module A = mod type t = int fun f(x) =... X.B.g(3)... end :> I A module B = mod... end :> I B end At this point, A.t is abstract, so all we know is that X.A.t = A.t.

60 Current and Future Work Designed an ML dialect based on our analysis, following [Harper-Stone 97] –Currently implementing it in TILT compiler for SML Future Work: –Further extensions to ML module system E.g. type classes –Generalizations of ML-style data abstraction

Thank you!