1. Linear Regions Are All You Need
Matthew Fluet
Cornell University
Greg Morrisett & Amal Ahmed
Harvard University

2. Memory Management
Dynamic allocation pervasive in computation

3. Memory Management Dynamic allocation pervasive in computation
Region-based Memory Management
Memory is divided into regions
Objects are individually allocated in a region
constant-time operation
All objects in a region are deallocated together

4. Application: Cyclone Cyclone Safe-C Project type-safety
with the “virtues” of C
low-level interface with manifest cost model
range of memory management options
regions are an organizing principle

5. Cyclone: Regions Region variety Allocation (objects) Deallocation
Aliasing
(what)
(when)
Stack
static
whole region
exit of lexical scope
unrestricted
Lexical
dynamic
Dynamic
manual
Dynamic seq.
Heap (`H)
single objects
automatic (BDW GC)
Unique (`U)
restricted
Ref-counted (`RC)

6. Application: Cyclone MediaNET TCP benchmark (packet forwarding)
Cyclone v.0.1 (lexical regions & garbage collector)
High water mark: 840 KB
130 collections
Basic throughput: 50 MB/s
Cyclone v.0.5 (unique pointers & dynamic regions)
High water mark: 8 KB
0 collections
Basic throughput: 74MB/s

7. Application: Cyclone MediaNET TCP benchmark (packet forwarding)
Cyclone v.0.1 (lexical regions & garbage collector)
High water mark: 840 KB
130 collections
Basic throughput: 50 MB/s
Cyclone v.0.5 (unique pointers & dynamic regions)
High water mark: 8 KB
0 collections
Basic throughput: 74MB/s

8. Proving type safety of Cyclone is a nightmare!!
Cyclone: Regions
Region variety
Allocation
(objects)
Deallocation
Aliasing
(what)
(when)
Stack
static
whole region
exit of lexical scope
unrestricted
Lexical
dynamic
Dynamic
manual
Dynamic seq.
Heap (`H)
single objects
automatic (BDW GC)
Unique (`U)
restricted
Ref-counted (`RC)
Proving type safety of Cyclone is a nightmare!!

9. Cyclone: Regions
Region variety
Allocation
(objects)
Deallocation
Aliasing
(what)
(when)
Stack
static
whole region
exit of lexical scope
unrestricted
Lexical
dynamic
Dynamic
manual
Dynamic seq.
Heap (`H)
single objects
automatic (BDW GC)
Unique (`U)
restricted
Ref-counted (`RC)
Goal: simple model where we can easily encode the key features of Cyclone in a target language with a simpler type system.
Minor chord: while the operational extensions were straightforward, it’s the type-system that becomes burdensome. So, maybe the original type system wasn’t quite the right abstraction.

10. Linear Regions Are All You Need
Cyclone: Regions
Region variety
Allocation
(objects)
Deallocation
Aliasing
(what)
(when)
Stack
static
whole region
exit of lexical scope
unrestricted
Lexical
dynamic
Dynamic
manual
Dynamic seq.
Heap (`H)
single objects
automatic (BDW GC)
Unique (`U)
restricted
Ref-counted (`RC)
Linear Regions Are All You Need
Minor chord: while the operational extensions were straightforward, it’s the type-system that becomes burdensome. So, maybe the original type system wasn’t quite the right abstraction.

11. Outline Introduction Monadic Type System (FRGN) [ICFP’04]
Substructural Type System (lrgnUL)
Translation Sketch
Conclusion

12. Monadic Type System for Regions [ICFP’04]
Extend the runST “trick” to nested regions [L-PJ ’94]
Polymorphic type system ensures safety
Key insights (FRGN):
Effects map to an indexed monadic type
Region subtyping witnessed by types
Sufficient for encoding Tofte-Talpin region calculus and “core” Cyclone region features

13. RGN monad: Types Monadic type RGN s t
computations in stack of regions s returning values of type t; a “stack” transformer
“in stack of regions s” means the computation will allocate data in and read/write data from a stack of regions, denoted by the type variable s. RGN is the “region monad,” as ST was the “state monad.”

14. RGN monad: Operations Monadic unit and bind returnRGN ::
8s,a. a ! RGN s a
thenRGN ::
8s,a,b. RGN s a ! (a ! RGN s b) ! RGN s b
returnRGN takes a value and gives the trivial computation (that does nothing with the region). thenRGN sequences two computations; the second has access to the value computed by the first.

15. RGN monad: Operations Monadic unit and bind returnRGN ::
8s,a. a ! RGN s a
thenRGN ::
8s,a,b. RGN s a ! (a ! RGN s b) ! RGN s b
Note that the operation allows the type of the value computed to change …

16. RGN monad: Operations Monadic unit and bind returnRGN ::
8s,a. a ! RGN s a
thenRGN ::
8s,a,b. RGN s a ! (a ! RGN s b) ! RGN s b
… but not the region of the computation.

17. RGN monad: Types Reference type Ref s t
values of type t allocated in region
at the top of the stack of regions s

18. RGN monad: Operations Create and read region allocated values new ::
8s,a. a ! RGN s (Ref s a)
read ::
8s,a. Ref s a ! RGN s a
Two operators for manipulating region allocated data.

19. RGN monad: Operations Create and read region allocated values new ::
8s,a. a ! RGN s (Ref s a)
read ::
8s,a. Ref s a ! RGN s a
As expected, the regions must match up.

20. RGN monad: Encapsulation
Encapsulate and run a monadic computation
runRGN ::
8a. (8s. RGN s a) ! a
We can mimic runST.

21. RGN monad: Encapsulation
Encapsulate and run a monadic computation
runRGN ::
8a. (8s. RGN s a) ! a

22. RGN monad: Encapsulation
Encapsulate and run a monadic computation
runRGN ::
8a. (8s. RGN s a) ! a
Note that quantification over the stack of regions means no assumptions.
“for all stacks” ) no assumptions about stack of regions

23. RGN monad: Encapsulation
Encapsulate and run a monadic computation
runRGN ::
8a. (8s. RGN s a) ! a
“for all stacks” ) no assumptions about stack of regions

24. RGN monad: Encapsulation
Encapsulate and run a monadic computation
runRGN ::
8a. (8s. RGN s a) ! a
In particular, the result cannot be a RGNVar s t.
result is independent of stack ) s 62 frv(a) ) region values don’t escape
“for all stacks” ) no assumptions about stack of regions

25. RGN monad: Example runRGN ( Ls1. do a à new [s1] 1 c à runRGN ( Ls2.
do b à new [s2] 7
… z = …
new [s1] z )
… c … )
Note that this uses the Haskell “do” notation; the  symbol corresponds to the thenRGN operation. There are a variety of things wrong with this translation. I’m going to touch on one in particular … r1

26. input allocated in first region
RGN monad: Example
runRGN ( Ls1.
do a à new [s1] 1
c à runRGN ( Ls2.
do b à new [s2] 7
… z = …
new [s1] z )
… c … )
Note that this uses the Haskell “do” notation; the  symbol corresponds to the thenRGN operation. There are a variety of things wrong with this translation. I’m going to touch on one in particular … r1
a : 1
input allocated in first region

27. input allocated in first region
RGN monad: Example
runRGN ( Ls1.
do a à new [s1] 1
c à runRGN ( Ls2.
do b à new [s2] 7
… z = …
new [s1] z )
… c … ) r2
Note that this uses the Haskell “do” notation; the  symbol corresponds to the thenRGN operation. There are a variety of things wrong with this translation. I’m going to touch on one in particular … r1
a : 1
input allocated in first region

28. RGN monad: Example runRGN ( Ls1. do a à new [s1] 1 c à runRGN ( Ls2.
temporary allocated in second region
runRGN ( Ls1.
do a à new [s1] 1
c à runRGN ( Ls2.
do b à new [s2] 7
… z = …
new [s1] z )
… c … ) r2
b : 7
Note that this uses the Haskell “do” notation; the  symbol corresponds to the thenRGN operation. There are a variety of things wrong with this translation. I’m going to touch on one in particular … r1
a : 1
input allocated in first region

29. RGN monad: Example runRGN ( Ls1. do a à new [s1] 1 c à runRGN ( Ls2.
temporary allocated in second region
runRGN ( Ls1.
do a à new [s1] 1
c à runRGN ( Ls2.
do b à new [s2] 7
… z = …
new [s1] z )
… c … ) r2
b : 7
Note that this uses the Haskell “do” notation; the  symbol corresponds to the thenRGN operation. There are a variety of things wrong with this translation. I’m going to touch on one in particular … r1
a : 1
input and output allocated in first region
c : 8

30. RGN monad: Example runRGN ( Ls1. do a à new [s1] 1 c à runRGN ( Ls2.
temporary allocated in second region
runRGN ( Ls1.
do a à new [s1] 1
c à runRGN ( Ls2.
do b à new [s2] 7
… z = …
new [s1] z )
… c … )
Note that this uses the Haskell “do” notation; the  symbol corresponds to the thenRGN operation. There are a variety of things wrong with this translation. I’m going to touch on one in particular … r1
a : 1
input and output allocated in first region
c : 8

31. RGN monad: Example runRGN ( Ls1. do a à new [s1] 1 c à runRGN ( Ls2.
do b à new [s2] 7
… z = …
new [s1] z )
… c … )
allocating in younger region requires RGN s2 t type
Note that allocating in the younger region will result in a computation in region r2, while allocating the result in the older region will result in a computation in region r1. Since these two computations must be sequenced, there is a mismatch.
allocating in older region requires RGN s1 t type

32. RGN monad: Witnesses Witness type Pf(s1 · s2) –
type-level proof that the stack of regions s1
is a substack of the stack of regions s2

33. RGN monad: Witnesses Witness operations coerceRGN ::
8s1,s2,a. Pf(s1 · s2) ! RGN s1 a ! RGN s2 a
transSub ::
8s1,s2,s3. Pf(s1 · s2) ! Pf(s2 · s3) ! Pf(s1 · s3)

34. RGN monad: Regions
Regions are created and destroyed with a lexically scoped construct
letRGN ::
8s1,a. (8s2. Pf(s1 · s2) ! RGN s2 a) ! RGN s1 a
We can mimic runST.

35. RGN monad: Regions
Regions are created and destroyed with a lexically scoped construct
letRGN ::
8s1,a. (8s2. Pf(s1 · s2) ! RGN s2 a) ! RGN s1 a
We can mimic runST.

36. RGN monad: Example letRGN ( Ls1. lpf1. do a à new [s1] 1
c à letRGN ( Ls2. lpf2.
do b à new [s2] 7
… z = …
coerceRgn pf (new [s1] z ))
… c … ) r2
b : 7
Note that this uses the Haskell “do” notation; the  symbol corresponds to the thenRGN operation. There are a variety of things wrong with this translation. I’m going to touch on one in particular … r1
a : 1
c : 8

37. Limitations of LIFO Regions
Lexical scope is ill-suited for
iterative computations
Conway’s Game of Life; copying GC
CPS-based computations
event-based computations

38. Limitations of LIFO Regions
Lexical scope is ill-suited for
iterative computations
Conway’s Game of Life; copying GC
CPS-based computations
event-based computations
But, lexical scope was ensuring that the stack of regions was used in a single-threaded manner

39. Substructural Type Systems
Provide core mechanisms to restrict the number and order of uses of data and operations
generalization of linear type systems

40. Substructural Type System: lUL
Qualifiers
q ::= U j L
PreTypes
t ::= 1 j t1 £ t2 j t1 ! t2 j 8a.t j 9a.t
Types
t ::= qt

41. Substructural Type System: lUL
Qualifiers
q ::= U j L
PreTypes
t ::= 1 j t1 £ t2 j t1 ! t2 j 8a.t j 9a.t
Types
t ::= qt
How may the value be used?

42. Substructural Type System: lUL
Qualifiers
q ::= U j L
PreTypes
t ::= 1 j t1 £ t2 j t1 ! t2 j 8a.t j 9a.t
Types
t ::= qt
How often may the value be used?
How may the value be used?

43. Substructural Qualifiers
Linear
must be “used” exactly once
Unrestricted
Drop Copy
may be “used” an arbitrary # of times

44. Substructural Type System for Regions
Provide core mechanisms to restrict the number and order of uses of data and operations
generalization of linear type systems
Key insights (lrgnUL):
Separate region names from region liveness
Region liveness witnessed by types
Sufficient for encoding FRGN calculus and “advanced” Cyclone region features

45. lrgnUL = lUL + Regions PreTypes
t ::= … j cap r j ref r t j 8r.t j 9r.t
“capability” for region r; mediates all access to a region for allocating, reading, and writing

46. lrgnUL: Region Primitives
Regions are created and destroyed with separate operations
newrgn ::
U1 ! (9r. Lcap r)
freergn ::
8r. (Lcap r ! U1)

47. lrgnUL: Region Primitives
Regions are created and destroyed with separate operations
newrgn ::
U1 ! (9r. Lcap r)
freergn ::
8r. (Lcap r ! U1)
Produces a capability.
Consumes a capability.

48. lrgnUL: Region Primitives
Regions are created and destroyed with separate operations
newrgn ::
U1 ! (9r. Lcap r)
freergn ::
8r. (Lcap r ! U1)

49. lrgnUL: Region Primitives
new ::
8r,a. ((Lcap r £ Ua) ! (Lcap r £ Uref r Ua))
read ::
8r,a. ((Lcap r £ Uref r Ua) !
(Lcap r £ Ua))

50. lrgnUL: Region Primitives
new ::
8r,a. ((Lcap r £ Ua) ! (Lcap r £ Uref r Ua))
read ::
8r,a. ((Lcap r £ Uref r Ua) !
(Lcap r £ Ua))
Requires a capability.
Returns a capability.

51. lrgnUL: Region Primitives
new ::
8r,a. ((Lcap r £ Ua) ! (Lcap r £ Uref r Ua)
read ::
8r,a. ((Lcap r £ Uref r Ua) !
(Lcap r £ Ua)

52. Translation: FRGN to lrgnUL, Types
« RGN s t ¬ = U(s ! L(s £ «t¬))

53. Translation: FRGN to lrgnUL, Types
« RGN s t ¬ = s ! (s £ «t¬)
operational behavior of monad is store/stack-passing

54. Translation: FRGN to lrgnUL, Types
« RGN s t ¬ = s ! (s £ «t¬)
operational behavior of monad is store/stack-passing

55. Translation: FRGN to lrgnUL, Types
« RGN s t ¬ = s ! (s £ «t¬)
operational behavior of monad is store/stack-passing
represent “stack of regions” as a sequence of linear capabilities, formed out of nested linear tuples

56. Translation: FRGN to lrgnUL, Types
« RGN s t ¬ = s ! (s £ «t¬)
operational behavior of monad is store/stack-passing
represent “stack of regions” as a sequence of linear capabilities, formed out of nested linear tuples

57. Translation: FRGN to lrgnUL, Ops
« returnRGN [s] [t] e ¬ = let res : «t¬ = «e¬ in Ulstk:s. Lhstk,resi
« thenRGN [s] [ta] [tb] e1 e2 ¬ = let f : «RGN s ta¬= «e1¬ in let g : «ta ! RGN s tb¬ = «e2¬ in Ulstk:s. let hstk,resi = f stk in g res stk

58. Translation: FRGN to lrgnUL, Ops
« returnRGN [s] [t] e ¬ = let res : «t¬ = «e¬ in Ulstk:s. Lhstk,resi
« thenRGN [s] [ta] [tb] e1 e2 ¬ = let f : «RGN s ta¬= «e1¬ in let g : «ta ! RGN s tb¬ = «e2¬ in Ulstk:s. let hstk,resi = f stk in g res stk
Store-passing encoding

59. Translation: FRGN to lrgnUL, Types
« Pf(s1 · s2) ¬ = U(9s’. Iso(s2, L(s1 £ s’)))

60. Translation: FRGN to lrgnUL, Types
« Pf(s1 · s2) ¬ = U(9s’. Iso(s2, L(s1 £ s’)))
Isomorphism between s2 and L(s1 £ s’), for some “slack” s’

61. Translation: FRGN to lrgnUL, Types
« Pf(s1 · s2) ¬ = U(9s’. Iso(s2, L(s1 £ s’)))
Isomorphism between s2 and L(s1 £ s’), for some “slack” s’
Proof that s1 is a substack of s2 is persistent
Liveness of s1 and s2 is ephemeral

62. Translation: FRGN to lrgnUL, Types
« Ref s t ¬ = U(9r. U(U(9s’. Iso(s, L(s’ £ Lcap r))) « Ref s t ¬ = U(9r. U(£ Uref r «t¬))

63. Translation: FRGN to lrgnUL, Types
« Ref s t ¬ = 9r. (9s’. Iso(s, L(s’ £ Lcap r)) « Ref s t ¬ = 9r. (£ Uref r «t¬)
Existential fixes region r

64. Translation: FRGN to lrgnUL, Types
« Ref s t ¬ = 9r. (9s’. Iso(s, L(s’ £ Lcap r)) « Ref s t ¬ = 9r. (£ Uref r «t¬)
Existential fixes region r
Isomorphism witnesses membership of r in s

65. Translation: FRGN to lrgnUL, Ops
« letRGN [s1] [t] e ¬ = let f : «8s2. Pf(s1·s2) ! Hnd s2 ! RGN s2 t¬ = «e¬ in Ulstk1:s1.let pack(r,hcap,hndi) = newrgn Lhi in Ulstk1:s1.let stk2 = Lhstk1,capi in Ulstk1:s1.let id = Ulstk: L(s1 ­ Lcap r).stk in Ulstk1:s1.let pwit = Upack(Lcap r,Uhid,idi) in Ulstk1:s1.let phnd = Upack(r,UhUpack(s1,Uhid,idi),hndi) in Ulstk1:s1.let hstk2,resi = f [L(s1 ­ Lcap r)] pwit phnd stk2 in Ulstk1:s1.let hstk1,capi = stk2 in Ulstk1:s1.let hi = freergn [r] Lhcap,hndi in Ulstk1:s1.Lhstk1,resi

66. Translation: FRGN to lrgnUL, Ops
Stack-passing encoding
« letRGN [s1] [t] e ¬ = let f : «8s2. Pf(s1·s2) ! RGN s2 t¬ = «e¬ in Ulstk1:s1.let pack(r,cap) = newrgn Uhi in Ulstk1:s1.let stk2 = Lhstk1,capi in Ulstk1:s1.let id = Ulstk: L(s1 ­ Lcap r).stk in Ulstk1:s1.let pwit = Upack(Lcap r,Uhid,idi) in Ulstk1:s1.let hstk2,resi = f [L(s1 ­ Lcap r)] pwit stk2 in Ulstk1:s1.let hstk1,capi = stk2 in Ulstk1:s1.let hi = freergn [r] cap in Ulstk1:s1.Lhstk1,resi

67. Translation: FRGN to lrgnUL, Ops
Create & destroy a new region
« letRGN [s1] [t] e ¬ = let f : «8s2. Pf(s1·s2) ! RGN s2 t¬ = «e¬ in Ulstk1:s1.let pack(r,cap) = newrgn Uhi in Ulstk1:s1.let stk2 = Lhstk1,capi in Ulstk1:s1.let id = Ulstk: L(s1 ­ Lcap r).stk in Ulstk1:s1.let pwit = Upack(Lcap r,Uhid,idi) in Ulstk1:s1.let hstk2,resi = f [L(s1 ­ Lcap r)] pwit stk2 in Ulstk1:s1.let hstk1,capi = stk2 in Ulstk1:s1.let hi = freergn [r] cap in Ulstk1:s1.Lhstk1,resi

68. Translation: FRGN to lrgnUL, Ops
Construct rep. of new stack
« letRGN [s1] [t] e ¬ = let f : «8s2. Pf(s1·s2) ! RGN s2 t¬ = «e¬ in Ulstk1:s1.let pack(r,cap) = newrgn Uhi in Ulstk1:s1.let stk2 = Lhstk1,capi in Ulstk1:s1.let id = Ulstk: L(s1 ­ Lcap r).stk in Ulstk1:s1.let pwit = Upack(Lcap r,Uhid,idi) in Ulstk1:s1.let hstk2,resi = f [L(s1 ­ Lcap r)] pwit stk2 in Ulstk1:s1.let hstk1,capi = stk2 in Ulstk1:s1.let hi = freergn [r] cap in Ulstk1:s1.Lhstk1,resi

69. Translation: FRGN to lrgnUL, Ops
Run comp. and recover old stack and new cap
« letRGN [s1] [t] e ¬ = let f : «8s2. Pf(s1·s2) ! RGN s2 t¬ = «e¬ in Ulstk1:s1.let pack(r,cap) = newrgn Uhi in Ulstk1:s1.let stk2 = Lhstk1,capi in Ulstk1:s1.let id = Ulstk: L(s1 ­ Lcap r).stk in Ulstk1:s1.let pwit = Upack(Lcap r,Uhid,idi) in Ulstk1:s1.let hstk2,resi = f [L(s1 ­ Lcap r)] pwit stk2 in Ulstk1:s1.let hstk1,capi = stk2 in Ulstk1:s1.let hi = freergn [r] cap in Ulstk1:s1.Lhstk1,resi

70. Translation: FRGN to lrgnUL, Ops
Construct isomorphism
« letRGN [s1] [t] e ¬ = let f : «8s2. Pf(s1·s2) ! RGN s2 t¬ = «e¬ in Ulstk1:s1.let pack(r,cap) = newrgn Uhi in Ulstk1:s1.let stk2 = Lhstk1,capi in Ulstk1:s1.let id = Ulstk: L(s1 ­ Lcap r).stk in Ulstk1:s1.let pwit = Upack(Lcap r,Uhid,idi) in Ulstk1:s1.let hstk2,resi = f [L(s1 ­ Lcap r)] pwit stk2 in Ulstk1:s1.let hstk1,capi = stk2 in Ulstk1:s1.let hi = freergn [r] cap in Ulstk1:s1.Lhstk1,resi

71. Translation: FRGN to lrgnUL, Ops
« letRGN [s1] [t] e ¬ = let f : «8s2. Pf(s1·s2) ! RGN s2 t¬ = «e¬ in Ulstk1:s1.let pack(r,cap) = newrgn Uhi in Ulstk1:s1.let stk2 = Lhstk1,capi in Ulstk1:s1.let id = Ulstk: L(s1 ­ Lcap r).stk in Ulstk1:s1.let pwit = Upack(Lcap r,Uhid,idi) in Ulstk1:s1.let hstk2,resi = f [L(s1 ­ Lcap r)] pwit stk2 in Ulstk1:s1.let hstk1,capi = stk2 in Ulstk1:s1.let hi = freergn [r] cap in Ulstk1:s1.Lhstk1,resi

72. Encoding Cyclone Features
Many of Cyclone’s features fit into this framework
Lexical Regions
Dynamic Regions
Heap
Reaps
Unique Pointers
See paper for more details.

73. Future Work In practice, capabilities shouldn’t be values
Encode results of region analyses
Aiken et.al. [PLDI’95], Henglein et.al. [PPDP’01]
Combine convenience of monadic encapsulation with power of substructural threading

74. Conclusion
Substructural type systems applicable to encoding region-based memory management
target-level language exposes commonalities in source-level language features
Scope vs. Lifetime
Lexical scope of region name
Un-scoped lifetime of region capability
Unrestricted witnesses vs. Linear capabilities