Linear Regions Are All You Need Matthew Fluet Cornell University Greg Morrisett & Amal Ahmed Harvard University

Memory Management Dynamic allocation pervasive in computation

Memory Management Dynamic allocation pervasive in computation Region-based Memory Management Memory is divided into regions Objects are individually allocated in a region constant-time operation All objects in a region are deallocated together

Application: Cyclone Cyclone Safe-C Project type-safety with the “virtues” of C low-level interface with manifest cost model range of memory management options regions are an organizing principle

Cyclone: Regions Region variety Allocation (objects) Deallocation Aliasing (what) (when) Stack static whole region exit of lexical scope unrestricted Lexical dynamic Dynamic manual Dynamic seq. Heap (`H) single objects automatic (BDW GC) Unique (`U) restricted Ref-counted (`RC)

Application: Cyclone MediaNET TCP benchmark (packet forwarding) Cyclone v.0.1 (lexical regions & garbage collector) High water mark: 840 KB 130 collections Basic throughput: 50 MB/s Cyclone v.0.5 (unique pointers & dynamic regions) High water mark: 8 KB 0 collections Basic throughput: 74MB/s

Application: Cyclone MediaNET TCP benchmark (packet forwarding) Cyclone v.0.1 (lexical regions & garbage collector) High water mark: 840 KB 130 collections Basic throughput: 50 MB/s Cyclone v.0.5 (unique pointers & dynamic regions) High water mark: 8 KB 0 collections Basic throughput: 74MB/s

Proving type safety of Cyclone is a nightmare!! Cyclone: Regions Region variety Allocation (objects) Deallocation Aliasing (what) (when) Stack static whole region exit of lexical scope unrestricted Lexical dynamic Dynamic manual Dynamic seq. Heap (`H) single objects automatic (BDW GC) Unique (`U) restricted Ref-counted (`RC) Proving type safety of Cyclone is a nightmare!!

Cyclone: Regions Region variety Allocation (objects) Deallocation Aliasing (what) (when) Stack static whole region exit of lexical scope unrestricted Lexical dynamic Dynamic manual Dynamic seq. Heap (`H) single objects automatic (BDW GC) Unique (`U) restricted Ref-counted (`RC) Goal: simple model where we can easily encode the key features of Cyclone in a target language with a simpler type system. Minor chord: while the operational extensions were straightforward, it’s the type-system that becomes burdensome. So, maybe the original type system wasn’t quite the right abstraction.

Linear Regions Are All You Need Cyclone: Regions Region variety Allocation (objects) Deallocation Aliasing (what) (when) Stack static whole region exit of lexical scope unrestricted Lexical dynamic Dynamic manual Dynamic seq. Heap (`H) single objects automatic (BDW GC) Unique (`U) restricted Ref-counted (`RC) Linear Regions Are All You Need Minor chord: while the operational extensions were straightforward, it’s the type-system that becomes burdensome. So, maybe the original type system wasn’t quite the right abstraction.

Outline Introduction Monadic Type System (FRGN) [ICFP’04] Substructural Type System (lrgnUL) Translation Sketch Conclusion

Monadic Type System for Regions [ICFP’04] Extend the runST “trick” to nested regions [L-PJ ’94] Polymorphic type system ensures safety Key insights (FRGN): Effects map to an indexed monadic type Region subtyping witnessed by types Sufficient for encoding Tofte-Talpin region calculus and “core” Cyclone region features

RGN monad: Types Monadic type RGN s t computations in stack of regions s returning values of type t; a “stack” transformer “in stack of regions s” means the computation will allocate data in and read/write data from a stack of regions, denoted by the type variable s. RGN is the “region monad,” as ST was the “state monad.”

RGN monad: Operations Monadic unit and bind returnRGN :: 8s,a. a ! RGN s a thenRGN :: 8s,a,b. RGN s a ! (a ! RGN s b) ! RGN s b returnRGN takes a value and gives the trivial computation (that does nothing with the region). thenRGN sequences two computations; the second has access to the value computed by the first.

RGN monad: Operations Monadic unit and bind returnRGN :: 8s,a. a ! RGN s a thenRGN :: 8s,a,b. RGN s a ! (a ! RGN s b) ! RGN s b Note that the operation allows the type of the value computed to change …

RGN monad: Operations Monadic unit and bind returnRGN :: 8s,a. a ! RGN s a thenRGN :: 8s,a,b. RGN s a ! (a ! RGN s b) ! RGN s b … but not the region of the computation.

RGN monad: Types Reference type Ref s t values of type t allocated in region at the top of the stack of regions s

RGN monad: Operations Create and read region allocated values new :: 8s,a. a ! RGN s (Ref s a) read :: 8s,a. Ref s a ! RGN s a Two operators for manipulating region allocated data.

RGN monad: Operations Create and read region allocated values new :: 8s,a. a ! RGN s (Ref s a) read :: 8s,a. Ref s a ! RGN s a As expected, the regions must match up.

RGN monad: Encapsulation Encapsulate and run a monadic computation runRGN :: 8a. (8s. RGN s a) ! a We can mimic runST.

RGN monad: Encapsulation Encapsulate and run a monadic computation runRGN :: 8a. (8s. RGN s a) ! a

RGN monad: Encapsulation Encapsulate and run a monadic computation runRGN :: 8a. (8s. RGN s a) ! a Note that quantification over the stack of regions means no assumptions. “for all stacks” ) no assumptions about stack of regions

RGN monad: Encapsulation Encapsulate and run a monadic computation runRGN :: 8a. (8s. RGN s a) ! a “for all stacks” ) no assumptions about stack of regions

RGN monad: Encapsulation Encapsulate and run a monadic computation runRGN :: 8a. (8s. RGN s a) ! a In particular, the result cannot be a RGNVar s t. result is independent of stack ) s 62 frv(a) ) region values don’t escape “for all stacks” ) no assumptions about stack of regions

RGN monad: Example runRGN ( Ls1. do a à new [s1] 1 c à runRGN ( Ls2. do b à new [s2] 7 … z = … new [s1] z ) … c … ) Note that this uses the Haskell “do” notation; the  symbol corresponds to the thenRGN operation. There are a variety of things wrong with this translation. I’m going to touch on one in particular … r1

input allocated in first region RGN monad: Example runRGN ( Ls1. do a à new [s1] 1 c à runRGN ( Ls2. do b à new [s2] 7 … z = … new [s1] z ) … c … ) Note that this uses the Haskell “do” notation; the  symbol corresponds to the thenRGN operation. There are a variety of things wrong with this translation. I’m going to touch on one in particular … r1 a : 1 input allocated in first region

input allocated in first region RGN monad: Example runRGN ( Ls1. do a à new [s1] 1 c à runRGN ( Ls2. do b à new [s2] 7 … z = … new [s1] z ) … c … ) r2 Note that this uses the Haskell “do” notation; the  symbol corresponds to the thenRGN operation. There are a variety of things wrong with this translation. I’m going to touch on one in particular … r1 a : 1 input allocated in first region

RGN monad: Example runRGN ( Ls1. do a à new [s1] 1 c à runRGN ( Ls2. temporary allocated in second region runRGN ( Ls1. do a à new [s1] 1 c à runRGN ( Ls2. do b à new [s2] 7 … z = … new [s1] z ) … c … ) r2 b : 7 Note that this uses the Haskell “do” notation; the  symbol corresponds to the thenRGN operation. There are a variety of things wrong with this translation. I’m going to touch on one in particular … r1 a : 1 input allocated in first region

RGN monad: Example runRGN ( Ls1. do a à new [s1] 1 c à runRGN ( Ls2. temporary allocated in second region runRGN ( Ls1. do a à new [s1] 1 c à runRGN ( Ls2. do b à new [s2] 7 … z = … new [s1] z ) … c … ) r2 b : 7 Note that this uses the Haskell “do” notation; the  symbol corresponds to the thenRGN operation. There are a variety of things wrong with this translation. I’m going to touch on one in particular … r1 a : 1 input and output allocated in first region c : 8

RGN monad: Example runRGN ( Ls1. do a à new [s1] 1 c à runRGN ( Ls2. temporary allocated in second region runRGN ( Ls1. do a à new [s1] 1 c à runRGN ( Ls2. do b à new [s2] 7 … z = … new [s1] z ) … c … ) Note that this uses the Haskell “do” notation; the  symbol corresponds to the thenRGN operation. There are a variety of things wrong with this translation. I’m going to touch on one in particular … r1 a : 1 input and output allocated in first region c : 8

RGN monad: Example runRGN ( Ls1. do a à new [s1] 1 c à runRGN ( Ls2. do b à new [s2] 7 … z = … new [s1] z ) … c … ) allocating in younger region requires RGN s2 t type Note that allocating in the younger region will result in a computation in region r2, while allocating the result in the older region will result in a computation in region r1. Since these two computations must be sequenced, there is a mismatch. allocating in older region requires RGN s1 t type

RGN monad: Witnesses Witness type Pf(s1 · s2) – type-level proof that the stack of regions s1 is a substack of the stack of regions s2

RGN monad: Witnesses Witness operations coerceRGN :: 8s1,s2,a. Pf(s1 · s2) ! RGN s1 a ! RGN s2 a transSub :: 8s1,s2,s3. Pf(s1 · s2) ! Pf(s2 · s3) ! Pf(s1 · s3)

RGN monad: Regions Regions are created and destroyed with a lexically scoped construct letRGN :: 8s1,a. (8s2. Pf(s1 · s2) ! RGN s2 a) ! RGN s1 a We can mimic runST.

RGN monad: Regions Regions are created and destroyed with a lexically scoped construct letRGN :: 8s1,a. (8s2. Pf(s1 · s2) ! RGN s2 a) ! RGN s1 a We can mimic runST.

RGN monad: Example letRGN ( Ls1. lpf1. do a à new [s1] 1 c à letRGN ( Ls2. lpf2. do b à new [s2] 7 … z = … coerceRgn pf (new [s1] z )) … c … ) r2 b : 7 Note that this uses the Haskell “do” notation; the  symbol corresponds to the thenRGN operation. There are a variety of things wrong with this translation. I’m going to touch on one in particular … r1 a : 1 c : 8

Limitations of LIFO Regions Lexical scope is ill-suited for iterative computations Conway’s Game of Life; copying GC CPS-based computations event-based computations

Limitations of LIFO Regions Lexical scope is ill-suited for iterative computations Conway’s Game of Life; copying GC CPS-based computations event-based computations But, lexical scope was ensuring that the stack of regions was used in a single-threaded manner

Substructural Type Systems Provide core mechanisms to restrict the number and order of uses of data and operations generalization of linear type systems

Substructural Type System: lUL Qualifiers q ::= U j L PreTypes t ::= 1 j t1 £ t2 j t1 ! t2 j 8a.t j 9a.t Types t ::= qt

Substructural Type System: lUL Qualifiers q ::= U j L PreTypes t ::= 1 j t1 £ t2 j t1 ! t2 j 8a.t j 9a.t Types t ::= qt How may the value be used?

Substructural Type System: lUL Qualifiers q ::= U j L PreTypes t ::= 1 j t1 £ t2 j t1 ! t2 j 8a.t j 9a.t Types t ::= qt How often may the value be used? How may the value be used?

Substructural Qualifiers Linear must be “used” exactly once Unrestricted Drop Copy may be “used” an arbitrary # of times

Substructural Type System for Regions Provide core mechanisms to restrict the number and order of uses of data and operations generalization of linear type systems Key insights (lrgnUL): Separate region names from region liveness Region liveness witnessed by types Sufficient for encoding FRGN calculus and “advanced” Cyclone region features

lrgnUL = lUL + Regions PreTypes t ::= … j cap r j ref r t j 8r.t j 9r.t “capability” for region r; mediates all access to a region for allocating, reading, and writing

lrgnUL: Region Primitives Regions are created and destroyed with separate operations newrgn :: U1 ! (9r. Lcap r) freergn :: 8r. (Lcap r ! U1)

lrgnUL: Region Primitives Regions are created and destroyed with separate operations newrgn :: U1 ! (9r. Lcap r) freergn :: 8r. (Lcap r ! U1) Produces a capability. Consumes a capability.

lrgnUL: Region Primitives Regions are created and destroyed with separate operations newrgn :: U1 ! (9r. Lcap r) freergn :: 8r. (Lcap r ! U1)

lrgnUL: Region Primitives new :: 8r,a. ((Lcap r £ Ua) ! (Lcap r £ Uref r Ua)) read :: 8r,a. ((Lcap r £ Uref r Ua) ! (Lcap r £ Ua))

lrgnUL: Region Primitives new :: 8r,a. ((Lcap r £ Ua) ! (Lcap r £ Uref r Ua)) read :: 8r,a. ((Lcap r £ Uref r Ua) ! (Lcap r £ Ua)) Requires a capability. Returns a capability.

lrgnUL: Region Primitives new :: 8r,a. ((Lcap r £ Ua) ! (Lcap r £ Uref r Ua) read :: 8r,a. ((Lcap r £ Uref r Ua) ! (Lcap r £ Ua)

Translation: FRGN to lrgnUL, Types « RGN s t ¬ = U(s ! L(s £ «t¬))

Translation: FRGN to lrgnUL, Types « RGN s t ¬ = s ! (s £ «t¬) operational behavior of monad is store/stack-passing

Translation: FRGN to lrgnUL, Types « RGN s t ¬ = s ! (s £ «t¬) operational behavior of monad is store/stack-passing

Translation: FRGN to lrgnUL, Types « RGN s t ¬ = s ! (s £ «t¬) operational behavior of monad is store/stack-passing represent “stack of regions” as a sequence of linear capabilities, formed out of nested linear tuples

Translation: FRGN to lrgnUL, Types « RGN s t ¬ = s ! (s £ «t¬) operational behavior of monad is store/stack-passing represent “stack of regions” as a sequence of linear capabilities, formed out of nested linear tuples

Translation: FRGN to lrgnUL, Ops « returnRGN [s] [t] e ¬ = let res : «t¬ = «e¬ in Ulstk:s. Lhstk,resi « thenRGN [s] [ta] [tb] e1 e2 ¬ = let f : «RGN s ta¬= «e1¬ in let g : «ta ! RGN s tb¬ = «e2¬ in Ulstk:s. let hstk,resi = f stk in g res stk

Translation: FRGN to lrgnUL, Ops « returnRGN [s] [t] e ¬ = let res : «t¬ = «e¬ in Ulstk:s. Lhstk,resi « thenRGN [s] [ta] [tb] e1 e2 ¬ = let f : «RGN s ta¬= «e1¬ in let g : «ta ! RGN s tb¬ = «e2¬ in Ulstk:s. let hstk,resi = f stk in g res stk Store-passing encoding

Translation: FRGN to lrgnUL, Types « Pf(s1 · s2) ¬ = U(9s’. Iso(s2, L(s1 £ s’)))

Translation: FRGN to lrgnUL, Types « Pf(s1 · s2) ¬ = U(9s’. Iso(s2, L(s1 £ s’))) Isomorphism between s2 and L(s1 £ s’), for some “slack” s’

Translation: FRGN to lrgnUL, Types « Pf(s1 · s2) ¬ = U(9s’. Iso(s2, L(s1 £ s’))) Isomorphism between s2 and L(s1 £ s’), for some “slack” s’ Proof that s1 is a substack of s2 is persistent Liveness of s1 and s2 is ephemeral

Translation: FRGN to lrgnUL, Types « Ref s t ¬ = U(9r. U(U(9s’. Iso(s, L(s’ £ Lcap r))) « Ref s t ¬ = U(9r. U(£ Uref r «t¬))

Translation: FRGN to lrgnUL, Types « Ref s t ¬ = 9r. (9s’. Iso(s, L(s’ £ Lcap r)) « Ref s t ¬ = 9r. (£ Uref r «t¬) Existential fixes region r

Translation: FRGN to lrgnUL, Types « Ref s t ¬ = 9r. (9s’. Iso(s, L(s’ £ Lcap r)) « Ref s t ¬ = 9r. (£ Uref r «t¬) Existential fixes region r Isomorphism witnesses membership of r in s

Translation: FRGN to lrgnUL, Ops « letRGN [s1] [t] e ¬ = let f : «8s2. Pf(s1·s2) ! Hnd s2 ! RGN s2 t¬ = «e¬ in Ulstk1:s1.let pack(r,hcap,hndi) = newrgn Lhi in Ulstk1:s1.let stk2 = Lhstk1,capi in Ulstk1:s1.let id = Ulstk: L(s1 ­ Lcap r).stk in Ulstk1:s1.let pwit = Upack(Lcap r,Uhid,idi) in Ulstk1:s1.let phnd = Upack(r,UhUpack(s1,Uhid,idi),hndi) in Ulstk1:s1.let hstk2,resi = f [L(s1 ­ Lcap r)] pwit phnd stk2 in Ulstk1:s1.let hstk1,capi = stk2 in Ulstk1:s1.let hi = freergn [r] Lhcap,hndi in Ulstk1:s1.Lhstk1,resi

Translation: FRGN to lrgnUL, Ops Stack-passing encoding « letRGN [s1] [t] e ¬ = let f : «8s2. Pf(s1·s2) ! RGN s2 t¬ = «e¬ in Ulstk1:s1.let pack(r,cap) = newrgn Uhi in Ulstk1:s1.let stk2 = Lhstk1,capi in Ulstk1:s1.let id = Ulstk: L(s1 ­ Lcap r).stk in Ulstk1:s1.let pwit = Upack(Lcap r,Uhid,idi) in Ulstk1:s1.let hstk2,resi = f [L(s1 ­ Lcap r)] pwit stk2 in Ulstk1:s1.let hstk1,capi = stk2 in Ulstk1:s1.let hi = freergn [r] cap in Ulstk1:s1.Lhstk1,resi

Translation: FRGN to lrgnUL, Ops Create & destroy a new region « letRGN [s1] [t] e ¬ = let f : «8s2. Pf(s1·s2) ! RGN s2 t¬ = «e¬ in Ulstk1:s1.let pack(r,cap) = newrgn Uhi in Ulstk1:s1.let stk2 = Lhstk1,capi in Ulstk1:s1.let id = Ulstk: L(s1 ­ Lcap r).stk in Ulstk1:s1.let pwit = Upack(Lcap r,Uhid,idi) in Ulstk1:s1.let hstk2,resi = f [L(s1 ­ Lcap r)] pwit stk2 in Ulstk1:s1.let hstk1,capi = stk2 in Ulstk1:s1.let hi = freergn [r] cap in Ulstk1:s1.Lhstk1,resi

Translation: FRGN to lrgnUL, Ops Construct rep. of new stack « letRGN [s1] [t] e ¬ = let f : «8s2. Pf(s1·s2) ! RGN s2 t¬ = «e¬ in Ulstk1:s1.let pack(r,cap) = newrgn Uhi in Ulstk1:s1.let stk2 = Lhstk1,capi in Ulstk1:s1.let id = Ulstk: L(s1 ­ Lcap r).stk in Ulstk1:s1.let pwit = Upack(Lcap r,Uhid,idi) in Ulstk1:s1.let hstk2,resi = f [L(s1 ­ Lcap r)] pwit stk2 in Ulstk1:s1.let hstk1,capi = stk2 in Ulstk1:s1.let hi = freergn [r] cap in Ulstk1:s1.Lhstk1,resi

Translation: FRGN to lrgnUL, Ops Run comp. and recover old stack and new cap « letRGN [s1] [t] e ¬ = let f : «8s2. Pf(s1·s2) ! RGN s2 t¬ = «e¬ in Ulstk1:s1.let pack(r,cap) = newrgn Uhi in Ulstk1:s1.let stk2 = Lhstk1,capi in Ulstk1:s1.let id = Ulstk: L(s1 ­ Lcap r).stk in Ulstk1:s1.let pwit = Upack(Lcap r,Uhid,idi) in Ulstk1:s1.let hstk2,resi = f [L(s1 ­ Lcap r)] pwit stk2 in Ulstk1:s1.let hstk1,capi = stk2 in Ulstk1:s1.let hi = freergn [r] cap in Ulstk1:s1.Lhstk1,resi

Translation: FRGN to lrgnUL, Ops Construct isomorphism « letRGN [s1] [t] e ¬ = let f : «8s2. Pf(s1·s2) ! RGN s2 t¬ = «e¬ in Ulstk1:s1.let pack(r,cap) = newrgn Uhi in Ulstk1:s1.let stk2 = Lhstk1,capi in Ulstk1:s1.let id = Ulstk: L(s1 ­ Lcap r).stk in Ulstk1:s1.let pwit = Upack(Lcap r,Uhid,idi) in Ulstk1:s1.let hstk2,resi = f [L(s1 ­ Lcap r)] pwit stk2 in Ulstk1:s1.let hstk1,capi = stk2 in Ulstk1:s1.let hi = freergn [r] cap in Ulstk1:s1.Lhstk1,resi

Translation: FRGN to lrgnUL, Ops « letRGN [s1] [t] e ¬ = let f : «8s2. Pf(s1·s2) ! RGN s2 t¬ = «e¬ in Ulstk1:s1.let pack(r,cap) = newrgn Uhi in Ulstk1:s1.let stk2 = Lhstk1,capi in Ulstk1:s1.let id = Ulstk: L(s1 ­ Lcap r).stk in Ulstk1:s1.let pwit = Upack(Lcap r,Uhid,idi) in Ulstk1:s1.let hstk2,resi = f [L(s1 ­ Lcap r)] pwit stk2 in Ulstk1:s1.let hstk1,capi = stk2 in Ulstk1:s1.let hi = freergn [r] cap in Ulstk1:s1.Lhstk1,resi

Encoding Cyclone Features Many of Cyclone’s features fit into this framework Lexical Regions Dynamic Regions Heap Reaps Unique Pointers See paper for more details.

Future Work In practice, capabilities shouldn’t be values Encode results of region analyses Aiken et.al. [PLDI’95], Henglein et.al. [PPDP’01] Combine convenience of monadic encapsulation with power of substructural threading

Conclusion Substructural type systems applicable to encoding region-based memory management target-level language exposes commonalities in source-level language features Scope vs. Lifetime Lexical scope of region name Un-scoped lifetime of region capability Unrestricted witnesses vs. Linear capabilities