1. Type Systems For Distributed Data Sharing
Ben Liblit, Alex Aiken, and Katherine Yelick
University of California, Berkeley

2. Why Shared/Private Matters
Data location management
Cache coherence
Race condition detection
Program/algorithm documentation
Consistency model relaxation
Synchronization elimination
Autonomous garbage collection
Security

3. Highlights of This Talk
Review of underlying memory model
Originally in [Liblit et al, POPL ’00]
Captures representation but not sharing
Suite of type systems for data sharing
One size does not fit all
Overview of type inference
Selected experimental findings

4. Distributed Memory Model
Multiple machines, each with local memory
Global memory is union of local memories
Distinguish two types of pointers:
Local points to local memory only: address
Global points anywhere: machine, address
Different representations & operations

5. Type Grammar Integers and pointers; unboxed pairs in paper
All indirection (boxing) is explicit
Pointers are either local or global
Coercion, not subtyping

6. Review of Global Dereferencing: Standard Approach Unsound
x = 5
x =

7. Review of Global Dereferencing: Standard Approach Unsound
x =  5
x =

8. Review of Global Dereferencing: Sound With Type Expansion5
x =

9. Review of Global Dereferencing: Sound With Type Expansion

10. Representation Versus Sharing
Consider obvious assumptions:
local pointers address private data
global pointers address shared data 5

11. Representation Versus Sharing
Locally pointed-to data might not be private 5

12. Representation Versus Sharing
Locally pointed-to data might not be private
Because of local / global aliasing
x = 5

13. Representation Versus Sharing
Locally pointed-to data might not be private
Because of transitivity + pointer widening
y = 5
y =

14. Representation Versus Sharing
Globally pointed-to data might not be shared
What if “y” never actually happens?
y = 5
y =

15. Distinct, but not Independent
Local pointer to shared data: 
Local pointer to private data: 
Global pointer to shared data: 
Global pointer to private data: ?!?
Several possible approaches
Determines what “private” really means
Determines which clients can benefit

16. Sharing Qualifiers Polymorphism needed in practice Top, but no bottom
Mixed: supertype of shared & private
Local access only, but assume others may be watching
Top, but no bottom
mixed = 
shared
private

17. Augmented Type Grammar
Allow subtyping of pointers
But not across pointers, since we allow assignment
Allocation is explicitly shared or private

18. Late Enforcement: Limited Use of Global Pointers

19. Late Enforcement: Applicability
Data location management
Cache coherence
Race condition detection
Program/algorithm documentation
Consistency model relaxation
Synchronization elimination
Autonomous garbage collection (in practice)
Security

20. Why Garbage Collection Breaks
Locally allocate some private data 5

21. Why Garbage Collection Breaks
Locally allocate some private data
Send its address to another machine 5

22. Why Garbage Collection Breaks
Forget the original local pointer 5

23. Why Garbage Collection Breaks
Forget the original local pointer
Garbage collect unreachable private data

24. Why Garbage Collection Breaks
Later, retrieve the global pointer
Coerce back to local (runtime check)

25. Export Enforcement: No Escape of Private Addresses
Note that τ′ might reference private data
Autonomous garbage collection: OK
Security: not OK

26. Early Enforcement: Shared is Transitively Closed

27. Recap of Enforcement Strategies
Late enforcement
Anything can point to anything
Restricted global dereference & assignment
y = 3 5

28. Recap of Enforcement Strategies
Export enforcement
Can only reveal shared addresses
Still restrict global pointer operations
y = 3 5

29. Recap of Enforcement Strategies
Early enforcement
Shared universe is transitively closed
Global pointer restrictions trivially satisfied
y = 3 5

30. Type Inference: Constraint Generation
Type structure already known
Including local / global
Induce constraints on sharing qualifiers
δ = shared from global deref / assign
δ ≤ δ′ from assignments
δ = δ′ from various other operations
Stricter enforcement adds more constraints
δ = shared Þ δ′ = shared

31. Type Inference: Constraint Resolutionδ1 δ2
private δ
shared
Given constraints
δ ≤ δ1 shared ≤ δ1
δ ≤ δ2 private ≤ δ2

32. Type Inference: Constraint Resolution
δ1  mixed
δ2  shared
private
δ  shared
shared
Two “minimal” solutions
δ  shared Þ δ1  mixed Ù δ2  shared

33. Type Inference: Constraint Resolution
δ1  private
δ2  mixed
private
δ  private
shared
Two “minimal” solutions
δ  shared Þ δ1  mixed Ù δ2  shared
δ  private Þ δ1  private Ù δ2  mixed

34. Type Inference: Biased Constraint Resolutionδ1
shared ≤ δ2
private δ
shared
Push “shared” and “mixed” forward

35. Type Inference: Biased Constraint Resolutionδ1
shared ≤ δ2
private δ
shared
Push “shared” and “mixed” forward
Identify qualifiers which cannot be private

36. Type Inference: Biased Constraint Resolution
δ1  private
shared ≤ δ2
private
δ  private
shared
Push “shared” and “mixed” forward
Identify qualifiers which cannot be private
Set all other qualifiers to private

37. Type Inference: Biased Constraint Resolution
shared ≤ δ2
private ≤ δ2
δ1  private
private
δ  private
shared
Identify qualifiers which cannot be private
Set all other qualifiers to private
Push “private” forward

38. Type Inference: Biased Constraint Resolution
δ1  private
δ2  mixed
private
δ  private
shared
Set all other qualifiers to private
Push “private” forward
Set remaining qualifiers to “shared” or “mixed”

39. Implementation For Titanium
Java + SPMD extensions
Objects, classes, interfaces, methods
Multidimensional arrays, templates
Local / global, communications primitives
Sharing validation as type checking
Sharing inference as compiler analysis
Late or early enforcement
Whole-program or partial

40. Experimental Findings: Consistency Model Relaxation
Titanium has very weak consistency model
Sequential model preferred, but too slow?
Sequential is overkill for private data
Weakly consistent on private data
Sequentially consistent on shared data
Compare to weak & fully sequential models
Four-way Pentium III SMP at 550 MHz

41. Experimental Findings: Consistency Model Relaxation

42. Experimental Findings: Data Location Management
Tally allocations by type at run time
Tremendous variation
1% - 100% of allocated bytes are private
45% in large gas benchmark
Sensitivity to enforcement policy
amr: 74% late / 19% early

43. Summary “Private” might not mean what you think
Generalize on earlier (often implicit) designs
Amenable to efficient type inference
Experimental implementation
Ideas & algorithms scale to real system
More aggressive clients needed
Potential for stronger, phase-aware inference