1. Memory Management
Kathryn McKinley

2. Isn’t GC a bit retro? Mark-Sweep McCarthy, 1960 Mark-Compact
Styger, 1967
Semi-Space
Cheney, 1970
“Languages without automated garbage collection are getting out of fashion. The chance of running into all kinds of memory problems is gradually outweighing the performance penalty you have to pay for garbage collection.”
Paul Jansen, managing director of TIOBE Software, in Dr Dobbs, April 2008
McKinley, UT 395T MM

3. Course Logistics Syllabus Critical reading & writing Presentations
Discussion
Critiques
Schedule
Volunteers for next week?
McKinley, UT 395T MM

4. Outline
Briefly introduce the challenges and key ideas in memory management
Context – modern VMs
Explicit vs automatic
Memory organization
Allocation
Garbage Identification
Reclamation
McKinley, UT 395T MM

5. Class Loader Verifier, etc.
Basic VM Structure
Program/Bytecode
Executing Program
Class Loader Verifier, etc.
Heap
Thread Scheduler
Interpreter &/or
Dynamic Compiler
Garbage Collector
McKinley, UT 395T MM

6. Dynamic memory allocation and reclamation
Heap contains dynamically allocated objects
Object allocation: malloc, new
Deallocation:
Manual/explicit: free, delete
automatic: garbage collection
McKinley, UT 395T MM

7. Memory Management Objects/data in heap memory
How does the runtime system efficiently create and recycle memory on behalf of the program?
What makes this problem important?
What makes this problem hard?
Why are researchers still working on it?
McKinley, UT 395T MM

8. Explicit memory management challenges
More code to maintain
Correctness
Free an object too soon - core dump
Free an object too late - waste space
Never free - at best waste, at worst fail
Efficiency can be very high
Gives programmers “control”
McKinley, UT 395T MM

9. Garbage collection: Automatic memory management
Reduces programmer burden
Eliminates sources of errors
which ones?
Integral to modern object-oriented languages
Java, C#, PHP, JavaScript
Mainstream
Challenge: performance
McKinley, UT 395T MM

10. Why use Garbage Collection?
Software engineering benefits
Less user code compared to expilict memory management (MM)
Less user code to get correct
Protects against some classes of memory errors
No free(), thus no premature free(), no double free(), or forgetting to free()
Not perfect, memory can still leak
Programmers still need to eliminate all pointers to objects the program no longer needs
McKinley, UT 395T MM

11. Why use Garbage Collection?
Performance: space/time tradeoff
Time proportional to dead objects (explicit mm, reference counting) or live objects (semi-space, mark-sweep)
Throughput versus pause time
Less frequent collection typically reduces total time but increases space requirements and pause times
Hidden locality benefits?
Layer of abstraction
What else can the collector do when it visits all objects?
McKinley, UT 395T MM

12. Key Issues For both Garbage Collection Fast allocation
Fast reclamation
Low fragmentation (wasted space)
How to organize the memory space
Garbage Collection
Discriminating live objects and garbage
McKinley, UT 395T MM

13. What is Garbage?
McKinley, UT 395T MM

14. Perfect live object detection
Live object has a future use
Prove that object is not live, and deallocate it
Deallocate as soon as possible after last use
McKinley, UT 395T MM

15. Estimating liveness in practice
Approximate liveness by reachability from outside the heap
An unreachable object cannot ever be used---it is garbage
Once dead always dead!
Find and preserve reachable objects
Tracing or reference counting
Recycle the space of garbage objects
McKinley, UT 395T MM

16. How does the GC implement reachability?
McKinley, UT 395T MM

17. How does the GC implement reachability?
Tracing
Counting
McKinley, UT 395T MM

18. How does the GC find the pointers to trace or count?
Managed languages couple GC with safe pointers
Programs may not access arbitrary addresses in memory
Compiler can identify and provide the GC with all the pointers….enforcing:
“Once garbage, always garbage”
Runtime system can move objects by updating pointers
Unsafe languages can do non-moving GC by assuming anything that looks like a pointer is one.
McKinley, UT 395T MM

19. Reachability with tracing
Compiler produces a stack-map at GC safe-points and Type Information Blocks
GC safe points: new(), method entry, method exit, & back-edges (thread switch points)
Stack-map: enumerate global variables, stack variables, live registers -- This code is hard to get right! Why?
Type Information Blocks: identify reference fields in objects A B C {
....
r0 = obj
PC -> p.f = obj
globals
stack
registers
heap
McKinley, UT 395T MM

20. Reachability with tracing
Compiler produces a stack-map at GC safe-points and Type Information Blocks
Type Information Blocks: identify reference fields in objects
for each type i (class) in the program, a map
TIBi 2 3 A B C {
....
r0 = obj
PC -> p.f = obj
globals
stack
registers
heap
McKinley, UT 395T MM

21. Reachability with tracing
Tracing collector (semispace, marksweep)
Marks the objects reachable from the roots live, and then performs a transitive closure over them
mark A B C {
....
r0 = obj
PC -> p.f = obj
globals
stack
registers
heap
McKinley, UT 395T MM

22. Reachability with tracing
Tracing collector (semispace, marksweep)
Marks the objects reachable from the roots live, and then performs a transitive closure over them
mark A B C {
....
r0 = obj
PC -> p.f = obj
globals
stack
registers
heap
McKinley, UT 395T MM

23. Reachability with tracing
Tracing collector (semispace, marksweep)
Marks the objects reachable from the roots live, and then performs a transitive closure over them
mark A B C {
....
r0 = obj
PC -> p.f = obj
globals
stack
registers
heap
McKinley, UT 395T MM

24. Reachability with tracing
Tracing collector (semispace, marksweep)
Marks the objects reachable from the roots live, and then performs a transitive closure over them
All unmarked objects are dead, and can be reclaimed
mark A B C {
....
r0 = obj
PC -> p.f = obj
globals
stack
registers
heap
McKinley, UT 395T MM

25. Reachability with tracing
Tracing collector (semispace, marksweep)
Marks the objects reachable from the roots live, and then performs a transitive closure over them
All unmarked objects are dead, and can be reclaimed
sweep A B C {
....
r0 = obj
PC -> p.f = obj
globals
stack
registers
heap
McKinley, UT 395T MM

26. Taxonomy of GC design choices
Heap Organization
Incrementality
Composability
Concurrency
Parallelism
Distribution
McKinley, UT 395T MM

27. Heap organization & basic algorithmic components
Allocation
Identification
Reclamation
Sweep-to-Free `
Tracing
(implicit)
Free List
Compact
Evacuate
Reference Counting
(explicit)
Bump Allocation
A GC algorithm can be broken down into of three algorithmic components. How objects are allocated, how garbage is identified, and how space is reclaimed. 3 1
Sweep-to-Region
McKinley, UT 395T MM

28. One Big Heap? Incrementality
Pause times
it takes too long to trace the whole heap at once
Throughput
the heap contains lots of long lived objects, why collect them over and over again?
Incremental collection
divide up the heap into increments and collect one at a time.
to space
from space
to space
from space
Increment Increment 2
McKinley, UT 395T MM

29. Incremental Collection
Ideally
perfect pointer knowledge of live pointers between increments
requires scanning whole heap, defeats the purpose
to space
from space
to space
from space
Increment Increment 2
McKinley, UT 395T MM

30. Incremental Collection
Ideally
perfect pointer knowledge of live pointers between increments
requires scanning whole heap, defeats the purpose
to space
from space
to space
from space
Increment Increment 2
McKinley, UT 395T MM

31. Incremental Collection
Ideally
perfect pointer knowledge of live pointers between increments
requires scanning whole heap, defeats the purpose
to space
from space
to space
from space
Increment Increment 2
McKinley, UT 395T MM

32. Incremental Collection
Ideally
perfect pointer knowledge of live pointers between increments
requires scanning whole heap, defeats the purpose
Mechanism: Write barrier
records pointers between increments when the mutator installs them, conservative approximation of reachability
to space
from space
to space
from space
Increment Increment 2
McKinley, UT 395T MM

33. Write barrier Increment 1 Increment 2
compiler inserts code that records pointers between increments when the mutator installs them
// original program // compiler support for incremental collection
p.f = o; if (incr(p) != incr(o) {
remembered set (incr(o)) U p.f; }
p.f = o;
remset1 ={w}
remset2 ={f,g}
a b c d e f g
t u v w x y z
to space
from space
to space
from space
Increment Increment 2
McKinley, UT 395T MM

34. Write barrier Increment 1 Increment 2 Install new pointer d -> v
// original program // compiler support for incremental collection
p.f = o; if (incr(p) != incr(o) {
remembered set (incr(o)) U p.f; }
p.f = o;
remset1 ={w}
remset2 ={f,g}
a b c d e f g
t u v w x y z
to space
from space
to space
from space
Increment Increment 2
McKinley, UT 395T MM

35. Write barrier Increment 1 Increment 2
Install new pointer d -> v, then update d-> y
// original program // compiler support for incremental collection
p.f = o; if (incr(p) != incr(o) {
remembered set (incr(o)) = p.f; }
p.f = o;
remset1 ={w}
remset2 ={f,g,d}
a b c d e f g
t u v w x y z
to space
from space
to space
from space
Increment Increment 2
McKinley, UT 395T MM

36. Write barrier Increment 1 Increment 2
Install new pointer d -> v, then update d-> y
// original program // compiler support for incremental collection
p.f = o; if (incr(p) != incr(o) {
remembered set (incr(o)) = p.f; }
p.f = o;
remset1 ={w}
remset2 ={f,g,d,d}
a b c d e f g
t u v w x y z
to space
from space
to space
from space
Increment Increment 2
McKinley, UT 395T MM

37. Write barrier At collection time
collector re-examines all entries in the remset for the increment, treating them like roots
Collect Increment 2
remset1 ={w}
remset2 ={f,g,d,d}
a b c d e f g
t u v w x y z
to space
from space
to space
from space
Increment Increment 2
McKinley, UT 395T MM

38. Write barrier At collection time
collector re-examines all entries in the remset for the increment, treating them like roots
Collect Increment 2
remset1 ={w}
remset2 ={f,g,d,d}
a b c d e f g
t u v w x y z
to space
from space
to space
from space
Increment Increment 2
McKinley, UT 395T MM

39. Summary of the costs of incremental collection
write barrier to catch pointer stores crossing boundaries
remsets to store crossing pointers
processing remembered sets at collection time
excess retention
remset1 ={w}
remset2 ={f,g,d,d}
a b c d e f g
t u v w x y z
to space
from space
to space
from space
Increment Increment 2
McKinley, UT 395T MM

40. Taxonomy of Design Choices
Incrementality
Composability
Concurrency
Parallelism
Distribution
McKinley, UT 395T MM

41. GC Ideas Generational collection - Young objects die fast
Older first - The longer you wait
Garbage first
Immix
Reference counting - Deferred & Ulterior
Concurrent collection [Steele, Djkistra et al.’78]
at the same time as the mutator
Parallel collection
Real time
make pause times tiny [Metronome, Petrank et al.]
What else can you do when you visit all the objects?
Memory utilization & fragmentation
Leaks
McKinley, UT 395T MM

42. Questions?