Memory Management Kathryn McKinley

Isn’t GC a bit retro? Mark-Sweep McCarthy, 1960 Mark-Compact Styger, 1967 Semi-Space Cheney, 1970 “Languages without automated garbage collection are getting out of fashion. The chance of running into all kinds of memory problems is gradually outweighing the performance penalty you have to pay for garbage collection.” Paul Jansen, managing director of TIOBE Software, in Dr Dobbs, April 2008 McKinley, UT 395T MM

Course Logistics Syllabus Critical reading & writing Presentations Discussion Critiques Schedule Volunteers for next week? McKinley, UT 395T MM

Outline Briefly introduce the challenges and key ideas in memory management Context – modern VMs Explicit vs automatic Memory organization Allocation Garbage Identification Reclamation McKinley, UT 395T MM

Class Loader Verifier, etc. Basic VM Structure Program/Bytecode Executing Program Class Loader Verifier, etc. Heap Thread Scheduler Interpreter &/or Dynamic Compiler Garbage Collector McKinley, UT 395T MM

Dynamic memory allocation and reclamation Heap contains dynamically allocated objects Object allocation: malloc, new Deallocation: Manual/explicit: free, delete automatic: garbage collection McKinley, UT 395T MM

Memory Management Objects/data in heap memory How does the runtime system efficiently create and recycle memory on behalf of the program? What makes this problem important? What makes this problem hard? Why are researchers still working on it? McKinley, UT 395T MM

Explicit memory management challenges More code to maintain Correctness Free an object too soon - core dump Free an object too late - waste space Never free - at best waste, at worst fail Efficiency can be very high Gives programmers “control” McKinley, UT 395T MM

Garbage collection: Automatic memory management Reduces programmer burden Eliminates sources of errors which ones? Integral to modern object-oriented languages Java, C#, PHP, JavaScript Mainstream Challenge: performance McKinley, UT 395T MM

Why use Garbage Collection? Software engineering benefits Less user code compared to expilict memory management (MM) Less user code to get correct Protects against some classes of memory errors No free(), thus no premature free(), no double free(), or forgetting to free() Not perfect, memory can still leak Programmers still need to eliminate all pointers to objects the program no longer needs McKinley, UT 395T MM

Why use Garbage Collection? Performance: space/time tradeoff Time proportional to dead objects (explicit mm, reference counting) or live objects (semi-space, mark-sweep) Throughput versus pause time Less frequent collection typically reduces total time but increases space requirements and pause times Hidden locality benefits? Layer of abstraction What else can the collector do when it visits all objects? McKinley, UT 395T MM

Key Issues For both Garbage Collection Fast allocation Fast reclamation Low fragmentation (wasted space) How to organize the memory space Garbage Collection Discriminating live objects and garbage McKinley, UT 395T MM

What is Garbage? McKinley, UT 395T MM

Perfect live object detection Live object has a future use Prove that object is not live, and deallocate it Deallocate as soon as possible after last use McKinley, UT 395T MM

Estimating liveness in practice Approximate liveness by reachability from outside the heap An unreachable object cannot ever be used---it is garbage Once dead always dead! Find and preserve reachable objects Tracing or reference counting Recycle the space of garbage objects McKinley, UT 395T MM

How does the GC implement reachability? McKinley, UT 395T MM

How does the GC implement reachability? Tracing Counting McKinley, UT 395T MM

How does the GC find the pointers to trace or count? Managed languages couple GC with safe pointers Programs may not access arbitrary addresses in memory Compiler can identify and provide the GC with all the pointers….enforcing: “Once garbage, always garbage” Runtime system can move objects by updating pointers Unsafe languages can do non-moving GC by assuming anything that looks like a pointer is one. McKinley, UT 395T MM

Reachability with tracing Compiler produces a stack-map at GC safe-points and Type Information Blocks GC safe points: new(), method entry, method exit, & back-edges (thread switch points) Stack-map: enumerate global variables, stack variables, live registers -- This code is hard to get right! Why? Type Information Blocks: identify reference fields in objects A B C { .... r0 = obj PC -> p.f = obj globals stack registers heap McKinley, UT 395T MM

Reachability with tracing Compiler produces a stack-map at GC safe-points and Type Information Blocks Type Information Blocks: identify reference fields in objects for each type i (class) in the program, a map TIBi 2 3 A B C { .... r0 = obj PC -> p.f = obj globals stack registers heap McKinley, UT 395T MM

Reachability with tracing Tracing collector (semispace, marksweep) Marks the objects reachable from the roots live, and then performs a transitive closure over them mark A B C { .... r0 = obj PC -> p.f = obj globals stack registers heap McKinley, UT 395T MM

Reachability with tracing Tracing collector (semispace, marksweep) Marks the objects reachable from the roots live, and then performs a transitive closure over them mark A B C { .... r0 = obj PC -> p.f = obj globals stack registers heap McKinley, UT 395T MM

Reachability with tracing Tracing collector (semispace, marksweep) Marks the objects reachable from the roots live, and then performs a transitive closure over them mark A B C { .... r0 = obj PC -> p.f = obj globals stack registers heap McKinley, UT 395T MM

Reachability with tracing Tracing collector (semispace, marksweep) Marks the objects reachable from the roots live, and then performs a transitive closure over them All unmarked objects are dead, and can be reclaimed mark A B C { .... r0 = obj PC -> p.f = obj globals stack registers heap McKinley, UT 395T MM

Reachability with tracing Tracing collector (semispace, marksweep) Marks the objects reachable from the roots live, and then performs a transitive closure over them All unmarked objects are dead, and can be reclaimed sweep A B C { .... r0 = obj PC -> p.f = obj globals stack registers heap McKinley, UT 395T MM

Taxonomy of GC design choices Heap Organization Incrementality Composability Concurrency Parallelism Distribution McKinley, UT 395T MM

Heap organization & basic algorithmic components Allocation Identification Reclamation Sweep-to-Free ` Tracing (implicit) Free List Compact Evacuate Reference Counting (explicit) Bump Allocation A GC algorithm can be broken down into of three algorithmic components. How objects are allocated, how garbage is identified, and how space is reclaimed. 3 1 Sweep-to-Region McKinley, UT 395T MM

One Big Heap? Incrementality Pause times it takes too long to trace the whole heap at once Throughput the heap contains lots of long lived objects, why collect them over and over again? Incremental collection divide up the heap into increments and collect one at a time. to space from space to space from space Increment 1 Increment 2 McKinley, UT 395T MM

Incremental Collection Ideally perfect pointer knowledge of live pointers between increments requires scanning whole heap, defeats the purpose to space from space to space from space Increment 1 Increment 2 McKinley, UT 395T MM

Incremental Collection Ideally perfect pointer knowledge of live pointers between increments requires scanning whole heap, defeats the purpose to space from space to space from space Increment 1 Increment 2 McKinley, UT 395T MM

Incremental Collection Ideally perfect pointer knowledge of live pointers between increments requires scanning whole heap, defeats the purpose to space from space to space from space Increment 1 Increment 2 McKinley, UT 395T MM

Incremental Collection Ideally perfect pointer knowledge of live pointers between increments requires scanning whole heap, defeats the purpose Mechanism: Write barrier records pointers between increments when the mutator installs them, conservative approximation of reachability to space from space to space from space Increment 1 Increment 2 McKinley, UT 395T MM

Write barrier Increment 1 Increment 2 compiler inserts code that records pointers between increments when the mutator installs them // original program // compiler support for incremental collection p.f = o; if (incr(p) != incr(o) { remembered set (incr(o)) U p.f; } p.f = o; remset1 ={w} remset2 ={f,g} a b c d e f g t u v w x y z to space from space to space from space Increment 1 Increment 2 McKinley, UT 395T MM

Write barrier Increment 1 Increment 2 Install new pointer d -> v // original program // compiler support for incremental collection p.f = o; if (incr(p) != incr(o) { remembered set (incr(o)) U p.f; } p.f = o; remset1 ={w} remset2 ={f,g} a b c d e f g t u v w x y z to space from space to space from space Increment 1 Increment 2 McKinley, UT 395T MM

Write barrier Increment 1 Increment 2 Install new pointer d -> v, then update d-> y // original program // compiler support for incremental collection p.f = o; if (incr(p) != incr(o) { remembered set (incr(o)) = p.f; } p.f = o; remset1 ={w} remset2 ={f,g,d} a b c d e f g t u v w x y z to space from space to space from space Increment 1 Increment 2 McKinley, UT 395T MM

Write barrier Increment 1 Increment 2 Install new pointer d -> v, then update d-> y // original program // compiler support for incremental collection p.f = o; if (incr(p) != incr(o) { remembered set (incr(o)) = p.f; } p.f = o; remset1 ={w} remset2 ={f,g,d,d} a b c d e f g t u v w x y z to space from space to space from space Increment 1 Increment 2 McKinley, UT 395T MM

Write barrier At collection time collector re-examines all entries in the remset for the increment, treating them like roots Collect Increment 2 remset1 ={w} remset2 ={f,g,d,d} a b c d e f g t u v w x y z to space from space to space from space Increment 1 Increment 2 McKinley, UT 395T MM

Write barrier At collection time collector re-examines all entries in the remset for the increment, treating them like roots Collect Increment 2 remset1 ={w} remset2 ={f,g,d,d} a b c d e f g t u v w x y z to space from space to space from space Increment 1 Increment 2 McKinley, UT 395T MM

Summary of the costs of incremental collection write barrier to catch pointer stores crossing boundaries remsets to store crossing pointers processing remembered sets at collection time excess retention remset1 ={w} remset2 ={f,g,d,d} a b c d e f g t u v w x y z to space from space to space from space Increment 1 Increment 2 McKinley, UT 395T MM

Taxonomy of Design Choices Incrementality Composability Concurrency Parallelism Distribution McKinley, UT 395T MM

GC Ideas Generational collection - Young objects die fast Older first - The longer you wait Garbage first Immix Reference counting - Deferred & Ulterior Concurrent collection [Steele, Djkistra et al.’78] at the same time as the mutator Parallel collection Real time make pause times tiny [Metronome, Petrank et al.] What else can you do when you visit all the objects? Memory utilization & fragmentation Leaks McKinley, UT 395T MM

Questions?