A Mostly Non-Copying Real-Time Collector with Low Overhead and Consistent Utilization David Bacon Perry Cheng (presenting) V.T. Rajan IBM T.J. Watson Research

n What is Real-time Garbage Collection? n Pause Time, CPU utilization (MMU), and Space Usage n Heap Architecture n Types of Fragmentation n Incremental Compaction n Read Barriers n Barrier Performance n Scheduling: Time-Based vs. Work-Based n Empirical Results n Pause Time Distribution n Minimum Mutator Utilization (MMU) n Pause Times n Summary and Conclusion Roadmap

n Real-time Embedded Systems n Memory usage important n Uniprocessor Problem Domain

3 Styles of Uniprocessor Garbage Collection: Stop-the-World vs. Incremental vs. Real-Time STW Inc RT time

Pause Times (Average and Maximum) STW Inc RT 1.5s 1.7s 0.5s 0.7s0.3s 0.5s0.9s 0.3s s 1.6s 0.5s 0.18s

Coarse-Grained Utilization vs. Time STW Inc RT 2.0 s window

Fine-Grained Utilization vs. Time STW Inc RT 0.4 s window

Minimum Mutator Utilization (MMU) STW Inc RT

Space Usage over Time max live trigger 2 X max live

Problems with Existing RT Collectors max live 2 X max live 3 X max live 4 X max live Non-moving Collector max live 2 X max live 3 X max live 4 X max live Replicating Collector Not fully incremental, Tight coupling, Work-based scheduling

Our Collector n GoalsResults n Real-Time ~10 ms n Low Space Overhead ~2X n Good Utilization during GC ~ 40% n Solution n Incremental Mark-Sweep Collector n Write barrier – snapshot-at-the-beginning [Yuasa] n Segregated free list heap architecture n Read Barrier – to support defragmentation [Brooks] n Incremental defragmentation n Segmented arrays – to bound fragmentation

n What is Real-time Garbage Collection? n Pause Time, CPU utilization (MMU), and Space Usage n Heap Architecture n Types of Fragmentation n Incremental Compaction n Read Barriers n Barrier Performance n Scheduling: Time-Based vs. Work-Based n Empirical Results n Pause Time Distribution n Minimum Mutator Utilization (MMU) n Pause Times n Summary and Conclusion Roadmap

Fragmentation and Compaction n Intuitively: available but unusable memory è avoidance and coalescing - no guarantees è compaction use d neede d fre e

Heap Architecture n Segregated Free Lists – heap divided into pages – each page has equally-sizes blocks (1 object per block) – Large arrays are segmented usedfree sz 24 sz 32 external internal page-internal

Controlling Internal and Page-Internal Fragmentation n Choose page size (page) and block sizes (s k ) n If s k = s k-1 (1 + q ), internal fragmentation [ q n page-internal fragmentation [ page / s max n E.g. If page = 16K, q = 1/8, s max = 2K, maximum non-external fragmentation to 12.5%.

Fragmentation - small heap ( q = 1/8 vs. q = 1/2 ) q =1/8 q =1/2

Incremental Compaction n Compact only a part of the heap è Requires knowing what to compact ahead of time n Key Problems è Popular objects è Determining references to moved objects use d

Incremental Compaction: Redirection n Access all objects via per-object redirection pointers n Redirection is initially self-referential n Move an object by updating ONE redirection pointer original replica

Consistency via Read Barrier [Brooks] n Correctness requires always using the replica n E.g. field selection must be modified x[offset] x x[redirect][offset ] x normal access read barrier access x

Some Important Details n Our read barrier is decoupled from collection n Complication: In Java, any reference might be null n actual read barrier for GetField(x,offset) must be augmented tmp = x[offset]; return (tmp == null) ? null : tmp[redirect] è CSE, code motion (LICM and sinking), null-check combining n Barrier Variants - when to redirect è lazy - easier for collector è eager - better for optimization

Barrier Overhead to Mutator n Conventional wisdom says read barriers are too expensive è Studies found overhead of 20-40% (Zorn, Nielsen) è Our barrier has 4-6% overhead with optimizations

Heap (one size only)Stack Program Start

HeapStack free allocated Program is allocating

HeapStack free unmarked GC starts

HeapStack free unmarked marked or allocated Program allocating and GC marking

HeapStack free unmarked marked or allocated Sweeping away blocks

HeapStack free allocated evacuated GC moving objects and installing redirection

HeapStack free unmarked evacuated marked or allocated 2 nd GC starts tracing and redirection fixup

HeapStack free allocated 2 nd GC complete

n What is Real-time Garbage Collection? n Pause Time, CPU utilization (MMU), and Space Usage n Heap Architecture n Types of Fragmentation n Incremental Compaction n Read Barriers n Barrier Performance n Scheduling: Time-Based vs. Work-Based n Empirical Results n Pause Time Distribution n Minimum Mutator Utilization (MMU) n Pause Times n Summary and Conclusion Roadmap

Scheduling the Collector n Scheduling Issues n bad CPU utilization and space usage n loose program and collector coupling n Time-Based n Trigger the collector to run for C T seconds whenever the program runs for Q T seconds n Work-Based n Trigger the collector to collect C W work whenever the program allocate Q W bytes

Time-Based Scheduling n Trigger the collector to run for C T seconds whenever the program runs for Q T seconds Space (Mb) Time (s) MMU (CPU Utilization) Window Size (s)

Work-Based Scheduling MMU (CPU Utilization) n Trigger the collector to collect C W bytes whenever the program allocates Q W bytes Window Size (s) Space (Mb) Time (s)

n What is Real-time Garbage Collection? n Pause Time, CPU utilization (MMU), and Space Usage n Heap Architecture n Types of Fragmentation n Incremental Compaction n Read Barriers n Barrier Performance n Scheduling: Time-Based vs. Work-Based n Empirical Results n Pause Time Distribution n Minimum Mutator Utilization (MMU) n Pause Times n Summary and Conclusion Roadmap

Pause Time Distribution for javac (Time-Based vs. Work-Based) 12 ms

Utilization vs. Time for javac (Time-Based vs. Work-Based) Utilization (%) Time (s) Utilization (%) 0.45

Minimum Mutator Utilization for javac (Time-Based vs. Work-Based)

Space Usage for javac (Time-Based vs. Work-Based)

n 3 inter-related factors: Space Bound (tradeoff) Utilization (tradeoff) Allocation Rate (lower is better) n Other factors Collection rate (higher is better) Pointer density (lower is better) Intrinsic Tradeoff

Summary: Mostly Non-moving RT GC n Read Barriers n Permits incremental defragmentation n Overhead is 4-6% with compiler optimizations n Low Space Overhead n Space usage is only about 2 X max live data n Fragmentation still bounded n Consistent Utilization n Always at least 45% at 12 ms resolution

Conclusions n Real-time GC is real n There are tradeoffs just like in traditional GC n Scheduling should be primarily time-based n Fallback to work-based due to user’s incorrect parameter estimations n Incremental defragmentation is possible n Compiler support is important!

Future Work n Lowering the real-time resolution n Sub-millisecond worst-case pause n Main issue: breaking up stack scan n Segmented array optimizations n Reduce segmented array cost below ~2% n Opportunistic contiguous layout n Type-based specialization with invalidation n Strip-mining