Garbage Collection Advantage: Improving Program Locality Xianglong Huang (UT) Stephen M Blackburn (ANU), Kathryn S McKinley (UT) J Eliot B Moss (UMass), Zhenlin Wang (MTU), Perry Cheng (IBM) Special Thanks to authors for providing many of these slides

Motivation Memory gap problem OO programs become more popular OO programs exacerbates memory gap problem Automatic memory management Pointer data structures Many small methods Goal: improve OO program locality

Cache Performance Matters

Opportunity Generational copying garbage collector reorders objects at runtime

Copying of Linked Objects 1 1 4 4 2 2 3 3 6 6 7 7 5 5 Breadth First

Copying of Linked Objects 1 1 4 4 2 2 3 3 6 6 7 7 5 5 Breadth First 1 2 3 4 5 6 7 Depth First

Copying of Linked Objects 1 1 4 4 2 2 3 3 6 6 7 7 5 5 Breadth First 1 1 2 3 4 4 5 6 7 Depth First 1 1 2 3 5 4 4 6 7 Online Object Reordering

Outline Motivation Online Object Reordering (OOR) Methodology Experimental Results Conclusion

OOR System Overview Records object accesses in each method (excludes cold basic blocks) Finds hot methods by adaptive sampling Reorders objects with hot fields in older generation during GC Copies hot objects into separate region

Online Object Reordering Where are the cache misses? How to identify hot field accesses at runtime? How to reorder the objects?

Where Are The Cache Misses? Heap structure: VM Objects Stack Older Generation Nursery Not to scale

Where Are The Cache Misses?

Where Are The Cache Misses? Two opportunities to reorder objects in the older generation Promote nursery objects Full heap collection

How to Find Hot Fields? Runtime info (intercept every read)? Compiler analysis? Runtime information + compiler analysis Key: Low overhead estimation

Which Classes Need Reordering? Step 1: Compiler analysis Excludes cold basic blocks Identifies field accesses Step 2: JIT adaptive sampling identifies hot methods Mark as hot field accesses in hot methods Key: Low overhead estimation

Example: Compiler Analysis Method Foo { Class A a; try { …=a.b; … } catch(Exception e){ …a.c Hot BB Collect access info Compiler Compiler Cold BB Ignore Access List: 1. A.b 2. …. ….

Example: Adaptive Sampling Method Foo { Class A a; try { …=a.b; … } catch(Exception e){ …a.c Adaptive Sampling Foo Accesses: 1. A.b 2. …. …. Foo is hot A.b is hot A A’s type information c b ….. c b B

Copying of Linked Objects Type Information 1 3 4 1 1 4 4 2 2 3 3 6 6 7 7 5 5 Online Object Reordering Hot space Cold space

OOR System Overview Hot Methods Source Code Look Up Access Info Database Adaptive Sampling Adaptive Sampling Baseline Compiler Optimizing Compiler Optimizing Compiler Adds Entries Register Hot Field Accesses GC: Copies Objects GC: Copies Objects Executing Code Affects Locality Improves Locality Advice Input/Output JikesRVM component OOR addition

Outline Motivation Online Object Reordering Methodology Experimental Results Conclusion

Methodology: Virtual Machine Jikes RVM VM written in Java High performance Timer based adaptive sampling Dynamic optimization Experiment setup Pseudo-adaptive 2nd iteration [Eeckhout et al.]

Methodology: Memory Management Memory Management Toolkit (MMTk): Allocators and garbage collectors Multi-space heap Boot image Large object space (LOS) Immortal space Experiment setup Generational copying GC with 4M bounded nursery

Overhead: OOR Analysis Only Benchmark Base Execution Time (sec) w/ only OOR Analysis (sec) Overhead jess 4.39 4.43 0.84% jack 5.79 5.82 0.57% raytrace 4.63 4.61 -0.59% mtrt 4.95 4.99 0.70% javac 12.83 12.70 -1.05% compress 8.56 8.54 0.20% pseudojbb 13.39 13.43 0.36% db 18.88 -0.03% antlr 0.94 0.91 -2.90% hsqldb 160.56 158.46 -1.30% ipsixql 41.62 42.43 1.93% jython 37.71 37.16 -1.44% ps-fun 129.24 128.04 -1.03% Mean -0.19%

Detailed Experiments Separate application and GC time Vary thresholds for method heat Vary thresholds for cold basic blocks Three architectures x86, AMD, PowerPC x86 Performance counter: DL1, trace cache, L2, DTLB, ITLB

Discussion What will be the result of cache affinity on multicore systems ? Will memory affinity benefit the system more ? Do we need some modification in algorithm to increase cache affinity per core ? Will it significantly improve performance ?

Performance Implications of Cache Affinity on Multicore Processors Vahid Kazempour, Alexandra Fedorova and Pouya Alagheband EuroPar 2008 “We hypothesized that cache affinity does not affect performance on multicore processors: on multicore uniprocessors — because reloading the L1 cache state is cheap, and on multicore multiprocessors – because L2 cache affinity is generally low due to cache sharing.” “Even though upper-bound performance improvements from exploiting cache affinity on multicore multiprocessors are lower than on unicore multiprocessors, they are still significant: 11% on average and 27% maximum. This merits consideration of affinity awareness on multicore multiprocessors.”

Performance javac

Performance db

Performance jython Any static ordering leaves you vulnerable to pathological cases.

Phase Changes

Conclusion Static traversal orders have up to 25% variation OOR improves or matches best static ordering OOR has very low overhead Past predicts future

Discussion In experiments OOR gives performance benefit of 10-40% compared to full heap mark sweep collector. Is the comparison valid – As OOR runs with generational copy collector whereas mark sweep collects full heap at a time. How much benefit we are getting through generational version of copy collector ? In Myths and Realities paper we see GenMS is 30-45% better than full heap MS.

Questions? Thank you!