2. OOPSLA 2003 Mostly Concurrent Garbage Collection Revisited Katherine Barabash - IBM Haifa Research Lab. Israel Yoav Ossia - IBM Haifa Research Lab. Israel Erez Petrank - Technion. Israel

3. IBM Labs in Haifa OOPSLA 2003 2 Outline  The mostly concurrent garbage collection (GC)  Internals of the collector  Write barrier and card table  Incremental collection  Our two improvements  And their implications on performance  Results  Conclusions

4. IBM Labs in Haifa OOPSLA 2003 3 Mark Sweep Stop-The-World (STW) Garbage Collection  The basic method  Mark all objects that are reachable from roots  Sweep - reclaim all unmarked objects  Done when Java mutation is suspended (STW)  Pause time - the length of the STW phase  Motivation for the mostly concurrent GC  Reduce the pause time at acceptable throughput hit

5. IBM Labs in Haifa OOPSLA 2003 4 Mostly Concurrent GC - The Basic Method  Perform marking concurrently with Java mutation  Traditionally done by a separate thread  While concurrent marking is active, record changes in objects  Otherwise…  When marking terminates do a short STW phase  Re-trace from  Roots  Marked objects that were not traced yet  Marked and changed objects  Sweep

6. IBM Labs in Haifa OOPSLA 2003 5 Mostly Concurrent GC - Perspective  Related Work  Steele and Dijkstra et al - Concurrent GC. 1976  Baker - Incremental collection. 1977  Boehm at al - Mostly concurrent collection. 1991  Printezis and Detlefs - Mostly concurrent and generational GC. 2000  Many others…  Ossia at al - Parallel, incremental and concurrent GC. 2002  Status  Collector well accepted both in academic research and industry  Used in many production JVMs: IBM, Sun, BEA JRockit

7. IBM Labs in Haifa OOPSLA 2003 6 Outline  The mostly concurrent garbage collection (GC)  Internals of the collector  Write Barrier and card table  Incremental collection  Our two improvements  And their implications on performance  Results  Conclusions

8. IBM Labs in Haifa OOPSLA 2003 7 The Write Barrier and Object “cleaning” interaction Tracer: Marks and traces Java Mutator: Modifies Blue and Green objects Write barrier on objects Tracer: Traces rest of graph Tracer: Clean blue object

9. IBM Labs in Haifa OOPSLA 2003 8 Mostly Concurrent GC – Card Cleaning  Heap is logically divided into cards  A card table is used, with a byte entry per each heap card  A card-marking write barrier  Whenever a reference field is modified, dirty the card table entry of the modified object  Card cleaning  Clean dirty mark  Retrace all marked objects on card  Card cleaning can be done concurrently  While this is done, more cards will be dirtied  Additional STW card cleaning phase must be done

10. IBM Labs in Haifa OOPSLA 2003 9 Incremental Mostly Concurrent GC  Marking done by the Java mutator threads  When allocating  Tracing Rate (TR) - configurable by user  The ratio between requested allocation size and required tracing work  Per every allocation request of K bytes, trace K* TR bytes of objects  Allocation rate of application & tracing rate of collector imply CPU percentage dedicated to the concurrent collection  Starting the concurrent collector  Must be done on time, to complete tracing when the heap is exhausted

11. IBM Labs in Haifa OOPSLA 2003 10 Concurrent Behavior Patterns  Higher tracing rate implies shorter concurrent cycle with smaller CPU share for Java  Numbers below refer to SPECjbb  STW GC  100% CPU for Java mutation  Mostly Concurrent  Tracing Rate 8  28% CPU for Java mutation  Mostly Concurrent  Tracing Rate 1  72% CPU for Java mutation CPU Utilization Time Java mutation Incremental Tracing Parallel STW

12. IBM Labs in Haifa OOPSLA 2003 11 Mostly concurrent GC – Summary of the Base Algorithm  Fast card marking write barrier always active (JITed)  Kickoff concurrent tracing when free space reached kickoff point  Reset the card table  Trace (incrementally) all objects reachable from roots  Do a single concurrent card cleaning pass on the card table  Initiate final (short) STW phase  Trace again the roots for new objects  Do another card cleaning pass  Trace all newly marked objects  Sweep

13. IBM Labs in Haifa OOPSLA 2003 12 Outline  The mostly concurrent garbage collection (GC)  Internals of the collector  Write Barrier and card table  Incremental collection  Our two improvements  And their implications on performance  Results  Conclusions

14. IBM Labs in Haifa OOPSLA 2003 13 The Repetitive Work Problem  Observations  Suppose a card is dirtied while concurrent marking is executing  Newly reached objects in this card are marked and traced by the collector  All these objects will later be traced again in the card cleaning phase  Outcome: repeated tracing  Improvement: Don’t trace through dirty cards

15. IBM Labs in Haifa OOPSLA 2003 14 Don’t Trace Through Dirty Cards  If an object resides in a dirty card, omit its tracing  Only a single tracing will be done, in the concurrent card cleaning phase  Advantages  Less marking work  Reduced floating garbage  More later…  Reduced cache miss rate  More later…  Thus, substantial throughput improvements  Disadvantage  Increased pause time Java mutation Concurrent tracing STW tracing Concurrent card cleaning STW card cleaning Base Method Don’t trace dirties

16. IBM Labs in Haifa OOPSLA 2003 15 Timing of Card Dirtying  Observations  Card (and object) dirtying indicates that previous tracing may have been insufficient  New objects may be reachable only from the dirtied object  Dirtying information is needed only if tracing was already done  Prior to tracing, card dirtying is irrelevant  Improvement: Undirdy cards with no traced objects  Undirtying via scanning  Undirtying via local allocation caches

17. IBM Labs in Haifa OOPSLA 2003 16 Undirtying via Scanning  Undirtying can be done periodically on the whole card table  An indication of “traced” cards is needed  Mark bit vector  A “traced” card table (first traced object marks the card as traced)  Method: scan the card table and undirty all cards with no traced objects  Very effective in undirtying cards (cuts cleaning by 65%)  Some extra cost of card table scan  Should be done frequently  Catch these cards before any marking or tracing occurs!

18. IBM Labs in Haifa OOPSLA 2003 17 Undirtying via Local Allocation Caches  Local allocation caches are used by most modern JVMs  Most cards of active caches are dirty  Objects usually have write barriers while (and shortly after) initializing  Objects in active cache are (usually) not traced before the cache is replaced  If no tracing in the active cache is guarantied, we can undirty its cards  Method: cooperation between allocators and concurrent tracers  Allocator (when replacing a local cache):  Undirty all the cache’s inner cards  Mark all cards as “traceable”  Take a new cache and mark all its cards as “untraceable”  Concurrent tracer  Defer tracing of objects in “untraceable” cards “for a while”  BTW, this hardly ever happens  Cuts the amount of dirty cards by more than 35%, at no cost

19. IBM Labs in Haifa OOPSLA 2003 18 Undirdy Cards with No Traced Objects  Advantages  Without “Don’t trace through dirty cards” – less work  With it, reduces the STW card cleaning significantly Java mutation Concurrent tracing STW tracing Concurrent card cleaning STW card cleaning Base Method Don’t trace dirties Don’t trace dirties + Undirty

20. IBM Labs in Haifa OOPSLA 2003 19 Characteristics of Dirty Cards  We believe that a recently dirtied card is good indication for more modification of objects in the near future  Change of references  Other writing activities  Indication applied to all the objects in the card  A recently dirtied card is probably hot and active  But we don’t trace through dirty cards!  By the time we get to clean them they will probably become more stable and colder

21. IBM Labs in Haifa OOPSLA 2003 20 Reduced Floating Garbage  Floating garbage is created when tracing is done before the object modification  Don’t trace through dirty cards!  Will probably defer the tracing until the card gets stable  Objects are no longer modified  No floating garbage will be created as a result of this late tracing

22. IBM Labs in Haifa OOPSLA 2003 21 Reduced Cache Miss Rate  Reducing the tracing work affects the cache miss rate  As tracing the object graph intensifies cache capacity misses  But also cache coherency misses are reduced  A write barriered card (hot and active) is probably modified by Java mutators  If a concurrent tracer scans objects on such card, it will suffer coherency misses  Don’t trace through dirty cards!  Deferring the tracing of these objects to the card cleaning phase reduces cache coherency misses  Our improved collector reduces L2 cache miss rate by 6.4%  Out of which 3.7% is reduction in cache coherency misses

23. IBM Labs in Haifa OOPSLA 2003 22 Outline  The mostly concurrent garbage collection (GC)  Internals of the collector  Write Barrier and card table  Incremental collection  Our two improvements  And their implications on performance  Results  Conclusions

24. IBM Labs in Haifa OOPSLA 2003 23 Implementation and Tests  Implementation  On top of the mostly concurrent collector that is part of the IBM production JVM 1.4.0.  Platforms  Tested on both an IBM 6-way pSeries server and an IBM 4-way Netfinity server  Benchmarks  The SPECjbb2000 benchmark and the SPECjvm98 benchmark suite  Measurements  Performance of the base collector Vs. the improved version  The effect of each improvement separately, and more…

25. IBM Labs in Haifa OOPSLA 2003 24 Results - Throughput Improvement  SPECjbb. 6-way PPC. Heap size 448 MB  26.7% improvement (in tracing rate 1)

26. IBM Labs in Haifa OOPSLA 2003 25 Results - Floating Garbage Reduction  SPECjbb. 6-way PPC. Heap size 448 MB  13.4% improvement in heap residency (in tracing rate 1)  Almost all floating garbage eliminated

27. IBM Labs in Haifa OOPSLA 2003 26 Results - Pause Time Reduction  SPECjbb. 6-way PPC. Heap size 448 MB  33.3% improvement in average pause  36.4% improvement in max pause

28. IBM Labs in Haifa OOPSLA 2003 27 Conclusions  Introducing two improvements to the mostly concurrent GC  Reduces repetitive GC work (don’t trace through dirty cards)  Reduces number of dirty cards (undirty cards with no traced objects)  Substantial improvement of the mostly concurrent GC  Improved throughput by 26%  Almost eliminated floating garbage (heap residency reduced by 13%)  Reduced average pause time by 33%  Additional effects of not tracing into dirty cards  Reduced floating garbage  Reduced cache miss rate  The improved algorithm has been incorporated into IBM's production JVM

29. IBM Labs in Haifa OOPSLA 2003 28 End

30. IBM Labs in Haifa OOPSLA 2003 29 Analyzing the Performance of Lower Tracing Rate  Throughput hit rate  Relative to MS STW GC  Java utilization  Relative to MS STW GC  Live Rate  Relative to heap size  Floating Garbage  Marked objects that become unreachable before the STW phase  More objects to trace. Less free space (more GCs)  Card cleaning rate  Relative to total number of cards  More work. Longer final STW phase

31. IBM Labs in Haifa OOPSLA 2003 30 Results - Throughput Improvement  SPECjbb. 6-way PPC. Heap size 448 MB  26.7% improvement in tracing rate 1

32. IBM Labs in Haifa OOPSLA 2003 31 Results - Floating Garbage Reduction  SPECjbb. 6-way PPC. Heap size 448 MB  13.4% improvement in tracing rate 1

33. IBM Labs in Haifa OOPSLA 2003 32 Results - Average Pause Time  SPECjbb. 6-way PPC. Heap size 448 MB

34. IBM Labs in Haifa OOPSLA 2003 33 Throughput Improvement for All Tracing Rates

35. IBM Labs in Haifa OOPSLA 2003 34 Heap Residency Reduction for All Tracing Rates

36. IBM Labs in Haifa OOPSLA 2003 35

37. IBM Labs in Haifa OOPSLA 2003 36 Reduced floating garbage  Potential Floating garbage root – reachable object that de-reference its sub-graph and thus make it unreachable  To become a floating garbage root, it must first be traced and then have a write barrier  We believe that a freshly dirty card is good indication for more write barriers  Deferring the tracing into a dirty card will defer the tracing to after the write barriers