Concurrent Mark-Sweep Presented by Eyal Dushkin GC Seminar, Tel-Aviv University

Outline of talk  Introduction  Tricolour Invariants (Reminder)  Mostly Concurrent Mark & Sweep  Concurrent Marking and Sweeping  On-the-fly marking  Abstract Concurrent Collection  Summary

Introduction  During the course we have seen different types of garbage collection algorithms, particularly of types: STW (stop the world), Incremental and Parallel.  Each of which has its own tradeoffs with respect to:  Throughput  Pause time  Space overhead  Scalability and portability  And more …  We’ll present the concurrent mark-sweep paradigm.

Incremental Collection ( Roman Kecher Lec.09 ) 4  First, on a Uniprocessor Time

Incremental Collection ( Roman Kecher Lec.09 ) 5  On a Multiprocessor:

Incremental Collection ( Roman Kecher Lec.09 ) 6  On a Multiprocessor:  Can also be parallelized:

Mostly Concurrent Collection ( Roman Kecher Lec.09 ) 7

8  Can also be incremental:

Concurrent Collection ( Roman Kecher Lec.09 ) 9

10  Can also be incremental:

Our Focus

Tricolour Abstraction – Reminder ( Pavel Lec.02 )  A convenient way to describe object states:  Initially, every node is white.  When a node is first encountered (during tracing), it is colored grey.  When it has been scanned, and its children identified, it is colored black.  At the end of each sweep, no references from black to white objects.  All the white objects are unreachable = garbage.

Grey Wavefront ( Roman Kecher Lec.09 ) 13 Roots

Grey Wavefront ( Roman Kecher Lec.09 ) 14 Roots

Grey Wavefront ( Roman Kecher Lec.09 ) 15 Roots

Grey Wavefront ( Roman Kecher Lec.09 ) 16 Roots

Grey Wavefront ( Roman Kecher Lec.09 ) 17 Roots

Grey Wavefront ( Roman Kecher Lec.09 ) 18 Roots

Grey Wavefront ( Roman Kecher Lec.09 ) 19 Roots

Grey Wavefront ( Roman Kecher Lec.09 ) 20 Roots

Grey Wavefront ( Roman Kecher Lec.09 ) 21 Roots

The Lost Object Problem  Direct and transitive.

Lost Object – Direct ( Roman Kecher Lec.09 ) 23-Dec-1423  D1 // Insertion  Write(X, b, Read(Y, a)) Roots XY a Z

Lost Object – Direct ( Roman Kecher Lec.09 ) 23-Dec-1424  D1 // Insertion  Write(X, b, Read(Y, a)) Roots XY a Z b

Lost Object – Direct ( Roman Kecher Lec.09 ) 23-Dec-1425  D2 // Deletion:  Write(Y, a, null) Roots XY a Z b

Lost Object – Direct ( Roman Kecher Lec.09 ) 23-Dec-1426  D2 // Deletion:  Write(Y, a, null) Roots XY Z b

Lost Object – Direct ( Roman Kecher Lec.09 ) 27  D3:  scan(Y) Roots XY Z b

Lost Object – Direct ( Roman Kecher Lec.09 ) 28 Roots XY Z b

Lost Object – Transitive ( Roman Kecher Lec.09 ) 29 Roots PQ c RS d

Lost Object – Transitive ( Roman Kecher Lec.09 ) 30  T1 // Insertion:  Write(P, e, Read(R, d)) Roots PQ c RS d e

Lost Object – Transitive ( Roman Kecher Lec.09 ) 31  T2 // Deletion:  Write(Q, c, null) Roots PQ RS d e

Lost Object – Transitive ( Roman Kecher Lec.09 ) 32  T3:  scan(Q) Roots PQ RS d e

Lost Object – Transitive ( Roman Kecher Lec.09 ) 33 Roots PQ RS d e

Tricolour Invariants – Reminder  The strong tricolour invariant: There are no pointers from black objects to white objects. root YX Z

Tricolour Invariants – Reminder  The strong tricolour invariant: There are no pointers from black objects to white objects.  The weak tricolour invariant: All white objects pointed to by a black object are reachable from some grey object through a chain of white objects. root YX Z

Barriers for Grey Mutator Weak or Strong?

Barriers for Black Mutator Weak or Strong?

Barriers – to be continued …

Introduction Tricolour Invariants (Reminder)  Mostly Concurrent Mark & Sweep  Concurrent Marking and Sweeping  On-the-fly marking  Abstract Concurrent Collection  Summary

Mostly Concurrent Mark & Sweep Initialisation:  Triggering is a critical decision.  Once a collection cycle has started, we would like to:  Minimize its impact on mutator throughput (efficiency).  Complete the cycle before mutator exhausts memory (pause time).

Mostly Concurrent Mark & Sweep Mutator Allocation: New(): CollectEnough() ref  allocate() if ref = null errror "out of memory" return ref atomic collectEnough(): while behind() if not markSome() return

Mostly Concurrent Mark & Sweep Let’s dive deep into the markSome method: markSome(): if isEmpty(worklist) scan(Roots) if isEmpty(worklist) sweep() return false ref  remove(worklist) scan(ref) return true

Mostly Concurrent Mark & Sweep Additional Methods: shade(ref): if not isMarked(ref) setMarked(ref) add(worklist, ref) scan(ref): for each fld in Pointers(ref) child  *fld if child ≠ null shade(child)

Mostly Concurrent Mark & Sweep Additional Methods: revert(ref): add(worklist, ref) isWhite(ref): return not isMarked(ref) isGrey(ref): return ref in worklist isBlack(ref): return isMarked(ref) && not isGrey(ref)

Mostly Concurrent Mark & Sweep Recall: Black and Grey mutators. markSome(): if isEmpty(worklist) scan(Roots) if isEmpty(worklist) sweep() return false ref  remove(worklist) scan(ref) return true

Mostly Concurrent Mark & Sweep Recall: Black and Grey mutators. markSome(): if isEmpty(worklist) scan(Roots) if isEmpty(worklist) sweep() return false ref  remove(worklist) scan(ref) return true

Mostly Concurrent Mark & Sweep Termination:  Depends on the mutator’s colour:  If black, no need to rescan.  If grey, need to rescan and sync with mutator’s thread.

Mostly Concurrent Mark & Sweep Allocation:  Depends on the mutator’s colour:  If black, could not be allocated as white under the strong invariant.  What about the weak invariant? Shall we allocate it grey?  If grey, then it’s non trivial:  White – might affect pause time and throughput.  Black – increase floating garbage quantity.  Yellow – hybrid method (Vechev et al [2006]).

Introduction Tricolour Invariants (Reminder) Mostly Concurrent Mark & Sweep  Concurrent Marking and Sweeping  On-the-fly marking  Abstract Concurrent Collection  Summary

Concurrent marking and sweeping  So far, marking-sweeping proceeding in series.  This of course is not always the case.  Lazy sweeping?

Recap: Lazy Sweeping ( Pavel Lec.02 ) atomic collect() markFromRoots() for each block in Blocks if not isMarked(block) add(blockAllocator, block) else add(reclaimList, block)

Recap: Lazy Sweeping ( Pavel Lec.02 ) lazySweep(sz): repeat block  nextBlock(relaimList, sz) if block != null sweep(start(block), end(block)) if spaceFound(block) return untill block = null allocSlow(sz) allocSlow(sz): block  allocateBlock() if block != null initalise(block, sz)

Concurrent marking and sweeping  We may also trigger concurrent sweeping from the previous marking phase.  Even if a gc cycle has been already running the next one …  How could we advise the marker and sweeper to distinguish between different white nodes?

Concurrent marking and sweeping  Example - Roots block

Concurrent marking and sweeping Roots block

Concurrent marking and sweeping Roots block

Concurrent marking and sweeping  Gc cycle is ended …  Now what? Roots block

Concurrent marking and sweeping How could we advise the marker and sweeper to distinguish between different white nodes?  We could use a new color: purple (Lamport [1976]).  But this means we need to: 1. Wait until all markers and sweeper have finished. 2. Changing colors – white to purple, black back to white. 3. Shade all roots. 4. Start the markers and sweepers again.

Concurrent marking and sweeping A Caveat: Changing the colors – white to purple, black back to white consumes time. Elegant solution (Lamport): don’t change colors, change perspective

Lamport Solution roots X YZ W

Lamport Solution roots X YZ W

Lamport Solution roots X YZ W

Lamport Solution roots X YZ W

Lamport Solution roots X YZ W

Lamport Solution roots X YZ W

Lamport Solution roots X YZ W

Lamport Solution roots X YZ W

Lamport Solution roots X YZ W

Introduction Tricolour Invariants (Reminder) Mostly Concurrent Mark & Sweep Concurrent Marking and Sweeping  On-the-fly marking  Abstract Concurrent Collection  Summary

On-the-fly marking (Heuristic)  So far we have assumed that the mutator threads are all stopped at once.  On-the-fly collection never stops the mutator threads at once.  An alternative is to sample the roots of each mutator thread separately, concurrently with other mutator threads working.  But it’s not as simple as it seems (if it seems) …

On-the-fly marking (Heuristic)  Usually mostly-concurrent collectors are using a black mutator with a deletion barrier.  Avoiding stw phase, some mutator's roots are neither black, nor grey.  As a result, we have black and white thread stacks. Thread Stack 1 x Thread Stack 2 Y

Barriers for Black Mutator (again) Weak or Strong? Merely Read Barrier?

On-the-fly marking (Heuristic)  Read barriers does not apply on stack operations, particularly the deletion barrier is not applied.  One can use stack operations barriers ( Visit Lecture no.6 by Oleg Dobkin ). Thread Stack 1 x Thread Stack 2 Y

On-the-fly marking (Heuristic)  Write(X,b,Y) Thread Stack 1 x Thread Stack 2 Y

On-the-fly marking (Heuristic)  Write(X,b,Y) Thread Stack 1 x Thread Stack 2 Y b

Doligez-Leroy-Gonthier [1993]  Let’s separate data allocated solely on a single thread and not shared with other threads.  In addition we have a global heap that allows sharing of objects among threads.  Private objects may move to global heap, if needed.

Doligez-Leroy-Gonthier [1993]

On-the-fly marking (Heuristic) This solves the first problem: Thread Stack 1 x Thread Stack 2 Y

On-the-fly marking (Heuristic) Thread Stack 1 x Thread Stack 2 Y

On-the-fly marking (Heuristic) Thread Stack 1 x Thread Stack 2 Y b

Doligez-Leroy-Gonthier [1993]

Introduction Tricolour Invariants (Reminder) Mostly Concurrent Mark & Sweep Concurrent Marking and Sweeping On-the-fly marking  Abstract Concurrent Collection  Summary

Abstract Concurrent Collection  We present an abstract framework for concurrent garbage collection [Vechev et al, 2005, 2006, 2007].  The correctness of a concurrent collector rely on the cooperation between the collector and the mutator.  Thus it’s reasonable to introduce a log event structure to append and synchronize the events of both.

Abstract Concurrent Collection - Preliminaries src old fld

Abstract Concurrent Collection - Preliminaries src new fld

Abstract Concurrent Collection - Preliminaries

Abstract Concurrent Collection - CMS

Abstract Tracing Algorithm – Reminder ( Yarden Lec.04 ) atomic collectTracing(): rootsTracing(W) scanTracing(W) sweepTracing() rootsTracing(R): for each fld in Roots ref ← *fld if ref ≠ null R ← R + [ref] scanTracing(W): while not isEmpty(W) src ← remove(W) (src) ← (src)+1 if (src) = 1 for each fld in Pointers(src) ref ← *fld if ref ≠ null W ← W + [ref]

Abstract Tracing Algorithm – Reminder ( Yarden Lec.04 )

The Concurrent Version shared log  () collectTracingInc(): atomic rootsTracing(W) log  () repeat scanTracingInc(W) addOrigins() until ? atomic addOrigins() scanTracingInc(W) sweepTracing() initialization termination

The Concurrent Version scanTracingInc(W): while not isEmpty(W) src  remove(W) if ρ (src) = 0 for each fld in Poinrters(src) atomic ref  *fld log  log T if ref ≠ null W  W + [ref] ρ (src)  ρ (src) + 1

The Concurrent Version addOrigins(): atomic origins  expose(log) for each src in origins W  W + [src] New(): ref  allocate() atomic ρ (ref)  0 log  log N return ref atomic Write(src, i, new): if src ≠ roots old  src[i] log  log W src[i]  new

Introduction Tricolour Invariants (Reminder) Mostly Concurrent Mark & Sweep Concurrent Marking and Sweeping On-the-fly marking Abstract Concurrent Collection  Summary

Introduction Tricolour Invariants (Reminder) Mostly Concurrent Mark & Sweep Concurrent Marking and Sweeping On-the-fly marking X Abstract Concurrent Collection  Summary

Discussion  What are CMS advantages?  Minimize Pause Time/Low Latency  Shared Resources between mutator and collector  What are CMS downsides?  Fragmentation  Promotion Failures  CPU Intensive  When would you use CMS?  Short GC times  Large sets of living objects

CMS in Practice  Is being used in JVM (not by default)  -XX:+UseConcMarkSweepGC  It has several flags, I (we) can now understand:  -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode  In practice, the vast majority of CMS are an implementation of mostly-concurrent algorithms with small stw phases.  It does minimize pause time.

CMS in Practice

 Additional nice metrics and benchmarks: other-java-7-garbage-collectors/

Summary  We’ve reviewed CMS algorithm –  Mostly concurrent  Mark and Sweep concurrently  On the fly  No free-lunch  It minimizes pause time, but at the expense of throughput and fragmentations.

Thank You! Questions and Feedback?